Warehousing transfer robot based on depth machine vision technology and transfer method
By integrating deep machine vision technology and multi-sensor collaborative modules in storage equipment, combined with mechatronic and hydraulic integrated systems, the shortcomings of existing equipment in identification accuracy, structural flexibility and automation levels are solved, and efficient and accurate cargo handling and intelligent operation are achieved.
Patent Information
- Application Number
- CN202510552336.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-29
- Publication Date
- 2025-06-20
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing smart warehousing equipment has problems such as insufficient identification accuracy, insufficient structural flexibility, low level of path planning automation and limited multifunction collaborative operation capabilities in visual processing, mechanism design, motion control and hardware integration, making it difficult to adapt to complex warehousing environments.
The warehousing and handling robot based on deep machine vision technology is adopted, and the omnidirectional chassis device, scissor lifting and transporting device, horizontal propulsion device, vertical lifting device, mechanical claw and multi-sensor collaboration module are integrated. The machine vision recognition module and multi-sensor collaboration module are used to achieve accurate identification and environmental perception of goods. Combined with advanced control algorithms and mechatronic and hydraulic integrated systems, automated and intelligent handling operations are achieved.
It realizes the accuracy of cargo segmentation and identification in complex contexts, omnidirectional mobility capabilities, improves the level of automation and intelligence of the robot, and enhances its efficient and stable operation capabilities in complex environments.
Smart Images
Figure CN120171964A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of intelligent logistics handling robots, and particularly relates to a warehousing handling robot and a handling method based on deep machine vision technology. Background Art
[0002] With the rapid development of the modern logistics industry, higher requirements are put forward for the intelligence and working efficiency of the warehousing and transportation links. However, traditional mechanical equipment such as forklifts or conveyor belts has a single function and is difficult to adapt to the warehousing environment with a wide variety of goods and complex and changeable scenarios. There are mainly limitations such as low handling efficiency, insufficient flexibility, and poor environmental adaptability.
[0003] In recent years, with the rise of artificial intelligence, deep learning and robotics technologies, deep vision technology has been gradually applied to the logistics field, providing technical support for the intelligent upgrade of warehousing transportation. In particular, the image processing technology based on convolutional neural network can accurately identify the type, position and placement state of objects, making it possible to realize autonomous and intelligent warehousing robots. In addition, the progress of mechatronics technology has also created conditions for the miniaturization, integration and multi-function of warehousing robot equipment.
[0004] However, most of the existing intelligent warehousing equipment on the market currently has the following deficiencies: (1) In terms of visual processing, there are problems such as insufficient recognition accuracy and poor algorithm adaptability. In particular, the accuracy of goods segmentation and recognition in complex backgrounds needs to be improved; (2) In terms of mechanism design, the existing equipment fails to fully consider the multi-task operation requirements in the structural design, and the design flexibility of some mechanical devices is insufficient, resulting in a single function of the mechanical equipment and poor coordination, affecting the overall efficiency and applicability; (3) In terms of motion control, the path planning and obstacle avoidance capabilities have not reached a high level of automation, and the accuracy of the underlying motion algorithm design is insufficient, restricting the stability and flexibility of the equipment; (4) In terms of hardware integration, there is a lack of effective design for multi-functional collaborative operation. For example, the integrated control ability of the hydraulic system and the grasping device is limited and cannot meet the refined and complex operation requirements. Summary of the Invention
[0005] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a warehousing handling robot and a handling method based on deep machine vision technology. This kind of warehousing handling robot integrates deep vision processing, automatic integrated object picking and handling functions. By integrating sensors, omnidirectional movement and control systems, it can efficiently and accurately complete warehousing tasks, which has a positive effect on the realization of intelligent warehousing.
[0006] To solve the above technical problems, the present invention adopts the following technical solutions: A warehousing handling robot based on deep machine vision technology, comprising a vehicle frame, on which an omnidirectional chassis device, a scissor lift material handling device and a horizontal propulsion device are sequentially arranged from bottom to top. A vertical lifting device is arranged on the horizontal propulsion device, and a mechanical claw is installed on the vertical lifting device. A mechanical system control module, a machine vision recognition module and a multi-sensor collaboration module are also fixed on the vehicle frame.
[0007] Preferably, the vehicle frame is built by multiple high-strength aluminum alloy profiles and right-angle profile connectors. The vehicle frame is the main frame structure of the warehousing handling robot, and the rest of the components need to be fixed on the vehicle frame.
[0008] Preferably, the machine vision recognition module uses an OpenMV 4PLUS intelligent vision camera. Its recognition tasks mainly include QR code recognition, color recognition, different object contour recognition, etc. The QR code recognition and color recognition tasks can directly call the relevant open-source programs of OpenMV for testing. For the recognition of different objects, machine learning methods need to be used. After collecting image data through the camera, preprocessing is carried out, and then the image is inferred through the trained model, and finally the object category is output.
[0009] The omnidirectional chassis device is located at the bottom of the vehicle frame, including omnidirectional wheels and chassis motors for driving the omnidirectional wheels. It can make the warehousing handling robot move freely on the ground and is the power driving mechanism of the robot.
[0010] Preferably, the omnidirectional chassis device includes four Mecanum wheels arranged on both sides and chassis motors for driving each Mecanum wheel. The Mecanum wheels are connected to the chassis motors through couplings. The chassis motors are DC coded motors, and the Mecanum wheels are fixed on the vehicle frame through a double-spring suspension damping mechanism.
[0011] More preferably, the first controller adjusts the speed of the chassis motor by means of the PID closed-loop control algorithm to cooperate with adjusting the PWM pulse duty cycle of the chassis motor driver. The chassis motor is a DC coded motor. Among them, the encoder real-time detects the actual speed y(t) of the motor and compares it with the target speed r(t) set in the program to obtain the speed error e(t)=r(t)−y(t). Then, the PID controller calculates the adjustment value u(t) according to the error, and its formula is: (1), Where, K p 、K i and K dThey are proportional, integral, and differential coefficients respectively. The PID controller calculates the adjustment amount u(t) according to the real-time error, which is used to change the duty cycle of the PWM signal, and then adjust the output voltage of the driver to achieve dynamic control of the motor speed.
[0012] A scissor-type lifting material transport device is arranged above the omnidirectional chassis device, and the scissor-type lifting material transport device lifts the goods to a suitable height and transports them. It includes a scissor-type bracket, and a conveyor belt mechanism is arranged above the scissor-type bracket. The bottom of the scissor-type bracket is fixed on the omnidirectional chassis device.
[0013] Preferably, the scissor-type bracket of the scissor-type lifting material transport device is a hydraulic scissor-type bracket driven by a hydraulic transmission system.
[0014] The scissor lift transport device can carry goods and is easy to adjust up and down to achieve the purpose of placing the target goods at the specified container height, and then use the storage conveyor belt mechanism above it to transport the target goods horizontally to the specified container.
[0015] A horizontal propulsion device is arranged at the top of the frame, and the horizontal propulsion device realizes horizontal forward and backward movement. The horizontal propulsion device includes two mutually parallel horizontal slide rails and a horizontal slider moving on the horizontal slide rails. The horizontal slider is driven by a horizontal motor through a horizontal gear rack mechanism. A load-bearing plate is arranged between the horizontal sliders, and a vertical lifting device is arranged on the load-bearing plate.
[0016] Preferably, the two horizontal motors of the horizontal propulsion device are stepper motors, and the horizontal motors are connected to the spur gears of the horizontal gear rack mechanism, converting the fixed-axis rotation of the horizontal motor into linear motion of the rack of the horizontal gear rack mechanism; the horizontal slider is fixed on the rack of the horizontal gear rack mechanism and driven by it to move back and forth in the horizontal slide rail; a bearing plate is connected and fixed between the two horizontal sliders, and the bearing plate is made of carbon fiber material.
[0017] The vertical lifting device realizes up and down movement in the vertical direction, and includes a vertical motor and a vertical gear rack mechanism cooperating with the vertical motor. A vertical guide rail slider mechanism is fixedly installed on the vertical gear rack mechanism, a machine vision recognition module is arranged on the vertical guide rail slider mechanism, and a mechanical claw is fixedly connected to the end of the vertical gear rack mechanism facing the ground.
[0018] Preferably, the vertical motor of the vertical lifting device is a DC motor, which is fixedly mounted on the bearing plate and connected to the spur gear of the vertical gear rack mechanism to convert the fixed-axis rotation of the vertical motor into reciprocating translational motion of the rack of the vertical gear rack mechanism; the vertical guide rail slider mechanism is mounted on the rack of the vertical gear rack mechanism to achieve lifting and lowering; the photoelectric sensor is fixed on the vertical gear rack mechanism and follows the up and down movement.
[0019] The horizontal propulsion device and the vertical lifting device are used in the picking stage. After reaching the target picking area, the horizontal propulsion device first moves horizontally back and forth to adjust the horizontal position of the vertical lifting device to a suitable position. Then, the vertical lifting device moves up and down at this horizontal position to a suitable vertical position for picking up goods.
[0020] The machine vision recognition module recognizes the goods model and monitors the environmental information around the machine. It is fixed below the carrier plate through the recognition structure fixing piece.
[0021] The multi-sensor collaboration module includes a grayscale sensor, an ultrasonic sensor, a lidar sensor, and a photoelectric sensor. The grayscale sensor is used to recognize the ground markings and is fixedly installed on the omnidirectional chassis device; the ultrasonic sensor is used to measure the distance to the goods and is installed above the robotic arm through the ultrasonic structure fixing piece; the lidar sensor is used to sense the surrounding working environment and is fixedly installed on the omnidirectional chassis device; the photoelectric sensor is used to recognize the goods information and is fixedly installed on the vertical rack and pinion mechanism.
[0022] Preferably, the multi-sensor collaboration module includes 4 five-channel grayscale sensors, which are fixed on the frame of the omnidirectional chassis device facing the forward direction through the grayscale bracket; there are 2 ultrasonic sensors, which are installed above the robotic arm through the ultrasonic structure fixing piece; there is 1 lidar sensor and 1 photoelectric sensor.
[0023] The mechanical system control module is the control core for the entire machine to operate autonomously, including a first controller and a second controller. The first controller is electrically connected to the omnidirectional chassis device, the scissor lift material handling device, and the multi-sensor collaboration module. The second controller is electrically connected to the horizontal propulsion device, the vertical lifting device, the machine vision recognition module, the multi-sensor collaboration module, and the robotic arm. The first controller is electrically connected to the second controller and the first controller controls the second controller.
[0024] Preferably, the first controller uses an Arduino Mega2560 single-chip microcomputer, and the second controller uses an Arduino UNO single-chip microcomputer.
[0025] The first controller in the mechanical system control module, the lidar sensor and the grayscale sensor in the multi-sensor collaboration module are fixedly installed on the frame of the omnidirectional chassis device, and the grayscale sensor is installed on the frame facing the traveling direction.
[0026] The handling method using this warehousing handling robot includes the following steps: S1: The warehousing handling robot is located at the starting position. After installing the power supply, the device is started; S2: The first controller in the mechanical system control module sends a command to the omnidirectional chassis device, and the storage and handling robot goes to the task area to obtain the task code. After arriving at the task area, the second controller in the mechanical system control module sends a command to the machine vision recognition module to make it scan the given task QR code. The second controller feeds back information and instructions to the first controller, and the first controller obtains the work task after analysis. S3: The first controller sends a command to the lidar sensor in the multi-sensor collaboration module to sense and feedback the surrounding working environment, and the first controller plans the best path to the target pickup area; S4: The first controller sends a command to the grayscale sensor in the multi-sensor coordination module to make it recognize and feedback the ground marking guide line, and the first controller sends a command to the omnidirectional chassis device to make it go to the target pickup area along the ground marking guide line; S5: After the storage and handling robot arrives at the target pickup area, the second controller sends a command to the machine vision recognition module to scan the code to identify whether it is the goods to be picked up; if the recognition result is the goods to be picked up, the second controller feeds back an information signal to the first controller, and the first controller sends a command to the ultrasonic sensor in the multi-sensor collaboration module, and the ultrasonic sensor feeds back a distance signal. The first controller controls the omnidirectional chassis device to move forward or backward according to the distance until the storage and handling robot is within the effective range of the goods to be picked up; if the recognition result is empty or the goods are not the task goods, it is fed back to the first controller, and the first controller controls the storage and handling robot to go to the next pickup point to scan the code again; S6: After the storage and handling robot is within the effective range of the goods to be picked up, the first controller sends a command to the second controller, and the second controller controls the horizontal propulsion device to work, and through the rotation of the horizontal motor, the horizontal gear rack mechanism and the horizontal guide rail slider mechanism are mechanically transmitted, until the horizontal propulsion device pushes the load-bearing plate above the horizontal guide rail slider mechanism to the top of the goods to be picked up; then the first controller sends a command to the second controller, and the second controller controls the vertical lifting device to work, and through the rotation of the vertical motor, the vertical gear rack mechanism and the vertical guide rail slider mechanism are mechanically transmitted, until the mechanical claw located at the end of the vertical lifting device moves to the effective range of the goods to be picked up; the first controller sends a command to the second controller, and the second controller sends a command to the mechanical claw, and the mechanical claw clamps the goods and places them on the conveyor belt mechanism of the scissor-type lifting material transport device, and feedbacks the signal after the action is completed; S7: After the first controller receives the signal indicating that the placement is completed, it sends an instruction to the omnidirectional chassis device. The warehousing and handling robot then moves to the inventory area. After reaching the designated inventory area, the first controller sends an instruction to the scissor lift and material handling device. The scissor support in the scissor lift and material handling device rises or falls under the drive of the hydraulic transmission system until the height of the material placement conveyor mechanism on the scissor lift and material handling device is the same as the height of the position of the goods cabinet to be loaded. The first controller sends an instruction to the material placement conveyor mechanism to drive the conveyor mechanism to transfer the goods into the goods cabinet to be loaded. After the action is completed, a feedback signal is sent. S8: After the first controller receives the signal indicating that the goods have been successfully stored, it sends an instruction to the omnidirectional chassis device. The warehousing and handling robot then returns to the starting position to prepare for the next round of work.
[0027] The present invention includes two stages: picking up goods and storing goods. Through the machine vision recognition module and various sensors, the accuracy of goods segmentation and recognition in a complex background can be achieved. The omnidirectional chassis device uses omnidirectional wheels to enable omnidirectional movement, realizing the picking up and orderly storage of goods. The beneficial effects are as follows: (1) Intelligent mechatronic integration system: By comprehensively applying multidisciplinary knowledge such as mechanical design, electrical and electronic technology, single-chip technology, and hydraulic transmission control, the mechanical structure, hydraulic system, and mechatronic control are combined with each other to achieve automatic integration of goods recognition, lifting, transportation, and sorting, greatly improving the automation and intelligence level of the machine. (2) Innovative design of the mechanical system: By combining knowledge in disciplines such as engineering mechanics, mechanical principles, mechanical vibration, engineering machinery chassis design, engineering materials, and mechanical manufacturing with the concept of mechanical innovative design, innovative design solutions for the walking system, steering system, and material selection of the robot chassis are developed, realizing comprehensive innovation in the robot system design and ensuring the efficient and stable operation of the robot in a complex environment. (3) Advanced algorithms for modern control: By combining relevant knowledge in fields such as mechanical control engineering, principles of automatic control, robot design, unmanned driving technology, industrial sensors, and computer vision, precise control of the robot is achieved through advanced control algorithms, and in-depth recognition and classification of various goods can be carried out. This enables the robot to efficiently execute tasks in a dynamic and complex warehousing environment. Description of the Drawings
[0028] Figure 1 is the three-dimensional axonometric drawing of the present invention; Figure 2 is Figure 1 the front view of Figure 3 is Figure 1 the top view of Figure 4 is Figure 1 the left view of Figure 5 It is a schematic structural diagram of an omnidirectional chassis device; Figure 6 It is a schematic structural diagram of a vertical lifting device; Figure 7 It is a top view structural diagram of a horizontal propulsion device; Figure 8 It is a left view structural diagram of a horizontal propulsion device; Figure 9 It is a schematic structural diagram of a mechanical claw; Figure 10 It is a schematic structural diagram of a scissor lift material handling device; Figure 11 It is a control schematic diagram of the present invention.
[0029] Reference numerals 1, vehicle frame; 2, omnidirectional chassis device; 3, vertical lifting device; 4, horizontal propulsion device; 5, mechanical claw; 6, scissor lift material handling device; 7, mechanical system control module; 8, machine vision recognition module; 9, multi-sensor collaboration module; 1-1, high-strength aluminum alloy profile; 1-2, right-angle profile connector; 2-1, chassis motor driver; 2-2, chassis motor; 2-3, double-spring suspension damping mechanism; 2-4, coupling; 2-5, omnidirectional wheel; 3-1, vertical motor driver; 3-2, vertical motor; 3-3, vertical gear-rack mechanism; 3-4, vertical guide rail slider mechanism; 3-5, recognition structure fixing part; 3-6, ultrasonic structure fixing part; 4-1, horizontal motor controller; 4-2, horizontal motor; 4-3, horizontal gear-rack mechanism; 4-4, horizontal guide rail slider mechanism; 4-5, bearing plate; 4-6, load-bearing strut; 4-7, bearing slider; 5-1, servo motor; 5-2, claw body connector; 5-3, copper column; 5-4, claw hook; 6-1, storage conveyor belt mechanism; 6-2, scissor support; 6-3, hydraulic transmission system; 6-4, lifting slider; 7-1, first controller; 7-2, second controller; 8, machine vision recognition module; 9-1, grayscale sensor; 9-2, ultrasonic sensor; 9-3, lidar sensor; 9-4, photoelectric sensor. Detailed implementation manners
[0030] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and implementation cases. It should be understood that the specific implementation cases described herein are only used to explain the present invention and are not used to limit the present invention.
[0031] As Figures 1-4As shown in the figure, a warehousing handling robot based on deep machine vision technology includes a vehicle frame 1. The vehicle frame 1 is sequentially provided with an omnidirectional chassis device 2, a scissor lift material handling device 6, and a horizontal propulsion device 4 from bottom to top. A vertical lifting device 3 is arranged on the horizontal propulsion device 4, and a mechanical claw 5 is installed on the vertical lifting device 3. A mechanical system control module 7, a machine vision recognition module 8, and a multi-sensor collaboration module 9 are also fixed on the vehicle frame 1. The vehicle frame 1 is built by multiple high-strength aluminum alloy profiles 1-1 and right-angle profile connectors 1-2.
[0032] As Figure 5 shown, the omnidirectional chassis device 2 is located at the bottommost of the vehicle frame 1 and includes omnidirectional wheels 2-5 and a chassis motor 2-2 for driving the omnidirectional wheels 2-5. In practical applications, Mecanum wheels can also be used. In this case, the omnidirectional chassis device 2 includes 4 Mecanum wheels arranged on both sides and a chassis motor 2-2 for driving each Mecanum wheel. The omnidirectional wheels 2-5 are connected to the chassis motor 2-2 through a coupling 2-4. The chassis motor 2-2 is a DC coded motor and is driven by a chassis motor driver 2-1. The omnidirectional wheels 2-5 are fixed on the vehicle frame 1 through a double-spring suspension damping mechanism 2-3.
[0033] In order to effectively reduce vibration, enable the robot to adapt to uneven road conditions, and improve the movement stability and accuracy, the omnidirectional wheels 2-5 are connected to the omnidirectional chassis device 2 through a double-spring suspension damping mechanism 2-3 and are connected to the chassis motor 2-2 through a coupling 2-4 to ensure the stable and efficient movement of the robot; the chassis motor 2-2 of each omnidirectional wheel 2-5 is controlled to rotate by a chassis motor driver 2-1, and the omnidirectional wheels 2-5 are connected to the omnidirectional chassis device 2 through a double-spring suspension damping mechanism 2-3.
[0034] In order to more precisely adjust the motor speed and enable the warehousing handling robot to exactly reach the specified position, the first controller 7-1 adjusts the speed of the chassis motor 2-2 by means of cooperating with the PID closed-loop control algorithm to adjust the PWM pulse duty ratio of the chassis motor 2-2 driver. The chassis motor 2-2 is a DC coded motor. Among them, the encoder real-time detects the actual speed y(t) of the motor and compares it with the target speed r(t) set in the program to obtain the speed error e(t)=r(t)−y(t). Then, the PID controller calculates the adjustment value u(t) according to the error, and its formula is: (1), where, K p , K i and K dThey are the proportional, integral, and derivative coefficients respectively. The PID controller calculates the adjustment amount u(t) based on the real-time error, which is used to change the duty cycle of the PWM signal, and then adjust the output voltage of the driver to achieve dynamic control of the motor speed.
[0035] As Figure 10 shown, a scissor lift and material handling device 6 is provided above the omnidirectional chassis device 2. The scissor lift and material handling device 6 lifts the goods to an appropriate height and transports them. It includes a scissor support 6-2. Above the scissor support 6-2, a storage conveyor mechanism 6-1 is provided. The bottom of the scissor support 6-2 is fixed to the omnidirectional chassis device 2. The scissor support 6-2 of the scissor lift and material handling device 6 is a hydraulically propelled telescopic support, driven by a hydraulic transmission system 6-3, and moves horizontally inwards or outwards through a lifting slider 6-4 movably connected to the storage conveyor mechanism 6-1 to achieve the rise or fall of the scissor support 6-2.
[0036] Among them, the storage conveyor mechanism 6-1 is provided with a motor and a transmission device for transporting the taken-out goods into the container; the hydraulic transmission system 6-3 is composed of components such as a telescopic hydraulic propulsion cylinder, a hydraulic pump, and a hydraulic valve body. By adjusting the electrical signal of the hydraulic pump through an electrical control system, the displacement of the hydraulic propulsion cylinder is controlled, and then the vertical movement of the storage conveyor mechanism 6-1 is achieved to meet the handling requirements at different heights. This scissor lift and material handling device 6 enables the robot to efficiently perform vertical handling and horizontal conveying of goods, enhancing its ability to adapt to various working environments.
[0037] As Figures 7-8 shown, a horizontal propulsion device 4 is provided at the topmost part of the vehicle frame 1. The horizontal propulsion device 4 realizes the forward and backward movement in the horizontal direction. It includes two mutually parallel horizontal slide rails and a horizontal slider moving on the horizontal slide rails. The horizontal slider is driven by a horizontal motor 4-2 through a horizontal gear and rack mechanism 4-3. A bearing plate 4-5 is provided between the horizontal sliders, and a vertical lifting device 3 is provided on the bearing plate 4-5.
[0038] The two horizontal motors 4-2 of the horizontal propulsion device 4 are stepper motors. The horizontal motors 4-2 are connected to the spur gears of the horizontal gear and rack mechanism 4-3, converting the fixed-axis rotation of the horizontal motors 4-2 into the linear motion of the rack of the horizontal gear and rack mechanism 4-3. The horizontal slider is fixed to the rack of the horizontal gear and rack mechanism 4-3 and moves forward and backward in the horizontal slide rail under its drive. A bearing plate 4-5 is fixedly connected between the two horizontal sliders, and the bearing plate 4-5 is made of carbon fiber material.
[0039] The stepping motor in the horizontal propulsion device 4 is controlled by the horizontal motor controller 4-1. The horizontal motor controller 4-1 is integrated through two core drive chips and an external expansion board, and can control the operation of two stepping motors simultaneously. The core chip uses the DRV8825 drive chip, which supports bipolar stepping motor drive. The horizontal gear-rack mechanism 4-3 in the horizontal propulsion device 4 consists of two spur gears and two racks. The spur gears are installed at the output end of the stepping motor, and drive the racks to achieve horizontal movement through gear-rack meshing transmission. The horizontal guide rail slider mechanism 4-4 includes two horizontally installed slide rails and four sliders, which are connected to the non-toothed end face of the rack through the sliders, converting the rotation of the motor into horizontal displacement. The carrier plate 4-5 in the horizontal propulsion device 4 is used for components such as the horizontal motor 4-2 and the vertical motor 3-2, and its load-bearing capacity is crucial. To enhance the load-bearing performance, it is made of carbon fiber material, and a carrier slider 4-7 and a load-bearing strut 4-6 are provided at the center under the carrier plate 4-5. The four rollers of the carrier slider 4-7 are slidably matched with the strut slots, effectively preventing the carrier plate 4-5 from bending or breaking, and ensuring the stable operation of each component of the robot.
[0040] As Figure 6 shown, the vertical lifting device 3 realizes vertical up and down movement, and includes a vertical motor 3-2 and a vertical gear-rack mechanism 3-3 that cooperates with the vertical motor 3-2. A vertical guide rail slider mechanism 3-4 is fixedly installed on the vertical gear-rack mechanism 3-3, and a machine vision recognition module 8 is provided on the vertical guide rail slider mechanism 3-4. A mechanical claw 5 is fixedly connected to the end of the vertical gear-rack mechanism 3-3 facing the ground.
[0041] The vertical motor 3-2 of the vertical lifting device 3 is a DC motor. The vertical motor 3-2 is fixedly installed on the carrier plate 4-5. The vertical motor 3-2 is connected to the spur gear of the vertical gear-rack mechanism 3-3, converting the fixed-axis rotation of the vertical motor 3-2 into the reciprocating linear motion of the rack of the vertical gear-rack mechanism 3-3. The vertical guide rail slider mechanism 3-4 is installed on the rack of the vertical gear-rack mechanism 3-3 to realize lifting and lowering. The photoelectric sensor 9-4 is fixed on the vertical gear-rack mechanism 3-3 and moves up and down accordingly.
[0042] The vertical lifting device 3 consists of a vertical motor driver 3-1, a DC motor, a vertical gear-rack mechanism 3-3, a vertical guide rail slider mechanism 3-4, etc. Among them, the vertical gear-rack mechanism 3-3 includes a spur gear and a rack. The spur gear is installed at the output end of the DC motor and drives the rack to move vertically through meshing transmission. The vertical guide rail slider mechanism 3-4 consists of vertically installed slide rails and two sliders. The upper surface of the slider is connected to the non-toothed end face of the rack, converting the fixed-axis rotation of the DC motor into the reciprocating linear motion of the rack, thereby realizing the lifting and lowering functions of the robot gripping device.
[0043] The machine vision recognition module 8 recognizes the goods model and monitors the environmental information around the machine. It is fixed below the carrier plate 4-5 through the recognition structure fixing part 3-5. The machine vision recognition module 8 uses an OpenMV 4PLUS intelligent vision camera, mainly performing tasks such as QR code recognition, color recognition, and object contour recognition. The QR code and color recognition tasks are directly implemented through the OpenMV open-source program, while object recognition is carried out through machine learning methods. After the camera collects image data, it is preprocessed and predicted through a trained model, and finally the object category is output.
[0044] The multi-sensor collaboration module 9 includes a grayscale sensor 9-1, an ultrasonic sensor 9-2, a lidar sensor 9-3, and a photoelectric sensor 9-4. The grayscale sensor 9-1 is used to recognize the ground markings and is fixedly installed on the omnidirectional chassis device 2; the ultrasonic sensor 9-2 is used to measure the distance to the goods and is installed above the robotic arm 5 through the ultrasonic structure fixing part 3-6; the lidar sensor 9-3 is used to sense the surrounding working environment and is fixedly installed on the omnidirectional chassis device 2; the photoelectric sensor 9-4 is used to recognize the goods information and is fixedly installed on the vertical gear-rack mechanism.
[0045] The multi-sensor collaboration module 9 includes 4 five-way grayscale sensors 9-1, which are fixed on the frame 1 of the omnidirectional chassis device 2 facing the forward direction through the grayscale brackets; there are 2 ultrasonic sensors 9-2, which are installed above the robotic arm 5 through the ultrasonic structure fixing part 3-6; there is 1 lidar sensor 9-3 and 1 photoelectric sensor 9-4.
[0046] The mechanical system control module 7 includes a first controller 7-1 and a second controller 7-2. The first controller 7-1 is electrically connected to the omnidirectional chassis device 2, the scissor lift material handling device 6, and the multi-sensor collaboration module 9. The second controller 7-2 is electrically connected to the horizontal propulsion device 4, the vertical lifting device 3, the machine vision recognition module 8, the multi-sensor collaboration module 9, and the robotic arm 5. The first controller 7-1 is electrically connected to the second controller 7-2 and the first controller 7-1 controls the second controller 7-2.
[0047] The first controller 7-1 uses an Arduino Mega2560 single-chip microcomputer, and the second controller 7-2 uses an Arduino UNO single-chip microcomputer. The first controller 7-1 in the mechanical system control module 7, the lidar sensor 9-3 and the grayscale sensor 9-1 in the multi-sensor cooperation module 9 are fixedly installed on the frame 1 of the omnidirectional chassis device 2, and the grayscale sensor 9-1 is installed on the frame 1 facing the traveling direction. Among them, the first controller 7-1 serves as the main control system, responsible for controlling the chassis motor driver 2-1 on the robot omnidirectional chassis device 2, the four five-way grayscale sensors 9-1, the two ultrasonic sensors 9-2 and the lidar sensor 9-3 in the multi-sensor cooperation module 9, and is also responsible for serial communication with the intelligent camera OpenMV4PLUS of the intelligent vision recognition module and the second controller 7-2; the second controller 7-2 serves as the secondary controller, responsible for controlling the vertical motor driver 3-1 on the vertical lifting device 3, the stepper motor driver on the horizontal propulsion device 4 and the rotation of the mechanical claw 5.
[0048] As Figure 9 shown, the mechanical claw 5 is composed of two servo motors 5-1, a claw body connecting piece 5-2, a copper column 5-3 and a replaceable claw hook 5-4. The mechanical claw 5 uses a two-degree-of-freedom servo motor 5-1. By adjusting the PWM pulse signal through the second controller 7-2, the action of the servo motor 5-1 is controlled, so as to realize the clamping function of the goods. In order to adapt to goods of different shapes, the claw hook 5-4 on the mechanical claw 5 can be replaced as needed to ensure that the robot can flexibly handle various types of goods.
[0049] In the present invention, the production and processing methods of some important components are as follows: the main bodies of components such as the double-spring suspension damping mechanism 2-3 and the motor bracket are modeled by Solid works software and then processed by 3D printing technology. For some threaded holes above them, a drill press is used to drill and then thread tapping is carried out; the gears and racks used in the vertical lifting device 3 and the horizontal propulsion device 4 are designed by Solid works and Auto CAD software and then processed by a gear shaper and a milling machine, and their materials are nylon resin; various structural fixing parts, claw body connecting pieces 5-2 and other components are all processed by 3D printing technology; the bearing plate 4-5 is processed by laser cutting through CAD drawings; the bearing slider 4-7 is a non-standard part, and its structural design is completed by Solid works software. The processing methods mainly include turning, milling, wire cutting and grinding; the core expansion board of the horizontal motor controller 4-1 is designed and manufactured by LCS EDA and then integrated by electronic welding; the remaining non-standard parts used will not be described in detail.
[0050] The handling method using this warehousing and handling robot is as Figure 11 shown, including the following steps: S1: The warehousing and handling robot is located at the starting position. After installing the power supply, start the device; S2: The first controller 7-1 in the mechanical system control module 7 sends an instruction to the omnidirectional chassis device 2. The warehousing and handling robot goes to the task area to obtain the task code. After arriving at the task area, the second controller 7-2 in the mechanical system control module 7 sends an instruction to the machine vision recognition module 8 to make it scan the given task two-dimensional code. The second controller 7-2 feeds back an information instruction to the first controller 7-1. After analysis, the first controller 7-1 learns the work task; S3: The first controller 7-1 sends an instruction to the lidar sensor 9-3 in the multi-sensor collaboration module 9 to make it sense and feedback the surrounding working environment. The first controller 7-1 plans the best path to the target picking area; S4: The first controller 7-1 sends an instruction to the grayscale sensor 9-1 in the multi-sensor collaboration module 9 to make it identify and feedback the ground marking guiding line. The first controller 7-1 sends an instruction to the omnidirectional chassis device 2 to make it go to the target picking area along the ground marking guiding line; S5: After the warehousing and handling robot arrives at the target picking area, the second controller 7-2 sends an instruction to the machine vision recognition module 8 for code scanning detection to identify whether it is the goods to be picked. If the recognition result is the goods to be picked, the second controller 7-2 feeds back an information signal to the first controller 7-1. The first controller 7-1 sends an instruction to the ultrasonic sensor 9-2 in the multi-sensor collaboration module 9. The ultrasonic sensor 9-2 feeds back a distance signal. The first controller 7-1 controls the omnidirectional chassis device 2 to move forward or backward according to the distance until the warehousing and handling robot is within the effective range of the goods to be picked. If the recognition result is empty or the goods are not the task goods, it is fed back to the first controller 7-1. The first controller 7-1 controls the warehousing and handling robot to go to the next picking point to perform code scanning detection again; S6: After the warehousing handling robot is within the effective range from the goods to be picked up, the first controller 7-1 sends an instruction to the second controller 7-2. The second controller 7-2 controls the horizontal propulsion device 4 to work. Through the rotation of the horizontal motor 4-2, mechanical transmission occurs between the horizontal gear-rack mechanism 4-3 and the horizontal guide-rail slider mechanism 4-4 until the horizontal propulsion device 4 pushes the carrier plate 4-5 above the horizontal guide-rail slider mechanism 4-4 directly above the goods to be picked up. Subsequently, the first controller 7-1 sends an instruction to the second controller 7-2. The second controller 7-2 controls the vertical lifting device to work. Through the rotation of the vertical motor 3-2, mechanical transmission occurs between the vertical gear-rack mechanism 3-3 and the vertical guide-rail slider mechanism 3-4 until the mechanical claw 5 at the end of the vertical lifting device moves within the effective range from the goods to be picked up. The first controller 7-1 sends an instruction to the second controller 7-2. The second controller 7-2 sends an instruction to the mechanical claw 5. The mechanical claw 5 grabs the goods and places them on the goods placement conveyor mechanism 6-1 of the scissor lift conveying device 6. After the action is completed, a feedback signal is sent. S7: After the first controller 7-1 receives the signal indicating that the goods have been placed, it sends an instruction to the omnidirectional chassis device 2. The warehousing handling robot goes to the inventory area. After arriving at the designated inventory area, the first controller 7-1 sends an instruction to the scissor lift conveying device 6. The scissor support 6-2 in the scissor lift conveying device 6 rises or falls driven by the hydraulic transmission system 6-3 until the height of the goods placement conveyor mechanism 6-1 on the scissor lift conveying device 6 is the same as the height of the position of the goods cabinet to be loaded. The first controller 7-1 sends an instruction to the goods placement conveyor mechanism 6-1 to make the goods placement conveyor mechanism 6-1 drive to convey the goods into the goods cabinet to be loaded. After the action is completed, a feedback signal is sent. S8: After the first controller 7-1 receives the signal indicating that the goods have been stored, it sends an instruction to the omnidirectional chassis device 2. The warehousing handling robot returns to the starting position to prepare for the next round of work.
[0051] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art in this technical field, without departing from the technical principle of the present invention, several improvements and deformations can still be made, and these improvements and deformations should also be regarded as the protection scope of the present invention.
Claims
1. A storage and handling robot based on deep machine vision technology, comprising a frame, characterized in that: The frame is provided with an omnidirectional chassis device, a scissor-type lifting material transport device and a horizontal propulsion device in sequence from bottom to top. A vertical lifting device is provided on the horizontal propulsion device. A mechanical claw is installed on the vertical lifting device. A mechanical system control module, a machine vision recognition module and a multi-sensor coordination module are also fixed on the frame. The omnidirectional chassis device is located at the bottom of the frame and includes omnidirectional wheels and chassis motors driving the omnidirectional wheels; A scissor-type lifting material transport device is arranged above the omnidirectional chassis device, and the scissor-type lifting material transport device lifts the goods to a suitable height and transports them, and comprises a scissor-type bracket, and a conveyor belt mechanism is arranged above the scissor-type bracket, and the bottom of the scissor-type bracket is fixed on the omnidirectional chassis device; A horizontal propulsion device is arranged at the top of the frame, and the horizontal propulsion device realizes horizontal forward and backward movement, and includes two mutually parallel horizontal slide rails and a horizontal slider moving on the horizontal slide rails, and the horizontal slider is driven by a horizontal motor through a horizontal gear rack mechanism, and a bearing plate is arranged between the horizontal sliders, and a vertical lifting device is arranged on the bearing plate; The vertical lifting device realizes up and down movement in the vertical direction, and includes a vertical motor and a vertical gear rack mechanism matched with the vertical motor, a vertical guide rail slider mechanism is fixedly installed on the vertical gear rack mechanism, a machine vision recognition module is arranged on the vertical guide rail slider mechanism, and a mechanical claw is fixedly connected to the end of the vertical gear rack mechanism facing the ground; The machine vision recognition module recognizes the cargo model and monitors the environmental information around the machine, and is fixed under the load-bearing plate through the recognition structure fixing piece; The multi-sensor collaborative module includes a grayscale sensor, an ultrasonic sensor, a laser radar sensor and a photoelectric sensor. The grayscale sensor is used to identify ground marks and is fixedly installed on the omnidirectional chassis device; the ultrasonic sensor is used to measure the distance between the robot and the cargo and is installed above the mechanical claw through an ultrasonic structural fixture; the laser radar sensor is used to sense the surrounding working environment and is fixedly installed on the omnidirectional chassis device; the photoelectric sensor is used to identify cargo information and is fixedly installed on the vertical gear rack mechanism; The mechanical system control module includes a first controller and a second controller. The first controller is electrically connected to the omnidirectional chassis device, the scissor-type lifting material transport device, and the multi-sensor coordination module. The second controller is electrically connected to the horizontal propulsion device, the vertical lifting device, the machine vision recognition module, the multi-sensor coordination module and the mechanical claw. The first controller is electrically connected to the second controller and the first controller controls the second controller.
2. A storage and handling robot based on deep machine vision technology according to claim 1, characterized in that: The omnidirectional chassis device includes four Mecanum wheels arranged on both sides and a chassis motor driving each Mecanum wheel, wherein the Mecanum wheel is connected to the chassis motor through a coupling, the chassis motor is a DC encoder motor, and the Mecanum wheel is fixed to the frame through a double spring suspension damping mechanism; The first controller in the mechanical system control module, the laser radar sensor and the grayscale sensor in the multi-sensor coordination module are fixedly mounted on the frame of the omnidirectional chassis device, and the grayscale sensor is mounted on the frame facing the direction of travel.
3. A storage and handling robot based on deep machine vision technology according to claim 2, characterized in that: The first controller adjusts the speed of the chassis motor by adjusting the PWM pulse duty cycle of the chassis motor driver through a PID closed-loop control algorithm. The chassis motor is a DC encoder motor, in which the encoder detects the actual speed y(t) of the motor in real time and compares it with the target speed r(t) set in the program to obtain the speed error e(t)=r(t)-y(t). Then, the PID controller calculates the adjustment value u(t) according to the error. The formula is: Among them, K p , K i and K d They are proportional, integral, and differential coefficients respectively. The PID controller calculates the adjustment amount u(t) according to the real-time error, which is used to change the duty cycle of the PWM signal, and then adjust the output voltage of the driver to achieve dynamic control of the motor speed.
4. The storage and handling robot based on deep machine vision technology according to claim 1, characterized in that: The scissor-type bracket of the scissor-type lifting material transport device is a hydraulic scissor-type bracket, which is driven by a hydraulic transmission system.
5. The storage and handling robot based on deep machine vision technology according to claim 1, characterized in that: The two horizontal motors of the horizontal propulsion device are stepper motors, which are connected to the spur gears of the horizontal gear rack mechanism to convert the fixed-axis rotation of the horizontal motor into linear motion of the rack of the horizontal gear rack mechanism; the horizontal slider is fixed on the rack of the horizontal gear rack mechanism and driven by it to move back and forth in the horizontal slide rail; a bearing plate is connected and fixed between the two horizontal sliders, and the bearing plate is made of carbon fiber material.
6. The storage and handling robot based on deep machine vision technology according to claim 1, characterized in that: The vertical motor of the vertical lifting device is a DC motor, which is fixedly mounted on the bearing plate and connected to the spur gear of the vertical gear rack mechanism to convert the fixed-axis rotation of the vertical motor into the reciprocating translation of the rack of the vertical gear rack mechanism; the vertical guide rail slider mechanism is mounted on the rack of the vertical gear rack mechanism to achieve lifting and lowering; the photoelectric sensor is fixed on the vertical gear rack mechanism and follows the up and down movement.
7. The storage and handling robot based on deep machine vision technology according to claim 1, characterized in that: The multi-sensor collaborative module includes four five-way grayscale sensors, which are fixed on the frame of the omnidirectional chassis device facing the forward direction through grayscale brackets; the ultrasonic sensors include two, which are installed above the mechanical claw through ultrasonic structure fixings; the laser radar sensor includes one, and the photoelectric sensor includes one.
8. The storage and handling robot based on deep machine vision technology according to claim 1, characterized in that: The first controller uses an Arduino Mega2560 single-chip microcomputer, the second controller uses an Arduino UNO single-chip microcomputer, and the machine vision recognition module uses an OpenMV 4PLUS intelligent vision camera.
9. The storage and handling robot based on deep machine vision technology according to claim 1, characterized in that: The vehicle frame is constructed by a plurality of high-strength aluminum alloy profiles and right-angle profile connectors.
10. A method for transporting a storage and transport robot based on deep machine vision technology according to any of the above claims, characterized in that: S1: The storage and handling robot is at the starting position, and the device is started after the power supply is installed; S2: The first controller in the mechanical system control module sends a command to the omnidirectional chassis device, and the storage and handling robot goes to the task area to obtain the task code. After arriving at the task area, the second controller in the mechanical system control module sends a command to the machine vision recognition module to make it scan the given task QR code. The second controller feeds back information and instructions to the first controller, and the first controller obtains the work task after analysis. S3: The first controller sends a command to the laser radar sensor in the multi-sensor collaboration module to make it sense and feedback the surrounding working environment, and the first controller plans the best path to the target pickup area; S4: The first controller sends a command to the grayscale sensor in the multi-sensor coordination module to make it recognize and feedback the ground marking guide line, and the first controller sends a command to the omnidirectional chassis device to make it go to the target pickup area along the ground marking guide line; S5: After the storage and handling robot arrives at the target pickup area, the second controller sends a command to the machine vision recognition module to scan the code to identify whether it is the goods to be picked up; if the recognition result is the goods to be picked up, the second controller feeds back an information signal to the first controller, and the first controller sends a command to the ultrasonic sensor in the multi-sensor collaboration module, and the ultrasonic sensor feeds back a distance signal. The first controller controls the omnidirectional chassis device to move forward or backward according to the distance until the storage and handling robot is within the effective range of the goods to be picked up; if the recognition result is empty or the goods are not the task goods, it is fed back to the first controller, and the first controller controls the storage and handling robot to go to the next pickup point to scan the code again; S6: After the storage and handling robot is within the effective range of the goods to be picked up, the first controller sends a command to the second controller, and the second controller controls the horizontal propulsion device to work, and through the rotation of the horizontal motor, the horizontal gear rack mechanism and the horizontal guide rail slider mechanism are mechanically transmitted, until the horizontal propulsion device pushes the load-bearing plate above the horizontal guide rail slider mechanism to the top of the goods to be picked up; then the first controller sends a command to the second controller, and the second controller controls the vertical lifting device to work, and through the rotation of the vertical motor, the vertical gear rack mechanism and the vertical guide rail slider mechanism are mechanically transmitted, until the mechanical claw located at the end of the vertical lifting device moves to the effective range of the goods to be picked up; the first controller sends a command to the second controller, and the second controller sends a command to the mechanical claw, and the mechanical claw clamps the goods and places them on the conveyor belt mechanism of the scissor-type lifting material transport device, and feedbacks the signal after the action is completed; S7: After receiving the placement completion signal, the first controller sends a command to the omnidirectional chassis device, and the storage and handling robot goes to the storage area. After arriving at the designated storage area, the first controller sends a command to the scissor lift material transport device, and the scissor bracket in the scissor lift material transport device rises or falls under the drive of the hydraulic transmission system until the height of the placement conveyor belt mechanism on the scissor lift material transport device is consistent with the height of the position of the container to be placed. The first controller sends a command to the placement conveyor belt mechanism to make the placement conveyor belt mechanism drive and transport the goods to the container to be placed, and feedback signals are sent after the action is completed; S8: After receiving the signal that the goods storage is completed, the first controller sends a command to the omnidirectional chassis device, and the storage and handling robot returns to the starting position to prepare for the next round of work.