Visual goods picking and placing method of unmanned omnidirectional forklift

Abstract

The invention relates to the technical field of mobile robot / AGV (Automatic Guided Vehicle) control, in particular to a visual goods taking and placing method of an unmanned omnidirectional forklift, which comprises a goods taking method and an omnidirectional forklift, and the omnidirectional forklift comprises a visual sensor, an omnidirectional chassis and a fork arm action mechanism. Flexible steering and accurate positioning can be achieved in a narrow space, and therefore the defect that a traditional unmanned forklift with a single-rudder chassis is insufficient in flexibility is overcome. Meanwhile, in combination with a visual recognition algorithm, the omnidirectional unmanned forklift can sense the position, form and state information of the goods in the working environment in real time, and accurate goods taking and placing operation is achieved.

Description

Technical Field

[0001] The invention relates to the field of mobile robot / AGV control technology, and in particular to a visual cargo picking and placing method for an unmanned omnidirectional forklift. Background Art

[0002] In the warehousing and logistics industry, unmanned forklifts, as a cutting-edge automated equipment, are gradually becoming a powerful assistant in various industries, and their application scope is becoming increasingly wide. Compared with traditional manual forklifts, unmanned forklifts have shown significant advantages in operation accuracy, consistency and efficiency. They can accurately perform various tasks, such as handling, storage and sorting of goods, thereby greatly improving the overall efficiency of warehousing and logistics operations.

[0003] However, although unmanned forklifts have shown great potential in automated operations, they still face some challenges in practical applications. In particular, in complex scenarios such as narrow aisle space, inaccurate cargo placement, and overloaded or over-height cargo on shelves, traditional single-rudder chassis unmanned forklifts are prone to not being able to successfully complete their tasks due to lack of flexibility.

[0004] Specifically, when turning in a narrow space, traditional unmanned forklifts with a single rudder chassis often require a large turning radius, which limits their flexible application in narrow aisles. In addition, when the goods are placed in an inaccurate position, the unmanned forklift with a single rudder chassis may not be able to accurately adjust the position of the body, resulting in the inability to accurately place the goods in the designated location. To make matters more complicated, when there are situations such as over-loading or over-height of goods on the shelf, the unmanned forklift with a single rudder chassis may not be able to operate due to limitations on the height or stability of the body. Summary of the invention

[0005] (I) Purpose of the invention

[0006] In view of this, the purpose of the present invention is to propose a visual picking and placing method for an unmanned omnidirectional forklift. The omnidirectional unmanned forklift adopts advanced omnidirectional mobile technology, which can achieve flexible steering and precise positioning in a small space, thereby overcoming the shortcomings of traditional single-rudder chassis unmanned forklifts in flexibility.

[0007] (II) Technical solution

[0008] A method for picking and placing goods by visual means of an unmanned omnidirectional forklift comprises a picking method and an omnidirectional forklift, wherein the omnidirectional forklift comprises a visual sensor, an omnidirectional chassis and a fork arm action mechanism, wherein the omnidirectional chassis comprises a plurality of steering wheels, each steering wheel having both steering and walking functions, and the angle and speed are regulated by an algorithm to realize lateral movement, forward / backward movement and spin, etc., wherein the fork arm action mechanism comprises a hydraulic system, a fork arm and a distance measuring sensor, and different operation logics are executed by controlling the lifting and rearward movement of the fork arm, wherein the visual sensor can be fixed at any position on the fork arm and on the fork tip and move therewith, or can be fixed on the body of the forklift to identify the position information of the goods, and the picking method comprises the following steps:

[0009] 1) Preprocessing stage: Collect multiple sets of pictures of goods and use reinforcement learning algorithm to build a model to identify goods in different positions;

[0010] 2) Execution of motion instructions: Start the unmanned omnidirectional forklift and execute the motion instruction to move to the preset storage location direction;

[0011] 3) Approaching the storage location judgment stage: During the movement of the forklift, it is judged whether it is close to the preset storage location; if it is close, it proceeds to the next step; if not, it continues to execute the motion instruction and moves towards the storage location; this judgment is based on the distance or time information between the forklift and the preset storage location;

[0012] 4) Visual recognition and target posture acquisition stage: When the forklift approaches the preset warehouse location, the visual sensor is activated to identify the posture of the target goods or shelves, and the recognition results are fed back to the control system;

[0013] 5) Path planning stage: Based on the position information of the goods or shelves fed back by the visual sensor, as well as the current position and posture of the forklift, a collision-free path is dynamically generated; the path planning strategy makes decisions based on the degree of deviation of the goods and the feasible space conditions, including generating a curved path or planning a straight path for lateral movement after deceleration and parking, while considering the kinematic limitations of the forklift and space occupancy optimization;

[0014] 6) Moving to the storage location: Control the unmanned omnidirectional forklift to move to the preset storage location along the planned path;

[0015] 7) Fork arm action execution stage: According to the position information of the target goods, the fork arm is controlled to rise and fall and move forward and backward to perform the task of picking up or releasing goods. During the execution of the task, if the visual system detects that the current position cannot meet the operation requirements, the position of the forklift is adjusted according to the position information provided by the visual system, and the path is replanned to ensure the safe and accurate completion of the task;

[0016] 8) Ending stage: When the task of picking up or releasing goods is completed, the entire process ends.

[0017] Preferably, the approach to the storage location in step 3 is determined based on the relative speed or acceleration information between the forklift and the preset storage location.

[0018] Preferably, the path planning strategy in step 5 also includes consideration of energy consumption optimization and path smoothness of the forklift.

[0019] Preferably, the fork arm action execution phase in step 7 also includes adopting different execution strategies for different types of storage locations (such as shelves, ground pile storage locations and stacked rack storage locations, etc.).

[0020] Preferably, for stacked rack storage locations, before executing the task of placing goods, a step is also included in which the characteristics of the rack are identified by a visual system to confirm whether the upper and lower racks are correctly aligned.

[0021] Preferably, the fork arm action execution phase in step 7 also includes real-time monitoring and feedback adjustment of the fork arm movement to ensure the accuracy and stability of the operation.

[0022] It can be seen from the above technical solutions that the present application has the following beneficial effects:

[0023] 1. The invented omnidirectional unmanned forklift adopts advanced omnidirectional mobile technology, which can achieve flexible steering and precise positioning in a small space, thus overcoming the lack of flexibility of traditional single-rudder chassis unmanned forklifts. At the same time, combined with visual recognition algorithms, the omnidirectional unmanned forklift can perceive the location, shape and status information of goods in the working environment in real time, and realize accurate picking and releasing operations. This innovative solution aims to improve the adaptability and operating efficiency of unmanned forklifts in complex working environments.

[0024] 2. The invented omnidirectional forklift visual picking / cargo strategy is also optimized for different task types, using visual sensors to determine in real time whether the picking / cargo release conditions are met, thereby ensuring the safety and reliability of the picking / cargo release action. Combined with advanced omnidirectional mobile technology, the unmanned forklift can achieve flexible steering in a small space, while accurately sensing the cargo information in the working environment, and achieving precise picking and releasing operations. This solution not only improves the adaptability of unmanned forklifts in complex working environments, but also further ensures the safety and efficiency of the operating process, providing strong support for the intelligent development of the warehousing and logistics industries. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a schematic diagram of the process of the omnidirectional forklift of the present invention moving to a storage location based on vision;

[0026] Figure 2 It is a schematic diagram of the overall structure of the omnidirectional vehicle of the present invention.

[0027] In the figure: 1. Omnidirectional forklift; 2. Omnidirectional chassis; 3. Fork arm action mechanism; 4. Steering wheel; 5. Visual sensor. DETAILED DESCRIPTION

[0028] The following description is merely exemplary in nature and is not intended to limit the present disclosure, application, and use. It should be understood that in all of these figures, the same or similar reference numerals indicate the same or similar parts and features. The various drawings only schematically represent the concepts and principles of the embodiments of the present disclosure, and do not necessarily show the specific dimensions and proportions of the various embodiments of the present disclosure. Specific parts in specific drawings may be exaggerated to illustrate the relevant details or structures of the embodiments of the present disclosure.

[0029] See also Figure 1-2 , an embodiment provided by the present invention:

[0030] A method for picking and placing goods by vision using an unmanned omnidirectional forklift includes a picking method and an omnidirectional forklift 1. The omnidirectional forklift 1 includes a visual sensor 5, an omnidirectional chassis 2, and a fork arm action mechanism 3. The omnidirectional chassis 2 includes multiple steering wheels 4, each steering wheel 4 has both steering and walking functions, and the angle and speed are regulated by an algorithm to achieve lateral movement, forward / backward and spin movements. The fork arm action mechanism 3 includes a hydraulic system, a fork arm, and a distance measurement sensor, and different operation logics are executed by controlling the lifting and rearward movement of the fork arm. The visual sensor 5 can be fixed at any position on the fork arm and on the fork tip and move therewith, or can be fixed on the body of the forklift to identify the position information of the goods. The picking method includes the following steps:

[0031] 1) Preprocessing stage: Collect multiple sets of pictures of goods and use reinforcement learning algorithm to build a model to identify goods in different positions;

[0032] 2) Execution of motion instructions: Start the unmanned omnidirectional forklift and execute the motion instruction to move to the preset storage location direction;

[0033] 3) Approaching the storage location judgment stage: During the movement of the forklift, it is judged whether it is close to the preset storage location; if it is close, it proceeds to the next step; if not, it continues to execute the motion instruction and moves towards the storage location; this judgment is based on the distance or time information between the forklift and the preset storage location;

[0034] 4) Visual recognition and target posture acquisition stage: When the forklift approaches the preset storage location, the visual sensor 5 is activated to identify the posture of the target goods or shelves, and the recognition result is fed back to the control system;

[0035] 5) Path planning stage: Based on the cargo or shelf posture information fed back by the visual sensor 5 and the current position and posture of the forklift, a collision-free path is dynamically generated; the path planning strategy makes decisions based on the degree of cargo deviation and feasible space conditions, including generating a curved path or planning a straight path for lateral movement after deceleration and parking, while considering the kinematic limitations of the forklift and space occupancy optimization;

[0036] 6) Moving to the storage location: Control the unmanned omnidirectional forklift to move to the preset storage location along the planned path;

[0037] 7) Fork arm action execution stage: According to the position information of the target goods, the fork arm is controlled to rise and fall and move forward and backward to perform the task of picking up or releasing goods. During the execution of the task, if the visual system detects that the current position cannot meet the operation requirements, the position of the forklift is adjusted according to the position information provided by the visual system, and the path is replanned to ensure the safe and accurate completion of the task;

[0038] 8) Ending stage: When the task of picking up or releasing goods is completed, the entire process ends.

[0039] Furthermore, the judgment of approaching the storage location in step 3 is based on the relative speed or acceleration information between the forklift and the preset storage location, so that it is possible to more accurately judge whether the forklift is close to the target location, thereby starting the visual recognition system in a timely manner, reducing unnecessary waiting time and improving work efficiency.

[0040] Furthermore, the path planning strategy in step 5 also includes consideration of the forklift's energy consumption optimization and path smoothness, which can ensure that the forklift moves in a more energy-efficient manner when performing tasks, thereby reducing operating costs.

[0041] Furthermore, the fork arm action execution stage in step 7 also includes adopting different execution strategies for different types of storage locations (such as shelves, ground pile storage locations, and stacked rack storage locations, etc.). By designing a specific execution strategy for each type of storage location, it can be ensured that the forklift can complete the task efficiently and accurately in different environments.

[0042] Furthermore, for stacked rack storage locations, before the release task is performed, the visual system is used to identify the characteristics of the rack and confirm whether the upper and lower racks are correctly aligned, thereby preventing safety hazards caused by improper stacking of goods. Before the release task is performed at the stacked rack storage location, the visual system is used to identify the characteristics of the rack and confirm whether the upper and lower racks are aligned, which can ensure that the goods are stacked neatly and stably, avoiding safety issues such as cargo collapse or forklift damage caused by improper stacking.

[0043] Furthermore, the fork arm movement execution phase in step 7 also includes real-time monitoring and feedback adjustment of the fork arm movement to ensure the accuracy and stability of the operation, which can ensure that the position and posture of the fork arm during the process of picking up and placing goods are accurate.

[0044] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A method for picking and placing goods by visual means using an unmanned omnidirectional forklift, characterized in that: The invention comprises a method for picking up goods and an omnidirectional forklift (1). The omnidirectional forklift (1) comprises a visual sensor (5), an omnidirectional chassis (2) and a fork arm action mechanism (3). The omnidirectional chassis (2) comprises a plurality of steering wheels (4), each steering wheel (4) has both steering and walking functions, and the angle of rotation and the rotation speed are regulated by an algorithm to realize lateral movement, forward / backward movement and self-spinning, etc. The fork arm action mechanism (3) comprises a hydraulic system, a fork arm and a distance measurement sensor, and different operation logics are executed by controlling the lifting and rearward movement of the fork arm. The visual sensor (5) can be fixed at any position on the fork arm and on the fork tip and move accordingly, or can be fixed on the body of the forklift to identify the position information of the goods. The method for picking up goods comprises the following steps: 1) Preprocessing stage: Collect multiple sets of pictures of goods and use reinforcement learning algorithm to build a model to identify goods in different positions; 2) Execution of motion instructions: Start the unmanned omnidirectional forklift and execute the motion instruction to move to the preset storage location direction; 3) Approaching the storage location judgment stage: During the movement of the forklift, it is judged whether it is close to the preset storage location; if it is close, it proceeds to the next step; if not, it continues to execute the motion instruction and moves towards the storage location; this judgment is based on the distance or time information between the forklift and the preset storage location; 4) Visual recognition and target position acquisition stage: When the forklift approaches the preset warehouse location, the visual sensor (5) is activated to identify the position of the target goods or shelves, and the recognition results are fed back to the control system; 5) Path planning stage: Based on the cargo or shelf position information fed back by the visual sensor (5) and the current position and posture of the forklift, a collision-free path is dynamically generated; the path planning strategy makes decisions based on the degree of cargo deviation and feasible space conditions, including generating a curved path or planning a straight path for lateral movement after deceleration and parking, while considering the kinematic limitations of the forklift and space occupancy optimization; 6) Moving to the storage location: Control the unmanned omnidirectional forklift to move to the preset storage location along the planned path; 7) Fork arm action execution stage: According to the position information of the target goods, the fork arm is controlled to rise and fall and move forward and backward to perform the task of picking up or releasing goods. During the execution of the task, if the visual system detects that the current position cannot meet the operation requirements, the position of the forklift is adjusted according to the position information provided by the visual system, and the path is replanned to ensure the safe and accurate completion of the task; 8) Ending stage: When the task of picking up or releasing goods is completed, the entire process ends.

2. According to claim 1, a method for picking and placing goods by visual means of an unmanned omnidirectional forklift is characterized in that: The judgment of approaching the storage location in step 3 is based on the relative speed or acceleration information between the forklift and the preset storage location.

3. According to claim 1, a method for picking up and placing goods by visual means using an unmanned omnidirectional forklift is characterized in that: The path planning strategy in step 5 also includes consideration of the energy consumption optimization and path smoothness of the forklift.

4. According to claim 1, a method for picking up and placing goods by visual means using an unmanned omnidirectional forklift is characterized in that: The fork arm action execution phase in step 7 also includes adopting different execution strategies for different types of storage locations (such as shelves, ground pile storage locations and stacked rack storage locations, etc.).

5. According to claim 1, a method for picking and placing goods by visual means using an unmanned omnidirectional forklift is characterized in that: For stacked rack storage locations, before executing the release task, it also includes the step of identifying the characteristics of the rack through a visual system to confirm whether the upper and lower racks are correctly aligned.

6. According to claim 1, a method for picking and placing goods by visual means using an unmanned omnidirectional forklift is characterized in that: The fork arm movement execution phase in step 7 also includes real-time monitoring and feedback adjustment of the fork arm movement to ensure the accuracy and stability of the operation.

Images