Visual identification-based picking operation obstacle avoidance robot

Abstract

The invention relates to the field of picking robots, in particular to a picking operation obstacle avoidance robot based on visual recognition, and aims to solve the problem that soil ridges or low plants are easily damaged by rolling due to the fact that a picking operation robot cannot fully adapt to a complex planting environment in the picking and automatic obstacle avoidance processes. According to the invention, a picking area, a path range and an operation picking range which are set manually are limited, the moving path of the picking operation obstacle avoidance robot is controlled, and a finer electronic fence is arranged on the basis of autonomous obstacle identification and path selection of the robot. Therefore, the picking operation obstacle avoidance robot cannot damage the environment of a planting area or unrecognized plants due to recognition errors in the moving process, the obstacle avoidance function of the picking operation obstacle avoidance robot in the automatic picking operation process is optimized, meanwhile, the picking moving path is dynamically adjusted when the path is automatically re-planned during obstacle avoidance, and the picking efficiency is improved. And the picking area sequence configuration flexibility is improved.

Description

Technical Field

[0001] The present invention relates to the field of picking robots, in particular to a picking operation obstacle avoidance robot based on visual recognition. Background Art

[0002] With the maturity of robot technology, the reduction of costs and the promotion of application, robots have gradually entered the agricultural field and will promote the development of modern agriculture towards industrialized production, unmanned and intelligent production. The picking of fruits and vegetables is characterized by strong seasonality, high labor intensity, and high environmental and operational requirements. Robotic operations are urgently needed in agricultural production.

[0003] For example, patent application CN116806552A discloses a technical solution, which realizes the clamping operation through the gravity of the fruit itself and the elastic claw rod, effectively avoiding the damage of the fruit skin during the picking process and improving the picking efficiency; for example, patent application CN112171667A discloses a technical solution, which adopts an embedded control system based on the picking robot, thereby improving the control efficiency and obstacle avoidance performance of the picking robot to a certain extent.

[0004] However, in the above-mentioned patents, obstacle avoidance operations are performed autonomously through real-scene collection during the picking operations and obstacle avoidance actions. In a complex mixed planting environment, there may be some relatively short plants, which may make it difficult for the obstacle avoidance robot to detect the small and short plants as obstacles when moving. This may cause the robot's moving path to be uncontrollable, and may easily crush and damage soil ridges or plants, resulting in economic losses. Summary of the invention

[0005] The present invention limits the picking area, path range and operation picking range set manually, controls the moving path of the picking operation obstacle avoidance robot, and sets a more sophisticated electronic fence based on the robot's autonomous obstacle recognition and path selection, so that the picking operation obstacle avoidance robot will not cause the environment of the planting area or unrecognized plants to be damaged due to recognition errors during movement, optimizes the obstacle avoidance function of the picking operation obstacle avoidance robot during automatic picking operations, and solves the problem that the picking operation robot cannot fully adapt to the complex planting environment during the picking and automatic obstacle avoidance processes, resulting in soil ridges or low plants being easily crushed and damaged, and a picking operation obstacle avoidance robot based on visual recognition is proposed.

[0006] The object of the present invention can be achieved by the following technical solutions: a picking operation obstacle avoidance robot based on visual recognition, comprising an equipment body, a storage box for storing picked products is installed above the equipment body, a sensor integrated block is fixedly installed in front of the equipment body, a main shaft is arranged above the sensor integrated block, at least one group of rotating rods is rotatably connected to the main shaft, a universal ball head is movably connected to the end of the rotating rod, and a mechanical picking claw is installed on the universal ball head; The main shaft is equipped with rotating structures of the same number as the rotating rods, and each rotating mechanism can rotate independently. The sensor integrated block is equipped with at least one set of millimeter wave radar, visible light camera and central controller. The central controller includes a picking operation obstacle avoidance robot control system. The equipment body, rotating rod, universal ball head and mechanical picking claw are controlled by the central controller. The picking operation obstacle avoidance robot control system includes an external information collection module, an operation planning module, an autonomous obstacle avoidance module and an information interaction module; The external information acquisition module senses the external environment information through the sensor integrated block, and digitally converts the sensed external environment information, and sends the conversion results to the operation planning module and the autonomous obstacle avoidance module; The information interaction module can communicate with the host platform through a wireless communication device to obtain operation-related control information, and obtain the current operation area and pre-set operation environment information through the wireless communication device; The operation planning module obtains operation-related control information and operation environment information through the information interaction module and plans the operation path. The autonomous obstacle avoidance module analyzes the obstacles on the operation path in real time through the external environment information conversion results and adjusts the operation path in real time.

[0007] As a preferred embodiment of the present invention, the external environment information collected by the external information collection module includes echo reflection information and visible light image information. After obtaining the echo reflection information, the external information collection module constructs an image of the echo reflection information to obtain 3D structure modeling; When acquiring the visible light picture, the external information acquisition module acquires the stereoscopic visible light picture through the binocular camera, processes the stereoscopic visible light picture, and generates 3D picture modeling; The external information acquisition module uses 3D structure modeling and 3D picture modeling as conversion results.

[0008] As a preferred embodiment of the present invention, the operation environment information acquired by the information interaction module includes: the total operation range, the operation picking range and the operation path range; The total operation range is the total area of ​​the picking environment, the operation picking range is the range of crop growth in the operation environment, and the operation path range is the range within which the obstacle avoidance robot can travel for the picking operation; The operation-related control information acquired by the information interaction module includes picking conditions and picking areas, wherein the picking conditions are the appearance information of the picked crops, and the picking area is the maximum range of motion of the mechanical picking claws when the picking operation obstacle avoidance robot body is stationary.

[0009] As a preferred embodiment of the present invention, after obtaining the working environment information and the working related control information, the working planning module overlaps the working picking range and the picking area, so that the working picking range is completely covered by the picking area, and marks the overlapping picking area; The operation planning module is then limited by the operation path range and picking conditions, and the picking path is automatically generated by the algorithm to obtain the initial picking path, wherein the initial picking path is limited within the operation path range, and the initial picking path can cover all marked overlapping areas after being covered by the maximum action range of the mechanical picking claw.

[0010] As a preferred embodiment of the present invention, after the operation planning module obtains the 3D picture modeling, it determines the position of the picking operation obstacle avoidance robot. When the picking operation obstacle avoidance robot is at the edge of the picking area, the picking crop picture in the 3D picture modeling is extracted and located through an intelligent algorithm, and the mechanical picking claws are controlled to pick the crops according to the positioning results.

[0011] As a preferred embodiment of the present invention, after the picking of the picking area is completed, the operation planning module controls the picking operation obstacle avoidance robot to move to the next picking area; When the picking operation obstacle avoidance robot moves to the next picking area, the autonomous obstacle avoidance module analyzes the obstacles on the path through 3D structural modeling and marks the areas. The picking operation obstacle avoidance robot records the unmarked areas in the 3D structural modeling as obstacle-free areas. The autonomous obstacle avoidance module obtains the operating path range through the information interaction module, and compares the overlap between the obstacle-free area and the operating path range, and records the overlapping part of the operating path range and the obstacle-free area as the drivable range. The autonomous obstacle avoidance module compares the drivable range with the set passing size of the picking operation obstacle avoidance robot. If the drivable range is less than or equal to the passing size, it is determined to be impassable; if the drivable range is greater than the passing size, it is determined to be passable.

[0012] As a preferred embodiment of the present invention, the autonomous obstacle avoidance module sends the drivable range to the operation planning module after determining that it is passable. The operation planning module corrects the drivable range in the operation path, so that the picking operation obstacle avoidance robot moves through the drivable range when heading to the next picking area.

[0013] As a preferred embodiment of the present invention, after determining that the autonomous obstacle avoidance module is impassable, the autonomous obstacle avoidance module sends a route interruption signal to the operation planning module, and sends the impassable position to the operation planning module. The operation planning module marks the impassable position and re-plans the operation path.

[0014] Compared with the prior art, the present invention has the following beneficial effects: In the present invention, during the picking operation, the picking area, path range and operation picking range are manually set to limit the movement path of the picking operation obstacle avoidance robot, and a more sophisticated electronic fence is set based on the robot's autonomous obstacle recognition and path selection, so that the picking operation obstacle avoidance robot will not cause the environment of the planting area or unrecognized plants to be damaged due to recognition errors during movement, thereby optimizing the obstacle avoidance function of the picking operation obstacle avoidance robot during automatic picking operations.

[0015] In the present invention, when the picking operation obstacle avoidance robot encounters an obstacle that it cannot pass, it automatically re-plans the path and dynamically adjusts the picking movement path according to the new path planning, thereby improving the flexibility of the picking area sequence configuration, improving the intelligence of the picking operation obstacle avoidance robot, greatly improving the rationality of the picking sequence of the picking operation obstacle avoidance robot during obstacle detour and path planning, and improving the picking efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0017] Figure 1 It is a schematic diagram of the three-dimensional structure of the present invention; Figure 2 is a system block diagram of the present invention; Figure 3 It is a system flow chart of the present invention.

[0018] In the figure: 1. Storage box; 2. Sensor integrated block; 3. Equipment body; 4. Rotating rod; 5. Universal ball head; 6. Mechanical picking claw. DETAILED DESCRIPTION

[0019] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] Example 1: Please refer to Figure 1 - Figure 3 As shown, the obstacle avoidance robot for harvesting operation based on visual recognition includes a device body 3, which includes a bearing body for installing various structures, a crawler for traveling on a complex muddy ground, a power supply for providing driving and other functions, and the like. A storage box 1 for storing harvested products is installed above the device body 3, which is used to store the harvested crops; A sensor integrated block 2 is fixedly installed in front of the equipment body 3, and at least one group of millimeter-wave radars, visible light cameras and a central controller are installed on the sensor integrated block 2 to identify obstacles in front and crops that need to be picked. A main shaft is arranged above the sensor integrated block 2, and at least one group of rotating rods 4 are rotatably connected to the main shaft. The ends of the rotating rods 4 are movably connected to a universal ball head 5, and a mechanical picking claw 6 is installed on the universal ball head 5. The universal ball head 5 can make the mechanical picking claw 6 move in multiple directions, so as to realize picking in complex areas and place the fruits after picking in the storage box 1. The mechanical picking claw 6 and the universal ball head 5 can also be equipped with different retractable structures according to the distribution height of the crops to be picked, so as to further expand the picking operation range; The main shaft is equipped with the same number of rotating structures as the rotating rods 4, and each rotating mechanism can rotate independently, and at the same time control multiple groups of rotating rods 4 and mechanical picking claws 6 to move, thereby improving the picking efficiency. The central controller includes a picking operation obstacle avoidance robot control system, and the equipment body 3, rotating rod 4, universal ball head 5 and mechanical picking claws 6 are controlled by the central controller.

[0021] Example 2: Please refer to Figure 1 - Figure 3 As shown, the picking operation obstacle avoidance robot control system includes an external information collection module, an operation planning module, an autonomous obstacle avoidance module and an information interaction module; The external information acquisition module senses the external environment information through the sensor integrated block 2. The external environment information acquired by the external information acquisition module includes echo reflection information and visible light image information. After obtaining the echo reflection information, the external information acquisition module constructs the image of the echo reflection information to obtain 3D structure modeling; When acquiring visible light images, the external information acquisition module acquires stereoscopic visible light images through a binocular camera, processes the stereoscopic visible light images, and generates 3D image modeling; The external information acquisition module digitally converts the 3D structure modeling and 3D picture modeling for easy transmission and sends the conversion results to the operation planning module and the autonomous obstacle avoidance module; The information interaction module can communicate with the host platform through wireless communication equipment to obtain operation-related control information. It is also equipped with a satellite positioning device and a small-area precise positioning device based on the positioning base station to improve the accuracy of the positioning of the obstacle avoidance robot in the picking operation, and obtain the current operation area and pre-set operation environment information through wireless communication equipment; The operation environment information obtained by the information interaction module includes: the total operation range, the operation picking range and the operation path range. The above information is manually entered by the operator according to the actual planting environment; The total operation range is the total area of ​​the picking environment, such as the total area of ​​the planting field or the total area of ​​the greenhouse. The operation picking range is the range of crop growth in the operation environment, such as the crop planting ridges and the distribution positions of the plants. The operation path range is the range available for the picking operation obstacle avoidance robot to travel, such as the reserved field roads, greenhouse roads or roads reserved for walking between planting areas. The road width should be greater than the width of the picking operation obstacle avoidance robot. The operation-related control information acquired by the information interaction module includes picking conditions and picking areas, wherein the picking conditions are the appearance information of the picked crops, and the picking area is the maximum range of motion of the mechanical picking claw 6 when the picking operation obstacle avoidance robot body is stationary; The operation planning module obtains the operation-related control information and the operation environment information through the information interaction module, and plans the operation path. When planning the path, it generates it through the path generation algorithm in the prior art; After obtaining the working environment information and the working control information, the working planning module overlaps the working picking range and the picking area, so that the working picking range is completely covered by the picking area, and marks the overlapping picking areas, and uses each picking area as the working area of ​​the picking operation obstacle avoidance robot. When the picking operation obstacle avoidance robot completes the work of a picking area, it moves to the next picking area again; The operation planning module is then limited by the operation path range and picking conditions, and the picking path is automatically generated by the mature path algorithm in the prior art to obtain the initial picking path, wherein the initial picking path is limited within the operation path range, and the initial picking path can cover all marked overlapping areas after being covered by the maximum action range of the mechanical picking claw 6, that is, the initial picking path connects all picking areas in series; After the operation planning module obtains the 3D picture modeling, it determines the position of the picking operation obstacle avoidance robot. When the picking operation obstacle avoidance robot is at the edge of the picking area, the picking crop picture in the 3D picture modeling is extracted and located through the intelligent algorithm, and the mechanical picking claw 6 is controlled according to the positioning result to pick the crops and perform operations in a picking area.

[0022] After the picking is completed in the picking area, the operation planning module controls the picking operation obstacle avoidance robot to move to the next picking area; The autonomous obstacle avoidance module analyzes obstacles on the operating path in real time through the conversion results of external environmental information and adjusts the operating path in real time.

[0023] Example 2: Please refer to Figure 1 - Figure 3 As shown, when the picking operation obstacle avoidance robot moves to the next picking area, the autonomous obstacle avoidance module analyzes the obstacles on the path through 3D structural modeling, and marks the areas where obstacles are not marked in the 3D structural modeling. The picking operation obstacle avoidance robot records the unmarked areas in the 3D structural modeling as obstacle-free areas. The autonomous obstacle avoidance module obtains the operating path range through the information interaction module, and compares the overlap between the obstacle-free area and the operating path range, and records the overlap between the operating path range and the obstacle-free area as the drivable range. The autonomous obstacle avoidance module compares the drivable range with the set passing size of the picking operation obstacle avoidance robot. If the drivable range is less than or equal to the passing size, it is determined to be impassable; if the drivable range is greater than the passing size, it is determined to be passable; After determining that the autonomous obstacle avoidance module is passable, the drivable range is sent to the operation planning module. The operation planning module corrects the drivable range in the operation path, so that the picking operation obstacle avoidance robot moves through the drivable range when heading to the next picking area, thereby achieving the function of avoiding obstacles; At the same time, since the obstacle avoidance route is obtained by correcting the working path and the drivable range, and the working path is obtained on the basis of the working path range, the picking operation obstacle avoidance robot will not exceed the set working path range when moving, thereby avoiding the situation where the picking operation obstacle avoidance robot cannot identify the soil ridges or low plants and cause damage; After determining that the route is impassable, the autonomous obstacle avoidance module sends a route interruption signal to the operation planning module, and sends the impassable location to the operation planning module. The operation planning module marks the impassable location and replans the operation path. When planning the new operation path, the impassable location is isolated to bypass the impassable location and ensure that the existing plants or soil ridges are not damaged. During the planning process of a new operation path, if it passes through other picking areas before the picking area that needs to be reached by obstacle avoidance, then the other picking areas will be given priority for picking, thereby improving the picking efficiency and avoiding frequent detours of the picking operation obstacle avoidance robot to reduce work efficiency. When the newly planned path of the picking operation obstacle avoidance robot cannot bypass the inaccessible position and reach the area that needs to be picked, the inaccessible position will be isolated, and other picking areas that have not been isolated will be given priority for picking. After the picking is completed, the inaccessible position will be reached again for a second obstacle recognition to confirm whether the obstacle has disappeared or changed, and recalculate whether it can pass.

[0024] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to only specific implementation methods. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A picking operation obstacle avoidance robot based on visual recognition, comprising a device body (3), characterized in that: A storage box (1) for storing picked products is installed above the device body (3); a sensor integrated block (2) is fixedly installed in front of the device body (3); a main shaft is arranged above the sensor integrated block (2); at least one set of rotating rods (4) is rotatably connected to the main shaft; a universal ball head (5) is movably connected to the end of the rotating rod (4); a mechanical picking claw (6) is installed on the universal ball head (5); The main shaft is provided with rotating structures having the same number as the rotating rods (4), and each rotating mechanism is capable of rotating independently. The sensor integrated block (2) is provided with at least one set of millimeter wave radars, visible light cameras and a central controller. The central controller includes a picking operation obstacle avoidance robot control system. The device body (3), the rotating rod (4), the universal ball head (5) and the mechanical picking claw (6) are controlled by the central controller. The picking operation obstacle avoidance robot control system includes an external information collection module, an operation planning module, an autonomous obstacle avoidance module and an information interaction module; The external information acquisition module senses external environmental information through the sensor integrated block (2), performs digital conversion on the sensed external environmental information, and sends the conversion result to the operation planning module and the autonomous obstacle avoidance module; The information interaction module can communicate with the host platform through a wireless communication device to obtain operation-related control information, and obtain the current operation area and pre-set operation environment information through the wireless communication device; The operation planning module obtains operation-related control information and operation environment information through the information interaction module and plans the operation path. The autonomous obstacle avoidance module analyzes the obstacles on the operation path in real time through the external environment information conversion results and adjusts the operation path in real time.

2. The picking operation obstacle avoidance robot based on visual recognition according to claim 1 is characterized in that: The external environment information collected by the external information collection module includes echo reflection information and visible light image information. After obtaining the echo reflection information, the external information collection module constructs an image of the echo reflection information to obtain a 3D structure model; When acquiring the visible light picture, the external information acquisition module acquires the stereoscopic visible light picture through the binocular camera, processes the stereoscopic visible light picture, and generates 3D picture modeling; The external information acquisition module uses 3D structure modeling and 3D picture modeling as conversion results.

3. The picking operation obstacle avoidance robot based on visual recognition according to claim 1, characterized in that: The operation environment information acquired by the information interaction module includes: the total operation range, the operation picking range and the operation path range; The total operation range is the total area of ​​the picking environment, the operation picking range is the range of crop growth in the operation environment, and the operation path range is the range within which the obstacle avoidance robot can travel for the picking operation; The operation-related control information acquired by the information interaction module includes picking conditions and picking areas, wherein the picking conditions are the appearance information of the crop to be picked, and the picking area is the maximum range of motion of the mechanical picking claw (6) when the picking operation obstacle avoidance robot body is stationary.

4. The picking operation obstacle avoidance robot based on visual recognition according to claim 1, characterized in that: After obtaining the operation environment information and the operation related control information, the operation planning module overlaps the operation picking range and the picking area so that the operation picking range is completely covered by the picking area, and marks the overlapping picking area; The operation planning module is then limited by the operation path range and the picking conditions, and the picking path is automatically generated by an algorithm to obtain an initial picking path, wherein the initial picking path is limited within the operation path range, and the initial picking path can cover all marked overlapping areas after being covered by the maximum range of motion of the mechanical picking claw (6).

5. The picking operation obstacle avoidance robot based on visual recognition according to claim 1, characterized in that: After the operation planning module obtains the 3D image modeling, it determines the position of the picking operation obstacle avoidance robot. When the picking operation obstacle avoidance robot is at the edge of the picking area, it extracts and locates the crop picking image in the 3D image modeling through an intelligent algorithm, and controls the mechanical picking claw (6) to pick the crops according to the positioning result.

6. The obstacle avoidance robot for picking operations based on visual recognition according to claim 1, characterized in that: After the picking of the picking area is completed, the operation planning module controls the picking operation obstacle avoidance robot to move to the next picking area; When the picking operation obstacle avoidance robot moves to the next picking area, the autonomous obstacle avoidance module analyzes the obstacles on the path through 3D structural modeling and marks the areas. The picking operation obstacle avoidance robot records the unmarked areas in the 3D structural modeling as obstacle-free areas. The autonomous obstacle avoidance module obtains the operating path range through the information interaction module, and compares the overlap between the obstacle-free area and the operating path range, and records the overlapping part of the operating path range and the obstacle-free area as the drivable range. The autonomous obstacle avoidance module compares the drivable range with the set passing size of the picking operation obstacle avoidance robot. If the drivable range is less than or equal to the passing size, it is determined to be impassable; if the drivable range is greater than the passing size, it is determined to be passable.

7. The picking operation obstacle avoidance robot based on visual recognition according to claim 6, characterized in that: After determining that the autonomous obstacle avoidance module is passable, the drivable range is sent to the operation planning module. The operation planning module corrects the drivable range in the operation path so that the picking operation obstacle avoidance robot moves through the drivable range when heading to the next picking area.

8. The picking operation obstacle avoidance robot based on visual recognition according to claim 6, characterized in that: After determining that the vehicle cannot pass, the autonomous obstacle avoidance module sends a route interruption signal to the operation planning module, and sends the inaccessible position to the operation planning module. The operation planning module marks the inaccessible position and replans the operation path.

Images