Obstacle-avoidance robot for picking operations based on visual recognition

Abstract

The present invention relates to the field of picking robots, and is used to solve the problem that picking operation robots cannot fully adapt to complex planting environments during the process of picking and automatic obstacle avoidance, which makes soil ridges or low plants easily crushed and damaged. Specifically, it is a picking operation obstacle avoidance robot based on visual recognition; the present invention limits the picking area, path range and operation picking range by manually setting, controls the moving path of the picking operation obstacle avoidance robot, and sets a more sophisticated electronic fence based on the robot's autonomous obstacle recognition and path selection, so that the picking operation obstacle avoidance robot will not cause the environment of the planting area or unrecognized plants to be damaged due to recognition errors during movement, optimizes the obstacle avoidance function of the picking operation obstacle avoidance robot during automatic picking operations, and at the same time, dynamically adjusts the picking movement path when automatically replanning the path to avoid obstacles, thereby improving the flexibility of the picking area sequence configuration.

Description

Technical Field

[0001] The present invention relates to the field of picking robots, and in particular to a picking operation obstacle avoidance robot based on visual recognition. Background Art

[0002] With the maturity of robot technology, the reduction of costs and the promotion of application, robots have gradually entered the agricultural field and will promote the development of modern agriculture towards industrialized production, unmanned and intelligent production. The picking of fruits and vegetables is characterized by strong seasonality, high labor intensity, and high environmental and operational requirements. There is an urgent need for robotic operations in agricultural production.

[0003] For example, patent application CN116806552A discloses a technical solution that realizes the clamping operation through the gravity of the fruit itself and the elastic claw rod, effectively avoiding the damage of the fruit skin during the picking process and improving the picking efficiency; for example, patent application CN112171667A discloses a technical solution that adopts an embedded control system based on the picking robot, which improves the control efficiency and obstacle avoidance performance of the picking robot to a certain extent.

[0004] However, in the above patents, obstacle avoidance is performed autonomously through real-scene collection during the picking operations and obstacle avoidance actions. In a complex mixed planting environment, there may be some relatively short plants, which makes it difficult for the obstacle avoidance robot to detect small and short plants as obstacles when moving. As a result, the robot's moving path is uncontrollable, and it is easy to crush and damage soil ridges or plants, causing economic losses. Summary of the Invention

[0005] The present invention limits the manually set picking area, path range and operation picking range, controls the moving path of the picking operation obstacle avoidance robot, and sets a more sophisticated electronic fence based on the robot's autonomous obstacle recognition and path selection, so that the picking operation obstacle avoidance robot will not cause the environment of the planting area or unrecognized plants to be damaged due to recognition errors during movement, optimizes the obstacle avoidance function of the picking operation obstacle avoidance robot during automatic picking operations, and solves the problem that the picking operation robot cannot fully adapt to the complex planting environment during the picking and automatic obstacle avoidance processes, resulting in soil ridges or low plants being easily crushed and damaged, and proposes a picking operation obstacle avoidance robot based on visual recognition.

[0006] The objectives of the present invention can be achieved through the following technical solutions: a picking operation obstacle avoidance robot based on visual recognition, comprising a device body, a storage box for storing picked products is installed above the device body, a sensor integrated block is fixedly installed in front of the device body, a main shaft is provided above the sensor integrated block, at least one set of rotating rods is rotatably connected to the main shaft, the ends of the rotating rods are movably connected to universal ball heads, and mechanical picking claws are installed on the universal ball heads;

[0007] The main shaft is equipped with the same number of rotating structures as rotating rods, and each rotating mechanism can rotate independently. The sensor integrated block is equipped with at least one set of millimeter-wave radar, visible light camera and central controller. The central controller includes a picking operation obstacle avoidance robot control system. The device body, rotating rod, universal ball head and mechanical picking claw are controlled by the central controller.

[0008] The picking operation obstacle avoidance robot control system includes an external information collection module, an operation planning module, an autonomous obstacle avoidance module and an information interaction module;

[0009] The external information acquisition module senses the external environment information through the sensor integrated block, digitally converts the sensed external environment information, and sends the conversion results to the operation planning module and the autonomous obstacle avoidance module;

[0010] The information interaction module can communicate with the host platform through wireless communication equipment to obtain operation-related control information, and obtain the current operation area and pre-set operation environment information through the wireless communication equipment;

[0011] The operation planning module obtains operation-related control information and operation environment information through the information interaction module and plans the operation path. The autonomous obstacle avoidance module analyzes the obstacles on the operation path in real time through the external environment information conversion results and adjusts the operation path in real time.

[0012] As a preferred embodiment of the present invention, the external environment information collected by the external information collection module includes echo reflection information and visible light image information. After obtaining the echo reflection information, the external information collection module constructs an image of the echo reflection information to obtain a 3D structure model;

[0013] When acquiring visible light images, the external information acquisition module acquires stereoscopic visible light images through a binocular camera, processes the stereoscopic visible light images, and generates 3D image modeling;

[0014] The external information acquisition module takes 3D structure modeling and 3D picture modeling as conversion results.

[0015] As a preferred embodiment of the present invention, the operation environment information acquired by the information interaction module includes: the total operation range, the operation picking range and the operation path range;

[0016] The total operation range is the total area of ​​the picking environment, the operation picking range is the range of crop growth in the operation environment, and the operation path range is the range within which the obstacle avoidance robot can travel during the picking operation.

[0017] The operation-related control information acquired by the information interaction module includes picking conditions and picking areas, wherein the picking conditions are the appearance information of the picked crops, and the picking area is the maximum range of movement of the mechanical picking claws when the picking operation obstacle avoidance robot body is stationary.

[0018] As a preferred embodiment of the present invention, after obtaining the working environment information and the working control information, the working planning module overlaps the working picking range and the picking area so that the working picking range is completely covered by the picking area, and marks the overlapping picking area;

[0019] The operation planning module is then limited by the operation path range and picking conditions, and the picking path is automatically generated by the algorithm to obtain the initial picking path, wherein the initial picking path is limited within the operation path range, and the initial picking path can cover all marked overlapping areas after being covered by the maximum movement range of the mechanical picking claw.

[0020] As a preferred embodiment of the present invention, after the operation planning module obtains the 3D picture modeling, it determines the position of the picking operation obstacle avoidance robot. When the picking operation obstacle avoidance robot is at the edge of the picking area, the picking crop picture in the 3D picture modeling is extracted and positioned through an intelligent algorithm, and the mechanical picking claws are controlled to pick the crops according to the positioning results.

[0021] As a preferred embodiment of the present invention, after the picking of the picking area is completed, the operation planning module controls the picking operation obstacle avoidance robot to move to the next picking area;

[0022] When the picking operation obstacle avoidance robot moves to the next picking area, the autonomous obstacle avoidance module analyzes the obstacles on the path through 3D structural modeling and marks the areas where obstacles are not marked in the 3D structural modeling. The picking operation obstacle avoidance robot records the unmarked areas in the 3D structural modeling as obstacle-free areas.

[0023] The autonomous obstacle avoidance module obtains the operating path range through the information interaction module, and compares the overlap between the obstacle-free area and the operating path range, and records the overlapping part of the operating path range and the obstacle-free area as the drivable range. The autonomous obstacle avoidance module compares the drivable range with the set passing size of the picking operation obstacle avoidance robot. If the drivable range is less than or equal to the passing size, it is determined that it cannot pass; if the drivable range is greater than the passing size, it is determined that it can pass.

[0024] As a preferred embodiment of the present invention, after determining that it is passable, the autonomous obstacle avoidance module sends the drivable range to the operation planning module. The operation planning module corrects the drivable range in the operation path, so that the picking operation obstacle avoidance robot moves through the drivable range when heading to the next picking area.

[0025] As a preferred embodiment of the present invention, after determining that the autonomous obstacle avoidance module is unable to pass, it sends a route interruption signal to the operation planning module, and sends the inaccessible position to the operation planning module. The operation planning module marks the inaccessible position and replans the operation path.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] In the present invention, during the picking operation, the picking area, path range and operation picking range are manually set to limit the movement path of the picking operation obstacle avoidance robot, and a more sophisticated electronic fence is set based on the robot's autonomous obstacle recognition and path selection, so that the picking operation obstacle avoidance robot will not cause the environment of the planting area or unrecognized plants to be damaged due to recognition errors during movement, and the obstacle avoidance function of the picking operation obstacle avoidance robot during automatic picking operations is optimized.

[0028] In the present invention, when the picking operation obstacle avoidance robot encounters an obstacle that it cannot pass through, it automatically re-plans the path and dynamically adjusts the picking movement path according to the new path planning, thereby improving the flexibility of the picking area sequence configuration, improving the intelligence level of the picking operation obstacle avoidance robot, greatly improving the rationality of the picking sequence of the picking operation obstacle avoidance robot during obstacle detour and path planning, and improving the picking efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] To facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0030] Figure 1 It is a schematic diagram of the three-dimensional structure of the present invention;

[0031] Figure 2 is a system block diagram of the present invention;

[0032] Figure 3 It is a system flow chart of the present invention.

[0033] In the figure: 1. Storage box; 2. Sensor integrated block; 3. Equipment body; 4. Rotating rod; 5. Universal ball head; 6. Mechanical picking claw. DETAILED DESCRIPTION

[0034] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0035] Example 1: Please refer to Figure 1 - Figure 3 As shown, the obstacle-avoiding robot for harvesting operations based on visual recognition includes a device body 3, which includes a supporting body for mounting various structures, a crawler for traveling on complex muddy ground, and a power supply for providing drive and other functions. A storage box 1 for storing harvested crops is installed above the device body 3.

[0036] A sensor integrated block 2 is fixedly installed in front of the equipment body 3. At least one set of millimeter-wave radar, visible light camera and central controller are installed on the sensor integrated block 2 to identify obstacles in front and crops that need to be picked. A main shaft is provided above the sensor integrated block 2. At least one set of rotating rods 4 are rotatably connected to the main shaft. The ends of the rotating rods 4 are movably connected to universal ball heads 5. Mechanical picking claws 6 are installed on the universal ball heads 5. The universal ball heads 5 can enable the mechanical picking claws 6 to move in multiple directions, thereby realizing picking in complex areas and placing the fruits after picking in the storage box 1. Different retractable structures can be adapted between the mechanical picking claws 6 and the universal ball heads 5 according to the distribution height of the crops to be picked, thereby further expanding the scope of picking operations.

[0037] The main shaft is equipped with the same number of rotating structures as rotating rods 4, and each rotating mechanism can rotate independently, and at the same time control multiple groups of rotating rods 4 and mechanical picking claws 6 to move, thereby improving picking efficiency. The central controller includes a picking operation obstacle avoidance robot control system. The equipment body 3, rotating rod 4, universal ball head 5 and mechanical picking claws 6 are controlled by the central controller.

[0038] Example 2: Please refer to Figure 1 - Figure 3 As shown, the picking operation obstacle avoidance robot control system includes an external information acquisition module, an operation planning module, an autonomous obstacle avoidance module, and an information interaction module;

[0039] The external information acquisition module senses the external environment information through the sensor integrated block 2. The external environment information collected by the external information acquisition module includes echo reflection information and visible light image information. After obtaining the echo reflection information, the external information acquisition module constructs the image of the echo reflection information to obtain a 3D structure model;

[0040] When acquiring visible light images, the external information acquisition module obtains stereoscopic visible light images through a binocular camera, processes the stereoscopic visible light images, and generates 3D image modeling;

[0041] The external information acquisition module digitally converts 3D structure modeling and 3D image modeling for easy transmission and sends the conversion results to the operation planning module and autonomous obstacle avoidance module;

[0042] The information interaction module can communicate with the host platform through wireless communication equipment to obtain operation-related control information. It is also equipped with a satellite positioning device and a small-area precise positioning device based on the positioning base station. This improves the accuracy of the picking operation obstacle avoidance robot's positioning and obtains the current operation area and pre-set operation environment information through wireless communication equipment.

[0043] The operating environment information obtained by the information interaction module includes: total operating range, operating picking range and operating path range. The above information is manually entered by the operator based on the actual planting environment;

[0044] The total operating range is the total area of ​​the picking environment, such as the total area of ​​a planting field or greenhouse. The operating picking range is the range of crop growth in the operating environment, such as crop planting ridges and plant distribution locations. The operating path range is the range within which the picking obstacle avoidance robot can travel, such as reserved field roads, greenhouse roads, or roads reserved for travel between planting areas. The road width should be greater than the width of the picking obstacle avoidance robot.

[0045] The operation-related control information obtained by the information interaction module includes picking conditions and picking areas, wherein the picking conditions are the appearance information of the picked crops, and the picking area is the maximum range of movement of the mechanical picking claw 6 when the picking operation obstacle avoidance robot body is stationary;

[0046] The operation planning module obtains operation-related control information and operation environment information through the information interaction module, and plans the operation path. When planning the path, it generates it through the path generation algorithm in the existing technology;

[0047] After obtaining the working environment information and the working control information, the working planning module overlaps the working picking range and the picking area so that the working picking range is completely covered by the picking area, and marks the overlapping picking areas. Each picking area is used as the working area of ​​the picking operation obstacle avoidance robot. When the picking operation obstacle avoidance robot completes the work of a picking area, it moves to the next picking area again.

[0048] The operation planning module is then limited by the operation path range and picking conditions, and the picking path is automatically generated by the mature path algorithm in the prior art to obtain the initial picking path, wherein the initial picking path is limited to the operation path range, and the initial picking path can cover all marked overlapping areas after passing through the maximum range of motion of the mechanical picking claw 6, that is, the initial picking path connects all picking areas in series;

[0049] After the operation planning module obtains the 3D picture modeling, it determines the position of the picking operation obstacle avoidance robot. When the picking operation obstacle avoidance robot is at the edge of the picking area, the picking crop picture in the 3D picture modeling is extracted and located through the intelligent algorithm. According to the positioning result, the mechanical picking claw 6 is controlled to pick the crops and perform operations in a picking area.

[0050] After completing the picking process in the picking area, the operation planning module controls the obstacle avoidance robot to move to the next picking area.

[0051] The autonomous obstacle avoidance module analyzes obstacles on the operation path in real time through the conversion results of external environment information and adjusts the operation path in real time.

[0052] Example 2: Please refer to Figure 1 - Figure 3 As shown in the figure, when the picking operation obstacle avoidance robot moves to the next picking area, the autonomous obstacle avoidance module analyzes the obstacles on the path through 3D structural modeling and marks the areas where obstacles are located. The picking operation obstacle avoidance robot records the unmarked areas in the 3D structural modeling as obstacle-free areas.

[0053] The autonomous obstacle avoidance module obtains the operating path range through the information interaction module, and compares the overlap between the obstacle-free area and the operating path range. The overlap between the operating path range and the obstacle-free area is recorded as the drivable range. The autonomous obstacle avoidance module compares the drivable range with the set passing size of the picking operation obstacle avoidance robot. If the drivable range is less than or equal to the passing size, it is determined that the robot cannot pass. If the drivable range is greater than the passing size, it is determined that the robot can pass.

[0054] After determining that the path is passable, the autonomous obstacle avoidance module sends the drivable range to the operation planning module. The operation planning module corrects the drivable range in the operation path, so that the picking operation obstacle avoidance robot moves through the drivable range when heading to the next picking area, thus achieving the function of avoiding obstacles.

[0055] At the same time, since the obstacle avoidance route is obtained by correcting the working path and the drivable range, and the working path is obtained based on the working path range, the picking operation obstacle avoidance robot will not exceed the set working path range when moving, thereby avoiding the situation where the picking operation obstacle avoidance robot cannot identify soil ridges or low plants and cause damage;

[0056] After determining that the route is impassable, the autonomous obstacle avoidance module sends a route interruption signal to the operation planning module and sends the impassable location to the operation planning module. The operation planning module marks the impassable location and replans the operation path. When planning the new operation path, the impassable location is isolated, bypassing the impassable location and ensuring that damage to existing plants or soil ridges is avoided.

[0057] During the planning process of a new operation path, if it passes through other picking areas before the picking area that needs to be avoided, then the other picking areas will be given priority for picking, thereby improving the picking efficiency and avoiding frequent detours of the picking operation obstacle avoidance robot to reduce work efficiency. When the newly planned path of the picking operation obstacle avoidance robot cannot bypass the inaccessible position and reach the area that needs to be picked, the inaccessible position will be isolated, and other non-isolated picking areas will be given priority for picking. After the picking is completed, it will reach the inaccessible position again for a second obstacle recognition to confirm whether the obstacle has disappeared or changed, and recalculate whether it can pass.

[0058] The preferred embodiments of the present invention disclosed above are intended only to help illustrate the present invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the present invention to specific embodiments. Obviously, many modifications and variations are possible based on the contents of this specification. These embodiments are selected and described in detail in this specification to better explain the principles and practical applications of the present invention, thereby enabling those skilled in the art to better understand and utilize the present invention. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A picking operation obstacle avoidance robot based on visual recognition, comprising a device body (3), characterized in that: A storage box (1) for storing picked products is installed above the device body (3), a sensor integrated block (2) is fixedly installed in front of the device body (3), a main shaft is provided above the sensor integrated block (2), at least one set of rotating rods (4) is rotatably connected to the main shaft, the ends of the rotating rods (4) are movably connected to a universal ball head (5), and a mechanical picking claw (6) is installed on the universal ball head (5); The main shaft is equipped with the same number of rotating structures as the rotating rods (4), and each rotating mechanism can rotate independently. The sensor integrated block (2) is equipped with at least one set of millimeter wave radars, visible light cameras and a central controller. The central controller includes a picking operation obstacle avoidance robot control system. The equipment body (3), the rotating rod (4), the universal ball head (5) and the mechanical picking claw (6) are controlled by the central controller. The picking operation obstacle avoidance robot control system includes an external information collection module, an operation planning module, an autonomous obstacle avoidance module and an information interaction module; The external information acquisition module senses the external environment information through the sensor integrated block (2), performs digital conversion on the sensed external environment information, and sends the conversion result to the operation planning module and the autonomous obstacle avoidance module; The information interaction module can communicate with the host platform through wireless communication equipment to obtain operation-related control information, and obtain the current operation area and pre-set operation environment information through the wireless communication equipment; The operation planning module obtains operation-related control information and operation environment information through the information interaction module and plans the operation path. The autonomous obstacle avoidance module analyzes the obstacles on the operation path in real time through the external environment information conversion results and adjusts the operation path in real time.

2. The visual recognition-based obstacle avoidance robot for picking operations according to claim 1, characterized in that: The external environment information collected by the external information collection module includes echo reflection information and visible light image information. After obtaining the echo reflection information, the external information collection module constructs an image of the echo reflection information to obtain a 3D structure model; When acquiring visible light images, the external information acquisition module acquires stereoscopic visible light images through a binocular camera, processes the stereoscopic visible light images, and generates 3D image modeling; The external information acquisition module takes 3D structure modeling and 3D picture modeling as conversion results.

3. The visual recognition-based obstacle avoidance robot for picking operations according to claim 1, characterized in that: The operation environment information acquired by the information interaction module includes: the total operation range, the operation picking range and the operation path range; The total operation range is the total area of ​​the picking environment, the operation picking range is the range of crop growth in the operation environment, and the operation path range is the range within which the obstacle avoidance robot can travel during the picking operation. The operation-related control information acquired by the information interaction module includes picking conditions and picking areas, wherein the picking conditions are the appearance information of the picked crops, and the picking area is the maximum range of movement of the mechanical picking claw (6) when the picking operation obstacle avoidance robot body is stationary.

4. The obstacle avoidance robot for picking operations based on visual recognition according to claim 1, characterized in that: After obtaining the operation environment information and operation-related control information, the operation planning module overlaps the operation picking range and the picking area so that the operation picking range is completely covered by the picking area, and marks the overlapping picking area; The operation planning module is then limited by the operation path range and the picking conditions, and the picking path is automatically generated by the algorithm to obtain an initial picking path, wherein the initial picking path is limited within the operation path range, and the initial picking path can cover all marked overlapping areas after being covered by the maximum range of motion of the mechanical picking claw (6).

5. The obstacle avoidance robot for picking operations based on visual recognition according to claim 1, characterized in that: After the operation planning module obtains the 3D image model, it determines the position of the picking operation obstacle avoidance robot. When the picking operation obstacle avoidance robot is at the edge of the picking area, the picking crop image in the 3D image model is extracted and positioned by an intelligent algorithm, and the mechanical picking claw (6) is controlled to pick the crops according to the positioning result.

6. The obstacle-avoiding robot for picking operations based on visual recognition according to claim 1, characterized in that: After the picking of the picking area is completed, the operation planning module controls the picking operation obstacle avoidance robot to move to the next picking area; When the picking operation obstacle avoidance robot moves to the next picking area, the autonomous obstacle avoidance module analyzes the obstacles on the path through 3D structural modeling and marks the areas where obstacles are not marked in the 3D structural modeling. The picking operation obstacle avoidance robot records the unmarked areas in the 3D structural modeling as obstacle-free areas. The autonomous obstacle avoidance module obtains the operating path range through the information interaction module, and compares the overlap between the obstacle-free area and the operating path range, and records the overlapping part of the operating path range and the obstacle-free area as the drivable range. The autonomous obstacle avoidance module compares the drivable range with the set passing size of the picking operation obstacle avoidance robot. If the drivable range is less than or equal to the passing size, it is determined that it cannot pass; if the drivable range is greater than the passing size, it is determined that it can pass.

7. The visual recognition-based obstacle avoidance robot for picking operations according to claim 6, characterized in that: After determining that the autonomous obstacle avoidance module is passable, the drivable range is sent to the operation planning module. The operation planning module corrects the drivable range in the operation path so that the picking operation obstacle avoidance robot moves through the drivable range when heading to the next picking area.

8. The visual recognition-based obstacle avoidance robot for picking operations according to claim 6, characterized in that: After determining that the vehicle cannot pass, the autonomous obstacle avoidance module sends a route interruption signal to the operation planning module and sends the inaccessible location to the operation planning module. The operation planning module marks the inaccessible location and replans the operation path.

Images