Movable agricultural image information acquisition robot with variable light source and processing method thereof
By introducing variable light source and robotic arm technology into agricultural image inspection robots, the problem of insufficient adaptability of robots under variable terrain and lighting conditions is solved, high-quality crop image acquisition and analysis is achieved, and production efficiency and cost-effectiveness are improved.
Patent Information
- Application Number
- CN202510067870.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-16
- Publication Date
- 2025-04-18
AI Technical Summary
The existing agricultural image inspection robots are not adaptable under varying terrain and ambient lighting conditions, and cannot meet the needs of crop image acquisition and analysis under various angles and light conditions.
A mobile agricultural image information acquisition robot with variable light sources is designed, using variable light source and robotic arm technology to realize crop image acquisition and analysis under various ground environments and lighting conditions by adjusting the angle of the camera and the brightness of the light source.
High-quality crop image acquisition and analysis under complex farmland terrain and variable lighting environments are achieved, which improves the adaptability and imaging quality of image acquisition, and reduces labor costs and management costs.
Smart Images
Figure CN119847163A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a mobile agricultural image information acquisition robot with a variable light source and a processing method thereof, belonging to the technical field of a mobile agricultural image information acquisition robot with a variable light source and a processing method thereof. Background Art
[0002] With the continuous development of productivity and the continuous upgrading of agricultural equipment, agricultural production has been developing on a large scale. At the same time, the complexity of farmland environment has been increasing, the variety of crops has increased, and the demand for refined agricultural management has been expanding. These have led to the increasing complexity of agricultural production information and the rapid increase in the labor cost of agricultural information collection and management systems.
[0003] In order to solve the above problems, agricultural image inspection robots have been rapidly developed. Agricultural image inspection robots use automatic navigation chassis to move to the designated location, and then cooperate with the onboard camera to collect and analyze images. This method is applied to the daily management of agricultural production, and gradually replaces manual operations, and conducts all-round image analysis and monitoring of crops. It can monitor crops in real time, provide data support for precision agriculture, save a lot of manpower and management costs, and improve production efficiency.
[0004] However, there are still some problems with the existing agricultural image inspection robots: First, they are not adaptable enough to the changing terrain of farmland. At present, many agricultural image inspection robots are only suitable for use in greenhouses or indoor test areas with relatively flat ground. Most of them use track guidance, or use SLAM navigation with fixed maps under stable environmental conditions, which cannot meet the navigation requirements of multi-terrain spaces between fields. Second, they are not adaptable enough to changes in ambient lighting conditions. The illumination of farmland in clear weather is approximately between 24,000 and 100,000 lux. Under such a large difference in illumination, images taken under conventional flashlights cannot meet the subsequent precise analysis and processing of image acquisition data results. Third, the vertical angle range of image acquisition is insufficient. General image inspection robots use mobile chassis or track movement to change the viewing angle for image acquisition. This method can basically achieve 360° horizontal viewing angle coverage. However, the movement of the vertical viewing angle of image acquisition is very limited, which makes the robot unable to adapt well to scenes of crops in different growth states. Summary of the invention
[0005] The technical problem to be solved by the present invention is to overcome the defects of the prior art and provide a mobile agricultural image information acquisition robot with a variable light source and a processing method thereof, which can complete mobile navigation traction in a variety of ground environments. At the same time, the present invention can adjust the angle of the camera and the brightness of the light source to meet the needs of completing crop image acquisition and analysis under a variety of angles and a variety of illumination conditions.
[0006] Preferably, the present invention provides a method for processing mobile agricultural image information with a variable light source, comprising: Step 1, based on the predetermined area to be inspected in the farmland, calculate and obtain the spatial coordinate position of the area to be inspected; Step 2: Plan and obtain the navigation route of the mobile agricultural image information collection robot to obtain the spatial coordinate position of the area to be inspected; Step 3, using the mobile agricultural image information collection robot to move to the spatial coordinate position of the area to be inspected; converting the camera data of the camera of the mobile agricultural image information collection robot into 3D point cloud data; Step 4, use the camera of the mobile agricultural image information collection robot to obtain camera photos of the area to be inspected; evaluate the imaging quality of the camera photos of the area to be inspected, if the imaging quality does not meet the preset imaging quality conditions, adjust the light source brightness and the image acquisition angle of the camera of the mobile agricultural image information collection robot based on the image lighting adjustment strategy; repeat step 4 until the imaging quality of the camera photos of the area to be inspected meets the preset imaging quality conditions.
[0007] Preferably, step 4 further comprises: If the image quality meets the preset image quality conditions, a camera photo of the area to be inspected is obtained.
[0008] Prioritize, all camera photos of the areas that need to be inspected are analyzed and processed, including fruit maturity detection and pest and disease analysis, to obtain test results.
[0009] Preferably, the preset imaging quality conditions include: the contrast of the camera photo is higher than the preset image contrast, the clarity of the camera photo is higher than the preset clarity threshold, and the number of pixels with maximum RGB values in the camera photo does not exceed a preset number.
[0010] Preferably, image lighting adjustment strategies include: If the contrast of the camera photo is lower than a preset image contrast or the clarity is lower than a preset clarity threshold, then the brightness of the light source is increased before the next camera photo is taken; If the distance between the RGB pixel area of the camera photo and the center of the camera photo is greater than a preset RGB pixel area distance threshold, adjusting the camera angle before taking the camera photo next time so that the RGB pixel area of the camera photo moves toward the center of the camera photo; If the number of pixels with maximum RGB values in the camera photo exceeds the preset number, the brightness of the light source is reduced.
[0011] Preferably, the camera data includes RGB pixel information as well as depth information.
[0012] Preferably, a mobile agricultural image information acquisition robot with a variable light source is used to execute any one of the mobile agricultural image information processing methods with a variable light source, and comprises a vehicle body, a chassis control module, a mechanical arm control module, a light source control module, an image acquisition module, a battery management module, a driving module, a navigation module, an angle adjustment module and a human-computer interaction module. The chassis control module, the robotic arm control module, the light source control module, the image acquisition module, the battery management module, the drive module, the navigation module, the angle adjustment module and the human-computer interaction module are all installed on the vehicle body. The chassis control module is electrically connected to the robotic arm control module, the light source control module, the image acquisition module, the battery management module, the drive module, the navigation module, the angle adjustment module, the safety module and the human-computer interaction module. The battery management module is electrically connected to the robotic arm control module, the chassis control module, the light source control module, the image acquisition module, the battery management module, the drive module, the navigation module, the angle adjustment module and the human-computer interaction module.
[0013] Preferably, the chassis control module includes an STM32F407 controller and a development board electrically connected to the STM32F407 controller, the human-computer interaction module is connected to the development board, and the navigation module is connected to the development board; The robotic arm control module includes an ESP32 chip, and the ESP32 chip is electrically connected to the development board; The light source control module includes a light source circuit board, and the light source circuit board is electrically connected to the development board; The image acquisition module includes a camera, which is electrically connected to the STM32F407 controller; The battery management module includes a power distribution board, a battery and a battery coulomb sensor, the battery coulomb sensor is electrically connected to the STM32F407 controller, and the battery is electrically connected to the STM32F407 controller, the ESP32 chip, the light source circuit board and the camera through the power distribution board; The safety module includes an electrical protection element, a mechanical protection element, a safety soft limit in a chassis control module and a safety soft limit in a robotic arm control module, and the electrical protection element and the mechanical protection element are electrically connected to the STM32F407 controller; The drive module comprises a servo motor driver, a first servo motor connected to the servo motor driver, a second servo motor, a mechanical driven device, master and slave wheels and a crawler chassis, the crawler chassis is mounted on the vehicle body, the mechanical driven device (7) and the master and slave wheels (8) are mounted on the crawler chassis, and the servo motor driver is connected to the development board; The navigation module includes a receiving unit and a planning unit; The receiving unit includes a WIFI router and an antenna rod connected to the WIFI router, and the WIFI router is electrically connected to the development board; The planning unit includes a 3D laser radar, an IMU posture sensor and a GPS positioning sensor, and the 3D laser radar, the IMU posture sensor and the GPS positioning sensor are electrically connected to the STM32F407 controller; The angle adjustment module includes a first mechanical arm steering gear, a second mechanical arm steering gear and a mechanical arm control module, the second mechanical arm steering gear is vertically arranged on the vehicle body, the first mechanical arm steering gear is horizontally arranged on the second mechanical arm steering gear, and the camera is arranged on the first mechanical arm steering gear; The human-computer interaction module includes an intelligent display screen, which is electrically connected to the development board; It also includes an emergency stop button, which is electrically connected to the STM32F407 controller.
[0014] Preferably, the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the methods described in the first aspect when executing the program.
[0015] Preferably, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the methods described in the first aspect.
[0016] The beneficial effects achieved by the present invention are: The present invention controls the output mode of the light source control module to ensure that the light source in the light source control module can effectively cover the key parts of the crop in various complex environments. By adjusting the shape, material and light distribution of the light source, a more uniform lighting effect can be achieved, and shadow and reflection problems can be reduced. The present invention reasonably designs the installation position of the light source and the image lighting adjustment strategy, so that the light source can be adaptively adjusted according to environmental conditions and detection requirements, further improving the performance and stability of agricultural image acquisition. The present invention starts 3D laser radar, IMU posture sensor and GPS positioning sensor when the robot moves to obtain multimodal data for data fusion. Based on the LIV-SAM framework, the present invention uses a method for automatically adjusting the weight of data factors in complex farmland terrain scenes for specific farmland environments, thereby improving the estimation accuracy and robustness of navigation, so that the robot's navigation traction can adapt to the complex terrain environment of various farmlands.
[0017] The present invention adopts a combined positioning method for the agricultural image acquisition equipment and the fill-light equipment, utilizes the all-terrain navigation of the crawler chassis in conjunction with the posture adjustment of the mechanical arm, and realizes the full-angle posture adjustment of the camera within a certain three-dimensional space, which can effectively expand the image acquisition angle range and the fill-light angle range, thereby improving the imaging quality of agricultural images.
[0018] The present invention autonomously adjusts the brightness and angle of the LED light source so that the light source can adapt to the illumination changes in the crop environment in real time. Through image acquisition under different environments, the image quality of the system under different lighting conditions is tested, and the clarity and contrast of the camera photos under different lighting conditions are analyzed. The effects of different image processing algorithms are compared, and the processing speed, processing accuracy and adaptability of the image processing algorithm under different ambient light are analyzed, so as to determine the illumination of the current environment and the imaging quality of the camera photos, and adjust the brightness of the light source according to the image illumination adjustment strategy of the camera photos: if the contrast of the camera photo is lower than the preset image contrast or the clarity of the camera photo is lower than the preset clarity threshold, it is necessary to increase the brightness of the light source when the camera photos are repeatedly collected next time; if the quality of the camera photos is reduced due to overexposure, it is necessary to reduce the brightness of the light source when the camera photos are repeatedly collected next time; if the distance between the RGB pixel area of the camera photo and the center of the camera photo is greater than the preset RGB pixel area distance threshold, the robot arm is driven to adjust the camera angle when the camera photos are repeatedly collected next time, so that the RGB pixel area of the camera photo moves toward the center of the camera photo, thereby improving the imaging quality of agricultural image acquisition under different lighting environments. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0020] Figure 1 is a flow chart of some embodiments of the present application; Figure 2 is an isometric view of some embodiments of the present application.
[0021] Figure 3 is a cross-sectional view of some embodiments of the present application; Figure 4 is a principle block diagram of some embodiments of the present application; Figure 5 is a principle block diagram of the system layer, software layer and hardware layer in some embodiments of the present application; The meaning of the accompanying drawings is: 1-camera; 2-first robotic arm servo; 3-second robotic arm servo; 4-robotic arm control module; 5-GPS positioning sensor; 6-3D laser radar; 7-mechanical slave device; 8-master and slave wheels; 9-antenna rod; 10-display screen; 11-emergency stop button; 12-first servo motor; 13-second servo motor; 14-power distribution board; 15-chassis control module; 16-WIFI router; 17-development board; 18-expansion memory disk; 19-battery; 20-IMU posture sensor; 21-battery coulomb sensor; 22-light source control module. DETAILED DESCRIPTION
[0022] See also Figure 4 , the present application discloses a mobile agricultural image information acquisition robot with a variable light source, including a chassis control module 15, a mechanical arm control module 4, a light source control module 22, an image acquisition module, a battery management module, a safety module, a drive module, a navigation module, an angle adjustment module and a human-computer interaction module; See also Figure 3 , the chassis control module 15 includes an STM32F407 controller, and a development board 17 electrically connected to the STM32F407 controller, the human-computer interaction module is connected to the development board 17, and the navigation module is connected to the development board 17; The robotic arm control module 4 includes an ESP32 chip, and the ESP32 chip is electrically connected to the development board 17; The light source control module 22 includes a light source circuit board, and the light source circuit board is electrically connected to the development board 17; The image acquisition module includes a camera 1, which is electrically connected to the STM32F407 controller; The battery management module includes a power distribution board 14, a battery 19 and a battery coulomb sensor 21, the battery coulomb sensor 21 is electrically connected to the STM32F407 controller, and the battery 19 is electrically connected to the STM32F407 controller, the ESP32 chip, the light source circuit board and the camera 1 through the power distribution board; The safety module includes an electrical protection element, a mechanical protection element, a safety soft limit in a chassis control module and a safety soft limit in a robotic arm control module, and the electrical protection element and the mechanical protection element are electrically connected to the STM32F407 controller; The driving module includes a servo motor driver, a first servo motor 12 connected to the servo motor driver, a second servo motor 13 connected to the servo motor driver, a mechanical follower 7, master and slave wheels 8 and a crawler chassis. The mechanical follower 7 and the master and slave wheels 8 are installed on the crawler chassis, and the servo motor driver is connected to the chassis control module.
[0023] The navigation module includes a receiving unit and a planning unit; the receiving unit includes a WIFI router 16, and an antenna rod 9 connected to the WIFI router 16, and the WIFI router 16 is electrically connected to the development board 17; see Figure 2 , the planning unit includes a 3D laser radar 6, an IMU posture sensor 20 and a GPS positioning sensor 5, and the 3D laser radar 6, the IMU posture sensor 20 and the GPS positioning sensor 5 are electrically connected to the STM32F407 controller; The angle adjustment module includes a first mechanical arm steering gear 2, a second mechanical arm steering gear 3 and a mechanical arm control module 4, the second mechanical arm steering gear 3 is vertically arranged on the vehicle body, the first mechanical arm steering gear 2 is horizontally arranged on the second mechanical arm steering gear 3, and the camera 1 is arranged on the first mechanical arm steering gear 2; The human-computer interaction module includes a smart display screen 10, and the smart display screen 10 is electrically connected to a development board 17; It also includes an emergency stop button 11, which is electrically connected to the STM32F407 controller. The display screen 10, the development board 17 and the expansion memory disk 18 can all adopt the Raspberry Pi brand.
[0024] The mobile agricultural image information acquisition robot with a variable light source described in the present invention has an obstacle crossing mechanism on the robot body. When the robot inspects farmland crops, the tracked chassis realizes overall movement through the cooperation of the navigation module and the driving module. The first servo motor and the second servo motor drive the mechanical driven device and the tracked chassis to rotate, which can realize obstacle crossing in various complex terrains and improve the adaptability of the robot in different farmland environments.
[0025] like Figure 5 As shown, the method for processing mobile agricultural image information with a variable light source described in the present invention adopts the Ubuntu system ROS2-Galactic development environment and the Ubuntu system Anaconda development environment at the system level; in the Ubuntu system ROS2-Galactic development environment, the chassis control module is used to obtain chassis posture information and environmental information, and the environmental information includes the current world coordinates of the robot, the point cloud data output by the laser radar, and the chassis posture information obtained by the IMU posture sensor. The laser radar analyzes the reflected energy size, the amplitude, frequency and phase of the reflected spectrum of the target object surface and then outputs the point cloud data, thereby presenting the accurate three-dimensional structure information of the target object, thereby constructing the map information.
[0026] At the software level, SLAM is used to build maps. Based on the chassis posture information and environmental information, the information collected by 3D lidar, IMU posture sensor and GPS positioning sensor is used to fuse the data factor graph; based on the fused data after factor graph optimization, Navigation2 navigation is used for navigation path planning; based on the angle of the light source control module, the robot arm control module is used to obtain the robot arm posture information, and Movet2 planning is used for robot arm path planning and adjustment of the robot arm posture; in the Anaconda development environment of the Ubuntu system, the light source control module is used to obtain lighting parameters, the camera takes photos to obtain image information, and SINet is used to detect light and dark areas in camera photos to generate lighting fill light strategies.
[0027] When the robot moves, it starts the 3D laser radar, IMU posture sensor and GPS positioning sensor to obtain multimodal data and perform data fusion. Based on the LIV-SAM framework, a method of automatically adjusting the weight of data factors in complex farmland terrain scenes is used for specific farmland environments to improve the estimation accuracy and robustness of navigation, so that the robot's navigation and traction can adapt to the complex terrain environment of various farmlands.
[0028] A combined positioning method is adopted for the agricultural image acquisition equipment and fill-light equipment. The all-terrain navigation of the tracked chassis is used in conjunction with the posture adjustment of the robotic arm to achieve full-angle posture adjustment of the camera within a certain three-dimensional space. This can effectively expand the image acquisition angle range and fill-light angle range, thereby improving the imaging quality of agricultural images.
[0029] The present invention adopts a self-developed light source control algorithm to adjust the brightness of the LED light source and the angle of the camera to verify whether the light source can adapt to the light changes in the crop environment in real time. Through image acquisition in different environments, the image quality of the system under different lighting conditions is tested, and the image clarity and contrast under different lighting conditions are analyzed. The effects of different image processing algorithms are compared, and the processing speed, accuracy and adaptability of the image processing algorithm under different ambient light are analyzed in detail, so as to determine the illumination of the current environment and the imaging quality of the camera photo, and adjust the lighting strategy according to the imaging quality of the camera photo: if the contrast of the camera photo is lower than the preset image contrast or the clarity of the camera photo is lower than the preset clarity threshold, it is necessary to increase the brightness of the light source when the camera photo is repeatedly collected next time; if the image quality is reduced due to overexposure, specifically, if the number of pixels with the maximum RGB value in the camera photo exceeds the preset number, it is necessary to reduce the brightness of the light source when the camera photo is repeatedly collected next time.
[0030] If the distance between the RGB pixel area of the camera photo and the center of the camera photo is greater than the preset RGB pixel area distance threshold, the robotic arm is driven to adjust the camera angle before taking the camera photo next time, so that the RGB pixel area of the current camera photo moves toward the center of the current camera photo, thereby improving the imaging quality of agricultural image acquisition under different lighting environments.
[0031] The autonomous navigation crawler chassis and robotic arm technology of the mobile agricultural image information collection robot with variable light source provide greater flexibility for the implementation of active lighting technology. In facility agriculture monitoring, the crawler chassis can autonomously navigate and move to the farmland area that needs to be inspected, and the robotic arm can flexibly adjust the position of the camera and light source, and can also adjust the lighting strategy according to real-time monitoring data, so that crops in different parts can be fully inspected and illuminated, reducing detection blind spots and errors. Through the robot's autonomous mobile navigation, intelligent detection of large areas of farmland can be achieved: through the precise operation of the robotic arm, the system can adapt to the needs of different crop types and different growth stages in facility agriculture, and realize more intelligent and precise agricultural monitoring.
[0032] Active lighting technology can adjust the brightness, angle and position of the light source in real time according to the actual lighting conditions of the crop, thereby effectively compensating for insufficient light in light and dark areas and improving the contrast and clarity of the image. This technology can help facility agriculture monitoring systems cope with complex and changing lighting environments, improve the impact of light and dark areas on automated detection, improve the quality of image acquisition, and improve the accuracy of monitoring crop health status.
[0033] like Figure 1 As shown, the overall operation process of the present invention is as follows: (1) Start robot initialization: After the system is powered on and self-check is completed, press the start button and the robot starts working. The system determines whether safety conditions are met, including normal battery voltage, normal battery current, normal battery temperature and normal communication heartbeat, and then initializes the system.
[0034] (2) Select the inspection area: Select the area to be inspected in the farmland on the human-computer interaction interface of the display screen.
[0035] (3) Navigation planning: Based on the selected patrol area, the system calculates the spatial coordinate position of the patrol area that the tracked chassis needs to reach; based on the real-time map built by 3D lidar, IMU posture sensor, GPS positioning sensor and multimodal data fusion, the system plans navigation for the tracked chassis.
[0036] (4) Robotic arm positioning: After the crawler chassis reaches the spatial coordinate position of the specified patrol area, the camera data is converted into 3D point cloud data. The camera data includes RGB pixel information and depth information, and the angle that needs to be adjusted is calculated; Robotic arm positioning: Plan the movement of the first servo motor and the second servo motor so that the camera is positioned at the specified angle.
[0037] (5) Light detection: The system detects the ambient light at the current camera position and angle, uses the camera to take pictures to obtain images, and evaluates the image quality.
[0038] (6) Photo strategy adjustment: Based on the image quality under the current lighting, use the image lighting adjustment strategy to adjust the light source brightness in the light source control module and the image acquisition angle of the camera.
[0039] (7) Image acquisition: Use the camera to acquire images and obtain camera photos.
[0040] (8) Data transmission: Transmitting collected camera photos.
[0041] (9) Data analysis and processing: By analyzing and processing camera photos, fruit maturity detection and pest and disease analysis can be performed based on different crops to obtain data processing results.
[0042] (10) Feedback on monitoring results: Feedback on data processing results and provide monitoring results.
[0043] In a specific embodiment, the mobile agricultural image information acquisition robot with a variable light source needs to be powered on and initialized first. The development board will send an initialization instruction to the chassis control module electrically connected to it, and then the chassis control module will send a self-test instruction to the safety module and the battery management module to check the external sensor signals and the communication status of the internal actuators.
[0044] After initialization, the mobile agricultural image information acquisition robot enters the standby state. The chassis control module controls the first servo motor and the second servo motor of the driving module to lock the shaft, thereby locking the driven mechanical device and the crawler chassis. The robotic arm control module controls the first servo and the second servo to lock the shaft, thereby locking the first joint and the second joint of the robotic arm, thereby fixing the position of the image acquisition module.
[0045] At this time, the 3D laser radar in the navigation module starts to scan the point cloud information of the surrounding environment and initially establishes the SLAM environment model, and sends the map information back to the industrial control board. At the same time, the IMU posture sensor starts to work, measuring the three-axis acceleration, three-axis angle and three-axis gyroscope of the mobile agricultural image information acquisition robot, thereby calculating the current posture of the robot and sending the posture information back to the development board. At the same time, the GPS positioning sensor starts to work, and according to the satellite positioning data, it calculates the current world coordinates of the mobile agricultural image information acquisition robot and sends it back to the development board.
[0046] Subsequently, the above three types of data are fused, and the current navigation environment is detected through these data. Then, the factor weights of these data are adjusted through the factor graph optimization principle to complete the data fusion optimization. Then, accurate odometer coordinates can be formed in the ROS2 system under the Linux environment, and the robot's crawler chassis can obtain accurate navigation map data in various complex farmland environments.
[0047] Then, specific navigation drive instructions are generated according to the optimized navigation map data. The ROS2 system sets the appropriate speed and acceleration, generates relative displacement values according to the target area coordinates, and then generates the predetermined encoder value of the servo motor and publishes it to the chassis control module, which then decodes and issues the instructions. At this time, the first servo motor and the second servo motor of the drive module are decoupled, and the first servo motor and the second servo motor are driven according to the speed and displacement instructions generated by the system, thereby driving the main and driven wheels and the driven mechanical device, and then driving the crawler chassis in contact with the ground to achieve navigation movement.
[0048] When the robot's crawler chassis navigation moves to the designated area, the chassis control module controls the first servo motor and the second servo motor of the drive module to lock the shaft, and the robotic arm control module controls the first servo and the second servo to release the shaft. At the same time, it sends motion instructions to run the robotic arm servo, adjusts the camera to a head-on viewing angle, and controls the camera to take the first photo of the crops.
[0049] Image processing algorithms are run on camera photos of crops to automatically detect light and dark areas of crops in camera photos after enhancement, denoising, and light balancing. Dark areas that need to be illuminated are quickly identified through edge detection, contrast enhancement, and deep learning technology. Based on optical principles, the optimal illumination angle and range are determined according to the light propagation characteristics of the light source and the position coordinates of the robotic arm, and then the three-dimensional coordinates of a camera target position and the optimized illumination parameter information of the variable light source are calculated.
[0050] Then, based on the three-dimensional coordinates of the camera's target posture, it is estimated whether the current coordinate posture of the crawler chassis of the mobile agricultural image information acquisition robot meets the robotic arm adjustment strategy. If so, it proceeds to the next process; if not, the chassis control module controls the first servo motor and the second servo motor of the drive module to decouple the axes, and enters the navigation process again to generate the chassis coordinate range of the robotic arm's reachable posture, and calculates and navigates to the predetermined chassis coordinate range.
[0051] Then the first servo motor and the second servo motor of the driving module lock the shaft, the robotic arm control module controls the first servo and the second servo to release the shaft, and at the same time the chassis control module sends a motion command to run the robotic arm servo, so that the camera adjusts the angle and reaches the position of the target three-dimensional coordinates after fill light optimization.
[0052] The light source control module is then run according to the optimized lighting parameters. The light source control module adjusts the illuminance and light distribution of the active light source to achieve a more uniform lighting effect, reduce the shadow and reflection problems of the captured crop images, and thus provide sufficient lighting for the key areas of the crop and reduce image distortion caused by shadows and reflections.
[0053] According to the optimized angle and active lighting fill light, the image acquisition module controls the camera to re-acquire the optimized camera photos and evaluate the image quality of the optimized camera photos again. If the requirements are not met, the above image acquisition optimization process is executed again to obtain higher quality camera photos.
[0054] In an embodiment of the present application, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the above methods when executing the program.
[0055] In an embodiment of the present application, the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any of the above methods are implemented.
[0056] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.
[0057] Those skilled in the art will readily appreciate other embodiments of the invention after considering the specification and practicing the invention invented herein. This application is intended to cover any variations, uses or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art that are not invented by the present invention. The specification and examples are intended to be exemplary only, and the true scope and spirit of the invention are indicated by the claims.
[0058] The above specific implementation methods further illustrate the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above are only specific implementation methods of the present application and are not used to limit the protection scope of the present application. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present application should be included in the protection scope of the present application.
Claims
1. A mobile agricultural image information processing method with a variable light source, characterized in that: include: Step 1, based on the predetermined area to be inspected in the farmland, calculate and obtain the spatial coordinate position of the area to be inspected; Step 2: Plan and obtain the navigation route of the mobile agricultural image information collection robot to obtain the spatial coordinate position of the area to be inspected; Step 3: Use the mobile agricultural image information collection robot to move to the spatial coordinate position of the area that needs to be inspected; Converting camera data of a camera (1) of a mobile agricultural image information collection robot into 3D point cloud data; Step 4, using the camera (1) of the mobile agricultural image information collection robot to obtain camera photos of the area to be inspected; evaluating the imaging quality of the camera photos of the area to be inspected, and if the imaging quality does not meet the preset imaging quality condition, adjusting the light source brightness of the mobile agricultural image information collection robot and the image acquisition angle of the camera (1) based on the image lighting adjustment strategy; repeating step 4 until the imaging quality of the camera photos of the area to be inspected meets the preset imaging quality condition.
2. The mobile agricultural image information processing method of variable light source according to claim 1 is characterized in that: Step 4 also includes: If the image quality meets the preset image quality conditions, a camera photo of the area to be inspected is obtained.
3. The mobile agricultural image information processing method of variable light source according to claim 2 is characterized in that: All camera photos of the areas that need to be inspected are analyzed and processed, including fruit maturity detection and pest and disease analysis, to obtain the test results.
4. The mobile agricultural image information processing method with a variable light source according to claim 1 is characterized in that: The preset imaging quality conditions include: the contrast of the camera photo is higher than the preset image contrast, the clarity of the camera photo is higher than the preset clarity threshold, and the number of pixels with maximum RGB values in the camera photo does not exceed the preset number.
5. The mobile agricultural image information processing method of variable light source according to claim 1 is characterized in that: Image lighting adjustment strategies, including: If the contrast of the camera photo is lower than a preset image contrast or the clarity is lower than a preset clarity threshold, then the brightness of the light source is increased before the next camera photo is taken; If the distance between the RGB pixel area of the camera photo and the center of the camera photo is greater than a preset RGB pixel area distance threshold, adjusting the angle of the camera (1) before taking the camera photo next time so that the RGB pixel area of the camera photo moves toward the center of the camera photo; If the number of pixels with maximum RGB values in the camera photo exceeds the preset number, the brightness of the light source is reduced.
6. The mobile agricultural image information processing method of variable light source according to claim 1, characterized in that: The camera data includes RGB pixel information as well as depth information.
7. A mobile agricultural image information collection robot with variable light source, characterized in that: The method for processing mobile agricultural image information using a variable light source according to any one of claims 1 to 6 comprises a mechanical arm control module (4), a chassis control module (15), a light source control module (22), a vehicle body, an image acquisition module, a battery management module, a drive module, a navigation module, an angle adjustment module and a human-computer interaction module. The mechanical arm control module (4), the chassis control module (15), the light source control module (22), the image acquisition module, the battery management module, the drive module, the navigation module, the angle adjustment module and the human-machine interaction module are all mounted on the vehicle body; the chassis control module (15) is electrically connected to the mechanical arm control module (4), the light source control module (22), the image acquisition module, the battery management module, the drive module, the navigation module, the angle adjustment module, the safety module and the human-machine interaction module; and the battery management module is electrically connected to the mechanical arm control module (4), the chassis control module (15), the light source control module (22), the image acquisition module, the battery management module, the drive module, the navigation module, the angle adjustment module and the human-machine interaction module.
8. The mobile agricultural image information acquisition robot with variable light source according to claim 7, characterized in that: The chassis control module (15) comprises an STM32F407 controller and a development board (17) electrically connected to the STM32F407 controller, the human-computer interaction module is connected to the development board (17), and the navigation module is connected to the development board (17); The robotic arm control module (4) includes an ESP32 chip, and the ESP32 chip is electrically connected to the development board (17); The light source control module (22) comprises a light source circuit board, and the light source circuit board is electrically connected to the development board (17); The image acquisition module comprises a camera (1), and the camera (1) is electrically connected to an STM32F407 controller; The battery management module comprises a power distribution board (14), a battery (19) and a battery coulomb sensor (21), the battery coulomb sensor (21) being electrically connected to the STM32F407 controller, and the battery (19) being electrically connected to the STM32F407 controller, the ESP32 chip, the light source circuit board and the camera (1) via the power distribution board; The safety module includes an electrical protection element, a mechanical protection element, a safety soft limit in a chassis control module and a safety soft limit in a robotic arm control module, and the electrical protection element and the mechanical protection element are electrically connected to the STM32F407 controller; The drive module comprises a servo motor driver, a first servo motor (12) connected to the servo motor driver, a second servo motor (13) connected to the servo motor driver, a mechanical driven device (7), master and slave wheels (8) and a crawler chassis, wherein the crawler chassis is mounted on the vehicle body, the mechanical driven device (7) and the master and slave wheels (8) are mounted on the crawler chassis, and the servo motor driver is connected to a development board (17); The navigation module includes a receiving unit and a planning unit; The receiving unit comprises a WIFI router (16) and an antenna rod (9) connected to the WIFI router (16); the WIFI router (16) is electrically connected to a development board (17); The planning unit comprises a 3D laser radar (6), an IMU posture sensor (20) and a GPS positioning sensor (5), and the 3D laser radar (6), the IMU posture sensor (20) and the GPS positioning sensor (5) are electrically connected to an STM32F407 controller; The angle adjustment module comprises a first mechanical arm steering gear (2), a second mechanical arm steering gear (3) and a mechanical arm control module (4), the second mechanical arm steering gear (3) being arranged vertically on the vehicle body, the first mechanical arm steering gear (2) being arranged horizontally on the second mechanical arm steering gear (3), and the camera (1) being arranged on the first mechanical arm steering gear (2); The human-computer interaction module comprises an intelligent display screen (10), and the intelligent display screen (10) is electrically connected to a development board (17); It also includes an emergency stop button (11), which is electrically connected to the STM32F407 controller.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 8 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.