Laser weeding method and system

Abstract

The invention discloses a laser weeding method and system based on machine vision. The laser weeding method comprises the steps of obtaining field ridge data in real time to autonomously plan a path, obtaining a crop image in real time, carrying out image recognition and uploading on target weeds, obtaining coordinates of central stalks of the weeds in the target weed image based on a post-processing algorithm, namely a weeding laser burning point, and obtaining a current laser weeding path. And the laser weeding assembly arrives at a given position to complete weeding work. The laser weeding system comprises a moving assembly, and a laser weeding assembly, a control and sensing assembly and an electric power assembly which are arranged on the moving assembly. By means of the visual recognition system and a built-in algorithm, the laser weeding robot has extremely high recognition precision, high laser weeding efficiency, small size, light weight and high flexibility, is suitable for laser weeding operation in various scenes such as mountainous regions, hills and greenhouses, has various operation modes such as remote control and fixed-point cruise, is easy to operate, and is suitable for popularization and application. Operation is fast, maintenance is convenient, and potential safety hazards are reduced to the maximum extent.

Description

Technical Field

[0001] The present invention relates to the field of crop weeding, and in particular to a laser weeding method and system based on machine vision. Background Art

[0002] Weed removal is of great significance for increasing the stable yield of cash crops such as vegetables. In field crop management, it is particularly important to ensure the growth environment of crops. The most critical and difficult to control is weeding. Laser weeding is a very critical step and a future trend.

[0003] Cash crops are widely planted throughout the country. At present, except for farmlands in plain areas such as the three northeastern provinces, Henan Province, and Shandong Province, which have achieved large-scale mechanized operation modes, farmlands in most areas are still dominated by small-scale mechanized operations. In terms of weeding operations on large-scale farmlands, chemical weeding and mechanical weeding are currently the main methods. Among them, chemical weeding does not conform to the current development trend of green agriculture; and although mechanical weeding can remove weeds, it is also easy to damage crops during the operation. As for the rest of the areas, most of them are mainly manual weeding. Although this method has high weeding cleanliness, it is inefficient and costly. With the large-scale loss of young and middle-aged labor in rural areas, the cost of manual weeding will rise year by year. Summary of the invention

[0004] In order to solve the above problems, the present invention provides a laser weeding method and system based on machine vision. The laser weeding method and system involved in the present invention have the characteristics of low cost, low energy consumption, light weight and high flexibility. It has a wide range of applicable areas and good market prospects. It can effectively solve the existing problems in weed removal operations, such as mechanical weeding destroying soil structure, chemical weeding causing great damage to crops, and manual weeding being time-consuming, labor-intensive and inefficient.

[0005] The technical solution adopted by the present invention to solve the technical problem is: Laser weed control methods based on machine vision include: Obtain field ridge data in real time to plan the path autonomously; Acquire crop images in real time, identify and upload images of target weeds; Based on the post-processing algorithm, the coordinates of the central stem of the weeds in the target weed image, i.e., the laser burning point for weeding, are obtained; The current laser weeding path is obtained, so that the laser weeding component reaches a given position to complete the weeding work.

[0006] Furthermore, the post-processing algorithm includes binarization of the weed image, extraction of pixel points at the ends of weed leaves, and fitting of the center of the weed stem to obtain pixel coordinates of the center stem of the weed.

[0007] In order to extract the pixel points at the end of the weed leaf, the curvature of the segmented edge of the leaf contour is calculated. to identify the blade tip.

[0008] In order to obtain the actual coordinates from the pixel coordinates of the central stem of the weed, the distortion parameters and intrinsic matrix of the camera are used to solve the projection transformation, obtain the two-dimensional coordinate system of the target weed relative to the camera, and further convert it to the laser coordinate system of the laser weeding component.

[0009] In order to obtain field ridge data in real time, high-frequency scanning is performed through lidar, and the inter-ridge route data is obtained and uploaded through the camera. Then, autonomous mapping and navigation path planning are realized through SLAM and IMU inertial navigation and attitude fusion algorithms.

[0010] Based on the post-processing algorithm, the coordinates of the central stem of the weeds in the target weed image are obtained, and at the same time, the number of weeds at that moment is obtained; When the number of weeds is less than or equal to 4, a dynamic weeding scheme is adopted, and laser weeding is carried out simultaneously while the mobile component moves along the planned path; when the number of weeds is greater than 4, a static weeding scheme is adopted, the mobile component stops moving, and the annealing method plus the particle swarm algorithm are combined to obtain the optimal weeding path, thereby completing weeding in a short time.

[0011] When using a static weeding solution, to obtain the optimal path for laser weeding: First, the path length of the particle is calculated, and then the exchange sequence (speed) of the particle is calculated through iteration, which represents the exchange operation from the current solution to the individual optimal solution and the global optimal solution, and the exchange sequence is applied to the position of the current particle; Generate the current solution Become an individual optimal solution The exchange sequence and generate the current solution Become the global optimal solution The exchange sequence ; ; Among them, the location ,speed , inertia weight , cognitive learning factor , social learning factor , random number ; Since the exchange sequence cannot be directly multiplied by a real number, the weight is reflected by the number of times the exchange sequence is replicated, and the weight is multiplied by a constant And round it up to get the number of replications;

[0012]

[0013]

[0014]

[0015] After obtaining the new solution path, according to the simulated annealing acceptance criterion, the annealing method needs to compare the energy difference E to determine the acceptance probability P; decide whether to accept the new solution, and may accept a worse solution to escape the local optimum; if the new solution is better, update the individual optimal solution and the global optimal solution.

[0016] The present invention adopts the following technical solutions to solve the technical problems: Laser weeding system, including a moving component and a device disposed thereon A laser weeding assembly, including a laser emitting device; Control and sensing components, including navigation and obstacle avoidance systems, visual recognition systems and control systems.

[0017] Furthermore, the mobile component includes a mobile chassis, which includes a hub motor, a universal wheel, and a frame. The hub motor is connected to a hub connecting steel pipe through a bracket, the universal wheel is connected to the hub connecting steel pipe, and the hub connecting steel pipe is connected to the frame through a connecting profile.

[0018] A stainless steel sleeve and a gasket are provided at the contact point between the hub motor and the bracket to fix the connection between the two.

[0019] Furthermore, the laser weeding assembly is arranged above the mobile chassis through a profile connection, and the laser emitting device includes a dual-CoreXY mobile platform, a slider bearing, a semiconductor laser focusing lens, a synchronous belt, a synchronous pulley, a 42-stepping motor, and a laser head fixing part; the 42-stepping motor controls the rotation of the synchronous pulley so that the synchronous pulley drives the synchronous belt to utilize the dual-CoreXY mobile platform to realize the free movement of the slider bearing in a two-dimensional plane, thereby driving the laser head fixing part to achieve precise positioning.

[0020] Furthermore, the laser emitting device also includes a fill light and a humidity sensor; the fill light is used to provide relatively stable lighting conditions to reduce noise interference, and the humidity sensor is used to detect the ambient humidity in real time to adjust the laser light power of the semiconductor laser focusing lens. When the ambient humidity is too high, the laser light power will be increased, otherwise it will be reduced.

[0021] Furthermore, the laser emitting device also includes a side protection plate and a smoke alarm; the side protection plate is used for isolation and protection to prevent the laser from threatening the personal safety of others, and the smoke alarm is used to monitor the smoke level in the weed removal area in real time. When the smoke density exceeds a safety threshold, a command is issued through the control system to immediately stop laser weeding.

[0022] Furthermore, the navigation and obstacle avoidance system includes a quad-frequency receiving antenna and a multi-line laser radar, which senses the dynamic changes of surrounding obstacles through high-frequency scanning characteristics. When an obstacle appears at a certain distance in front of the driving path, the control system controls the mobile component to stop or change the path to achieve intelligent obstacle avoidance.

[0023] Furthermore, the visual recognition system includes a binocular depth camera and a Raspberry Pi camera to achieve path planning and accurate identification of weeds.

[0024] The binocular depth camera adopts structured light measurement method to obtain the route data between ridges, so as to avoid the radar from planning the wrong route and causing damage to the farmland due to the inability to scan the ground information.

[0025] The Raspberry Pi camera recognizes the target weeds based on the convolutional neural network visual algorithm and uploads the image, and obtains the coordinates of the central stem of the weeds, that is, the laser burning point for weeding, through the control system post-processing algorithm.

[0026] Furthermore, the control system includes a single chip microcomputer, a host computer, a driver and a control display screen.

[0027] Furthermore, the post-processing algorithm includes binarization of the recognition image, extraction of pixel points at the ends of weed leaves, and fitting of the center of weed stems; The weed leaf end pixel extraction is performed by calculating the segmentation edge curvature of the leaf contour. to identify the blade tip; For each point on the contour ( ), the first-order derivative can be approximated as:

[0028]

[0029] The second-order derivative can be approximated as:

[0030]

[0031] For a plane curve, substitute the approximate derivative into the curvature formula:

[0032] The Gaussian function is used as the weight to perform weighted averaging on the points in the neighborhood to obtain the end pixel of the weed leaf. ; The central stem pixel of the weed is obtained by the geometric center calculation method:

[0033]

[0034] Finally, the center point is fitted to obtain the pixel coordinates of the center stem of the weed.

[0035] Furthermore, in order to obtain the actual coordinates from the pixel coordinates of the central stem of the weeds, the control system uses the distortion parameters and internal parameter matrix of the camera to solve the projection transformation, obtains the two-dimensional coordinate system of the target weeds relative to the camera, and further converts it to the semiconductor laser coordinate system of the laser weeding component.

[0036] Furthermore, the control system obtains the number of weeds at that moment while obtaining the coordinates of the central stem of the weeds.

[0037] The control system is based on the number of weeds at a certain moment. When the number of weeds is less than or equal to 4, a dynamic weeding scheme is adopted, and the laser weeding work will be carried out simultaneously when the mobile component moves along the planned path; when the number of weeds is greater than 4, a static weeding scheme is adopted, the mobile component will stop moving, and the annealing method plus the particle swarm algorithm are combined to obtain the optimal weeding path, thereby completing the weeding in a short time.

[0038] Furthermore, the power components include ternary lithium batteries, various switches and fans.

[0039] The beneficial effects brought by the present invention are: The laser weeding system of the present invention has a mobile chassis composed of two wheel hub motors and two universal wheels. The wheel hub motors can be controlled by a single-chip computer program to achieve the purpose of differential steering. At the same time, the deep-grooved wheel hub motors have stronger anti-dumping and obstacle-crossing capabilities to ensure smooth operation in various complex operating scenarios.

[0040] The laser weeding system of the present invention enables the laser weeding robot to accurately identify and remove weeds through the set visual recognition system and the built-in post-processing algorithm; at the same time, the present invention integrates radar and depth camera to perceive the field environment and autonomously plan the weeding path.

[0041] The laser weeding system of the present invention has two mode options during laser weeding operations. The number of weeds on the path is calculated based on a visual recognition system. When the number of weeds is small, a dynamic weeding mode is selected to maintain a high weeding accuracy during the movement of the robot. When the number of weeds is large, a static weeding mode is selected to ensure a high weeding accuracy.

[0042] The present invention is small in size, lightweight, and highly flexible, and is suitable for laser weeding operations in various scenarios such as mountains, hills, and greenhouses. In addition, the present invention has multiple operation modes such as remote control and fixed-point cruise, and is simple to operate, fast to run, highly safe, and easy to maintain. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments: Figure 1 A schematic diagram of a weeding method of the laser weeding robot of the present invention; Figure 2 A schematic diagram of a post-processing algorithm for weed images by the laser weeding robot of the present invention; Figure 3 Schematic diagram of different working modes of laser weeding by the laser weeding robot of the present invention; Figure 4 This is a schematic diagram of the three-dimensional structure of the laser weeding robot of the present invention; Figure 5 It is a structural schematic diagram of the laser weeding actuator of the laser weeding robot of the present invention; Figure 6 This is a schematic diagram of the structure of the wheel set of the laser weeding robot of the present invention; Figure 7 This is a schematic diagram of the structure of the mobile chassis of the laser weeding robot of the present invention; Figure 8 It is a structural schematic diagram of the navigation and obstacle avoidance system of the laser weeding robot of the present invention; Fig. 9 It is a structural schematic diagram of the visual recognition system of the laser weeding robot of the present invention; Fig.10 This is a schematic diagram of the structure of the control system of the laser weeding robot of the present invention; Fig.11 This is a schematic diagram of the structure of the power system of the laser weeding robot of the present invention; Fig.12 The figure is a schematic diagram of the structure of the semiconductor laser and its focusing lens of the laser weeding robot of the present invention. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] Moreover, the following description is for the purpose of illustration rather than limitation, and specific details such as specific system structures, data models, etc. are provided to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, models, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0046] Reference Figure 1 This embodiment relates to a laser weeding method based on machine vision, the method comprising: Obtain field ridge data in real time to plan the path autonomously; Acquire crop images in real time, identify and upload images of target weeds; Based on the post-processing algorithm, the coordinates of the central stem of the weeds in the target weed image, i.e., the laser burning point for weeding, are obtained; Get the current laser weeding path, so that the laser weeding component reaches a given position to complete the weeding work; A new round of laser weeding operations is replanned until all weed plants identified and located within the machine's field of view are successfully removed.

[0047] In this embodiment, in order to obtain the field ridge data in real time, the surrounding obstacle data is detected by high-frequency scanning through the quad-frequency receiving antenna 311 and the multi-line laser radar 312, and the inter-ridge route data is obtained through the binocular depth camera 321 and the data is uploaded to the control system, and then the autonomous running map construction and navigation path planning are realized through the SLAM and IMU inertial navigation and attitude fusion algorithm, so as to achieve unmanned automatic decision-making. The autonomous planning path of the weeding robot can be viewed through the rviz software in the host computer system, and if the planning is wrong, it can be corrected manually. In this embodiment, the visual camera is integrated into the path planning process to effectively identify the inter-ridge route, so as to avoid the possibility of damage to farmland caused by the radar because it cannot scan the ground information and thus plans the path incorrectly.

[0048] The laser weeding robot wheel motor 11 moves forward, making it travel along the planned path. The Raspberry Pi camera 322 located above the laser weeding actuator 2 and facing the farmland collects images in real time and uploads them to the robot control system 33. Then the host computer 332 completes the identification and center positioning of the weeds.

[0049] First, the target weeds are identified and segmented based on a convolutional neural network visual algorithm to obtain a weed leaf segmentation image; the convolutional neural network visual algorithm includes the training, construction and application of a convolutional neural weed model, wherein the convolutional neural weed model is not limited to one; Subsequently, the post-processing algorithm was used to obtain the pixel coordinates of the central stem of the weeds. Figure 2 , specifically including: Binarization of weed images, extraction of pixel points at the end of weed leaves, and fitting of the center of weed stems are performed to obtain the pixel coordinates of the center stems of weeds.

[0050] In this embodiment, in order to extract the pixel points at the end of the weed leaf, the curvature of the segmented edge of the leaf contour is calculated. To identify the blade tip: For each point on the contour ( ), the first-order derivative can be approximated as:

[0051]

[0052] The second-order derivative can be approximated as:

[0053]

[0054] For a plane curve, substitute the approximate derivative into the curvature formula:

[0055] The curvature calculated at this time There will be a large error. In order to find the pixel point at the end of the leaf more accurately, first, the leaf contour data needs to be smoothed before calculating the derivative and curvature to reduce the impact of noise on the calculation results and improve the accuracy of curvature calculation. Second, use the Gaussian function as the weight to perform weighted averaging on the points in the neighborhood, with closer points having a larger weight and farther points having a smaller weight. Gaussian function is the standard deviation of the Gaussian distribution, which determines the degree of smoothing, is the index of the point in the neighborhood, is the window size. In this embodiment, Take it as 1.0, Take it as 7:

[0056]

[0057]

[0058] At this time, the end pixel of each leaf is obtained; Then the central stem pixel of the weed is obtained by the geometric center calculation method (the coordinate set of the end pixel points of all leaves is , N is the number of blades):

[0059]

[0060] Finally, the center point is fitted to obtain the pixel coordinates of the center stem of the weed.

[0061] In order to obtain the actual coordinates from the pixel coordinates of the central stem of the weed, it is necessary to use the camera's distortion parameters and intrinsic parameter matrix to solve the projection transformation, obtain the two-dimensional coordinate system of the target weed relative to the camera, and further convert it to the laser coordinate system of the laser weeding component.

[0062] In a preferred embodiment, the coordinates of the central stem of the weeds in the target weed image are obtained based on the post-processing algorithm, and at the same time, the number of weeds in the image at that moment is obtained; Reference Figure 3 When the number of weeds is less than or equal to 4, a dynamic weeding scheme is adopted. The laser weeding work is carried out while the mobile chassis 1 moves according to the planned path, and laser weeding is performed by the Monte Carlo method; when the number of weeds is greater than 4, a static weeding scheme is adopted. The mobile chassis 1 will stop moving, and the annealing method plus the particle swarm algorithm are combined to obtain the optimal weeding path, thereby completing the weeding in a short time.

[0063] When using a static weeding solution, to obtain the optimal path for laser weeding: First, the path length of the particle is calculated, and then the exchange sequence (speed) of the particle is calculated through iteration, which represents the exchange operation from the current solution to the individual optimal solution and the global optimal solution, and the exchange sequence is applied to the position of the current particle; Generate the current solution Become an individual optimal solution The exchange sequence and generate the current solution Become the global optimal solution The exchange sequence ; ; Among them, the location ,speed , inertia weight , cognitive learning factor , social learning factor , random number ; Since the exchange sequence cannot be directly multiplied by a real number, the weight is reflected by the number of times the exchange sequence is replicated, and the weight is multiplied by a constant And round it up to get the number of replications;

[0064]

[0065]

[0066]

[0067] After obtaining the new solution path, according to the simulated annealing acceptance criterion, the annealing method needs to compare the energy difference E to determine the acceptance probability P; decide whether to accept the new solution, and may accept a worse solution to escape the local optimum; if the new solution is better, update the individual optimal solution and the global optimal solution.

[0068] For static and dynamic weeding schemes, the evaluation of the number of weeds is based on experiments and simulations. When the number of weeds is less than 4, the dynamic weeding scheme can still achieve the weeding effect.

[0069] During the laser weeding process, the Raspberry Pi camera 322 obtains the relative position of the weed plants in real time. If the weeds are not successfully removed due to problems with the strength and path planning of the laser weeding actuator 2 components, the laser weeding actuator 2 will perform a new round of weed removal operations under the re-planning of the robot control system 33 until the weed plants located within the camera's field of view are successfully removed.

[0070] Reference Figure 4 Another embodiment of the present invention relates to a laser weeding system based on machine vision, namely a laser weeding robot, comprising a mobile chassis 1 and a A laser weeding actuator 2, which is connected to and arranged above the mobile chassis 1 through a profile 21; Control and sensing components 3, including navigation and obstacle avoidance system 31, visual recognition system 32 and robot control system 33; The power system 4 includes a middle acrylic plate 41, a battery pack, various switches 44 and a fan 45. The power system 4 is arranged in the middle space above the mobile chassis 1 to provide energy support for the mobile chassis 1, the laser weeding actuator 2 and the control and sensor component 3 of the robot; the robot control system 33 is arranged on the middle acrylic plate 41.

[0071] This embodiment exemplarily applies the laser weeding robot to remove weeds in the seedling stage of greenhouse crops, such as the removal of purslane, but the application of the present invention is not limited to this.

[0072] Reference Figure 6 , Figure 7The mobile assembly includes a mobile chassis 1, which includes a hub motor 11, a universal wheel 12, a frame 13, a connection profile 14, a hub connection steel pipe 15, a hub motor bracket 16, a stainless steel sleeve 17, a gasket 18, a long fixing plate 19, and a short fixing plate 110. The hub motor 11 is connected to the hub connection steel pipe 15 through the hub motor bracket 16; the universal wheel 12 is connected to the hub connection steel pipe 15; the hub connection steel pipe 15 is connected to the frame 13 through the connection profile 14; the stainless steel sleeve 17 and the gasket 18 are arranged at the contact between the hub motor 11 and the hub motor bracket 16 to fix the connection between the two.

[0073] Reference Figure 5 The laser weeding component is a laser weeding actuator 2, which includes a profile 21, a left fixed angle 22, a right fixed angle 23, a middle fixing part 24, a slider bearing 25, a semiconductor laser focusing lens 26, a synchronous belt 27, a synchronous pulley 28, a 42 stepper motor 29, and a laser head fixing part 210; the laser weeding actuator 2 is connected to the mobile chassis 1 through the profile 21, and the 42 stepper motor 29 controls the rotation of the synchronous pulley 28, so that the synchronous pulley 28 drives the synchronous belt 27 to use the dual CoreXY mobile platform to realize the free movement of the slider bearing 25 in a two-dimensional plane, and drives the laser head fixing part 210 to achieve precise positioning. Fig.12 As shown, the semiconductor laser focusing lens 26 includes a laser, a collimating lens, and a focusing lens.

[0074] In a preferred embodiment, in order to ensure the high efficiency of the laser, the laser weeding actuator 2 also includes a fill light 211 and a humidity sensor 336; the fill light 211 is used to provide a relatively stable lighting condition to reduce noise interference, and the humidity sensor 336 is used to detect the ambient humidity in real time to adjust the laser optical power of the semiconductor laser focusing lens 26. When the ambient humidity is too high, the laser optical power will be increased, and vice versa. The purpose of saving electricity is achieved by adjusting the laser power, and the two cooperate with each other to maximize the efficiency of laser weeding.

[0075] In a preferred embodiment, in order to reduce safety hazards, the laser weeding actuator 2 also includes a side protection plate 212 and a smoke alarm 335; the side protection plate 212 is used for isolation and protection to prevent the laser from threatening the personal safety of others, and the smoke alarm 335 is used to monitor the smoke level in the weed removal area in real time. When the smoke density exceeds the safety threshold, the control system will issue an order to immediately stop the laser weeding and upload the information to the base station staff, thereby minimizing the fire hazard.

[0076] Reference Figure 8The navigation and obstacle avoidance system 31 includes a quad-frequency receiving antenna 311, a multi-line laser radar 312, a fixing 313, and a top acrylic plate 314. The quad-frequency receiving antenna 311 and the multi-line laser radar 312 are fixed on the fixing 313, and the fixing 313 is fixed on the top acrylic plate 314. This system senses the dynamic changes of surrounding obstacles through high-frequency scanning characteristics. When an obstacle appears at a certain distance in front of the driving path, the robot control system 33 controls the mobile chassis 1 to stop or change the path to achieve intelligent obstacle avoidance. At the same time, the integration of visual cameras effectively identifies the routes between ridges to avoid the damage to farmland caused by the radar's inability to scan ground information and thus incorrectly planning the path.

[0077] Also refer to Fig. 9 The visual recognition system 32 includes a binocular depth camera 321 and a Raspberry Pi camera 322 to achieve path planning and accurate identification of weeds. The visual recognition system 32 is arranged below the middle acrylic plate 41 and on the top profile.

[0078] The binocular depth camera 321 adopts structured light measurement method with an accuracy of up to 0.1mm and a measurement range of 0 to 3m. It has a resolution of 1080x720 and is used to obtain image data of the route between ridges to prevent the radar from incorrectly planning the route due to the inability to scan the ground information and causing damage to the farmland.

[0079] After the Raspberry Pi camera 322 collects the field image, it recognizes the target weeds based on the convolutional neural network visual algorithm and uploads the image. The coordinates of the central stem of the weeds, that is, the laser burning point for weeding, are obtained through the control system post-processing algorithm. Then, under the control of the robot control system 33, the stepper motor 29 is driven 42 to adjust the laser path of the semiconductor laser focusing lens 26, so that the laser weeding actuator 2 reaches a given position to complete the weeding work.

[0080] Reference Fig.10 The robot control system 33 includes a single chip microcomputer 331, a host computer 332, a driver (TB6600) 333, and a control display screen 334.

[0081] In order to obtain the pixel coordinates and location of the central stem of the weed, the robot control system 33 mainly extracts the pixel points at the end of the weed leaf by binarizing the recognition image. The pixel extraction of this end is the most important part of the post-processing algorithm, which can be obtained by calculating the segmented edge curvature of the leaf. The end pixel is obtained. Since the end of the purslane weed is very sharp, its The larger the value, the better. Therefore, the post-processing algorithm includes the binarization of the recognition image, the extraction of the end pixel points of the weed leaves, and the center fitting of the weed stems, wherein the extraction of the end pixel points of the weed leaves is performed by calculating the segmentation edge curvature of the leaf contour. to identify the blade tip.

[0082] In a preferred embodiment, the robot control system 33 obtains the number of weeds at that moment while obtaining the coordinates of the central stem of the weeds. When the number of weeds is small, a dynamic weeding scheme is selected to maintain a high weeding accuracy during the movement of the robot. When the number of weeds is large, a static weeding scheme is selected to ensure a high weeding accuracy. After the weeds are successfully removed, the laser weeding robot will perform the next round of weeding operations under the control of the control and sensor component 3.

[0083] Reference Fig.11 The power assembly 4 includes a middle acrylic plate 41, a 24V 50A·h ternary lithium battery 42, a semiconductor laser 43, various switches 44 and a fan 45. The power system 4, except for the switch, is arranged in the interlayer space between the middle acrylic plate 41 and the top acrylic plate 314. The 24V 50A·h ternary lithium battery 42 and the semiconductor laser 43 are both arranged on the middle acrylic plate 41. Various switches 44 and fans 45 are respectively arranged above and below the top acrylic plate 314. The function of the fan 45 is to timely cool down the devices in the interlayer space to ensure the normal operation of the robot.

[0084] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned relevant functional units and modules is used as an example for illustration. In practical applications, some functions can be assigned to different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above. The relevant functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the relevant functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working principles of the units and modules in the above-mentioned system can refer to the corresponding processes in the aforementioned method embodiments, which will not be repeated here.

[0085] Another embodiment of the present invention relates to an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the relevant steps of the machine vision-based laser weeding method when executing the computer program. Alternatively, the processor implements the functions of the relevant modules in the above-mentioned machine vision-based laser weeding system embodiment when executing the computer program.

[0086] Exemplarily, the computer program may be divided into one or more modules / units, one or more modules / units are stored in a memory and executed by a processor to implement the present invention. One or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, and the instruction segments are used to describe the execution process of the computer program in an electronic device.

[0087] The electronic device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The electronic device may include, but is not limited to, a processor, a memory, for example, the electronic device may also include an input / output device, a network access device, a bus, etc.

[0088] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0089] The memory may be an internal storage unit of an electronic device, such as a hard disk or memory of the electronic device. The memory may also be an external storage device of the electronic device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device. Furthermore, the memory may include both an internal storage unit of the electronic device and an external storage device. The memory is used to store computer programs and other programs and data required by the electronic device, and may also be used to temporarily store data that has been output or is to be output.

[0090] Another embodiment of the present invention relates to a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the computer program implements the relevant steps of the machine vision-based laser weeding method. Alternatively, when the computer program is executed by a processor, the computer program implements the functions of the relevant modules in the above-mentioned machine vision-based laser weeding system embodiment.

[0091] The computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. Computer readable media may include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0092] It should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A laser weeding method based on machine vision, characterized in that: include Obtain field ridge data in real time to plan the path autonomously; Acquire crop images in real time, identify and upload images of target weeds; Based on the post-processing algorithm, the coordinates of the central stem of the weeds in the target weed image, that is, the laser burning point for weeding, are obtained; the post-processing algorithm includes binarization of the weed image, extraction of pixel points at the end of the weed leaves, and fitting of the center of the weed stem to obtain the pixel coordinates of the central stem of the weeds; The current laser weeding path is obtained, so that the laser weeding component reaches a given position to complete the weeding work.

2. The laser weeding method according to claim 1, characterized in that: In order to extract the pixel points at the end of the weed leaf, the curvature of the segmented edge of the leaf contour is calculated. to identify the blade tip.

3. The laser weeding method according to claim 2, characterized in that: In order to obtain the actual coordinates from the pixel coordinates of the central stem of the weed, the distortion parameters and intrinsic matrix of the camera are used to solve the projection transformation, obtain the two-dimensional coordinate system of the target weed relative to the camera, and further convert it to the laser coordinate system of the laser weeding component.

4. The laser weeding method according to claim 1, characterized in that: Based on the post-processing algorithm, the coordinates of the central stem of the weeds in the target weed image are obtained, and at the same time, the number of weeds at that moment is obtained; When the number of weeds is less than or equal to 4, a dynamic weeding scheme is adopted, and laser weeding is carried out simultaneously while the mobile component moves along the planned path; when the number of weeds is greater than 4, a static weeding scheme is adopted, the mobile component stops moving, and the annealing method plus the particle swarm algorithm are combined to obtain the optimal weeding path, thereby completing weeding in a short time.

5. Laser weeding system based on machine vision, characterized by: include Mobile components and components mounted thereon A laser weeding assembly, including a laser emitting device; The control and sensing components include a navigation and obstacle avoidance system, a visual recognition system and a control system; the visual recognition system recognizes the target weeds based on a convolutional neural network visual algorithm and uploads images, and obtains the coordinates of the central stem of the weeds, that is, the laser burning point for weeding, through a post-processing algorithm of the control system; the post-processing algorithm includes binarization processing of the recognition image, extraction of pixel points at the end of the weed leaves, and fitting of the center of the weed stem.

6. The laser weeding system according to claim 5, characterized in that: The laser weeding assembly is connected to the mobile chassis through a profile, and the laser emitting device includes a dual-CoreXY mobile platform, a slider bearing, a semiconductor laser focusing lens, a synchronous belt, a synchronous pulley, a stepper motor, and a laser head fixing part; the stepper motor controls the rotation of the synchronous pulley so that the synchronous pulley drives the synchronous belt to utilize the dual-CoreXY mobile platform to realize the free movement of the slider bearing in a two-dimensional plane, thereby driving the laser head fixing part to realize precise positioning.

7. The laser weeding system according to claim 6, characterized in that: The laser emitting device also includes a fill light and a humidity sensor; the fill light is used to provide a relatively stable lighting condition to reduce noise interference, and the humidity sensor is used to detect the ambient humidity in real time to adjust the laser optical power of the semiconductor laser focusing lens. When the ambient humidity is too high, the laser optical power will be increased, otherwise it will be reduced. The laser emitting device also includes a side protection plate and a smoke alarm; the side protection plate is used for isolation and protection to prevent the laser from threatening the personal safety of others, and the smoke alarm is used to monitor the smoke level in the weed removal area in real time. When the smoke density exceeds the safety threshold, the control system issues a command to immediately stop laser weeding.

8. The laser weeding system according to claim 5, characterized in that: The navigation and obstacle avoidance system includes a quad-frequency receiving antenna and a multi-line laser radar, which sense the dynamic changes of surrounding obstacles through high-frequency scanning characteristics. When an obstacle appears at a certain distance in front of the driving path, the control system controls the mobile component to stop or change the path to achieve intelligent obstacle avoidance.

9. The laser weeding system according to claim 5, characterized in that: The weed leaf end pixel extraction is performed by calculating the segmentation edge curvature of the leaf contour. to identify the blade tip.

10. The laser weeding system according to claim 5, characterized in that: The control system obtains the coordinates of the central stem of the weeds and the number of weeds at that moment; The control system is based on the number of weeds at a certain moment. When the number of weeds is less than or equal to 4, a dynamic weeding scheme is adopted, and the laser weeding work will be carried out simultaneously when the mobile component moves along the planned path; when the number of weeds is greater than 4, a static weeding scheme is adopted, the mobile component will stop moving, and the annealing method plus the particle swarm algorithm are combined to obtain the optimal weeding path, thereby completing the weeding in a short time.

Images