Autonomous laser weed eradication

Abstract

Disclosed herein are methods, devices, modules, and systems that can be employed for automated weed identification, control, and eradication. These methods, devices, modules, and systems provide an alternative to manual cultivation or chemical herbicides. The devices disclosed herein can be configured to locate, identify, and autonomously target weeds using beams, such as laser beams, that can burn or irradiate the weeds. The methods, devices, modules, and systems can be used for agricultural crop management or for home weed control.

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application claims the benefit of U.S. Provisional Application No. 62 / 901,641, filed Sep. 17, 2019, which application is incorporated herein by reference in its entirety. [Background technology]

[0002] Agricultural production is worth trillions of dollars annually worldwide. Agriculture is an essential component of food production and involves the raising and cultivation of both livestock and plants. Declining crop yields due to population growth and a changing climate threaten global food security. Methods to increase agricultural production by improving crop yields and increasing labor efficiency could help alleviate food shortages. Summary of the Invention [Means for solving the problem]

[0003] The present disclosure provides various methods, devices, modules, and systems that can be employed for automatic identification, maintenance, control, or targeting of plants. For example, the methods, devices, modules, and systems disclosed herein can be used to autonomously identify and eradicate weeds located within a field of crops. The methods, devices, modules, and systems can be used as an alternative to manual cultivation or chemical herbicides. For example, the methods, devices, modules, and systems can be used for agricultural crop management or for home weed control.

[0004] In various aspects, the present disclosure provides an optical control system comprising: an emitter configured to emit a beam along an optical path toward a target location on a surface, the target location being determined by autonomously locating a target on the surface; a first reflective element positioned to intersect the optical path and deflect the beam; a first targeting actuator connected to the first reflective element and configured to rotate the first reflective element and deflect the beam toward the target location; and a combining element positioned in the optical path between the emitter and the first reflective element and configured to differentially deflect the beam and scattered light from the target location traveling along the optical path in a direction opposite to the beam.

[0005] In some aspects, the optical control system further comprises a targeting camera optically coupled to the combining element and configured to receive the scattered light reflected from the first reflective element and image a targeting field of view including the target location. In some aspects, the optical control system is configured to direct the beam toward the target location while the optical control system is moving relative to the surface. In some aspects, the optical control system further comprises a targeting system computer configured to detect pixel movement of the targeting field of view relative to the target location and translate from pixel movement of the targeting field of view to a rotation of the first reflective element.

[0006] In some aspects, the conversion from pixel translation to rotation of the first reflective element includes referencing a calibration function. In some aspects, the calibration function is obtained by correlating locations of reference markers on a calibration surface with camera pixel translations.

[0007] In some aspects, the optical control system further comprises an inertial measurement unit coupled to the optical control system, the inertial measurement unit configured to measure an acceleration of the optical control system, a rotation of the optical control system relative to the surface, or a combination thereof. In some aspects, the targeting system computer is configured to adjust the target location based on the amount of time since imaging, the acceleration of the optical control system, the rotation of the optical control system relative to the surface, or a combination thereof.

[0008] In some aspects, the optical control system is enclosed within an enclosure, the enclosure including an escape window capable of transmitting the emitted and visible light and positioned in the optical path between the first reflective element and the surface. In some aspects, the optical control system is fully enclosed within the enclosure. In some aspects, the optical control system further includes an air source configured to direct the air flow. In some aspects, the enclosure further includes a wall opposite the opening configured to control the direction of the air flow and reduce turbulence without obstructing the beam.

[0009] In some aspects, the first reflective element is a mirror. In some aspects, the combining element transmits the beam and reflects visible light. In some aspects, the emitter is a laser emitter. In some aspects, the laser emitter is selected from the group consisting of an infrared laser, an ultraviolet laser, and a visible laser. In some aspects, the optical control system further comprises a second targeting actuator connected to the first reflective element and configured to rotate the first reflective element and deflect the beam toward the target location. In some aspects, the optical control system further comprises a second reflective element positioned to intersect the optical path and deflect the beam deflected by the first reflective element, and a second targeting actuator connected to the second reflective element and configured to rotate the second reflective element and deflect the beam toward the target location. In some aspects, the first targeting actuator deflects the beam along a first axis and the second targeting actuator deflects the beam along a second axis, the first axis and the second axis being orthogonal. In some aspects, the combining element is positioned after the emitter, the first reflective element is positioned after the combining element, and the second reflective element is positioned after the first reflective element, relative to the direction of the beam. In some aspects, weeds are positioned at the target location.

[0010] In various aspects, the present disclosure provides a method for weed eradication that includes capturing an image of a predicted field of view with a predictive camera, locating a target within the predicted field of view, assigning the target to one of a plurality of targeting modules having a targeting camera with a targeting field of view that overlaps with the target location, capturing an image of the targeting field of view with the targeting camera, locating the target within the targeting field of view, and directing a beam toward the target location.

[0011] In some aspects, locating the target within the predicted field of view further includes identifying a location of the target within the predicted field of view. In some aspects, the weed eradication method further includes identifying an area including the target, the area being defined by a polygon. In some aspects, the weed eradication method further includes converting the location to a predicted surface location. In some aspects, the weed eradication method further includes determining a desired movement within the targeted field of view. In some aspects, the weed eradication method further includes converting the desired movement to an actuator position change. In some aspects, locating the target includes identifying the target using a trained neural net. In some aspects, the trained neural net can provide a bounding box, a polygonal mask, or a combination thereof around the target. In some aspects, the trained neural net is trained using images of a field.

[0012] In some aspects, locating the target within the targeting field of view further includes referencing a calibration function obtained by correlating a location of a fiducial marker on the calibration surface with a camera pixel coordinate and correcting the target location. In some aspects, assigning the target to one of the plurality of targeting modules includes providing the target location to one of the plurality of targeting modules. In some aspects, directing the beam toward the target location further includes referencing a calibration function obtained by correlating a pixel movement of a fiducial marker on the calibration surface with an actuator tilt value and correcting the actuator tilt value. In some aspects, the weed eradication method further includes deactivating the beam once the target is damaged or killed.

[0013] In some aspects, capturing an image of a targeting field of view with a targeting camera, locating a target within the targeting field of view, and directing a beam toward the target location are performed with high accuracy. In some aspects, the target is a weed.

[0014] In some aspects, the weed eradication method further comprises damaging or killing the weeds. In some aspects, damaging or killing the weeds comprises irradiating the weeds. In some aspects, damaging or killing the weeds comprises burning the weeds. In some aspects, locating the target comprises distinguishing between weeds and desired plants.

[0015] In various aspects, the present disclosure provides a targeting system comprising a prediction module, a targeting module, and an optical control module, the prediction module comprising a prediction camera configured to image a predicted field of view on a surface and identify a location of a target within the predicted field of view, and a prediction module controller configured to convert a location of the target within the predicted field of view to a predicted location on the surface and assign the target to the targeting module, the targeting module comprising a targeting module controller configured to convert the predicted location to a position of a targeting actuator, the optical control module comprising an emitter configured to emit a beam along an optical path toward the target, and a targeting actuator configured to receive position information from the targeting module controller and deflect the beam toward the target.

[0016] In some aspects, the targeting system further comprises a targeting camera configured to image the targeting field on the surface and identify a location of the target within the targeting field. In some aspects, the optical control module further comprises a first reflective element controlled by the targeting actuator and positioned to intersect the optical path and deflect the beam, and a combining element positioned in the optical path between the emitter and the first reflective element and configured to differentially deflect the beam and scattered light from the targeting field traveling along the optical path in a direction opposite to the beam.

[0017] In some aspects, the optical control module is configured to direct the beam towards the target while the targeting system is moving relative to the surface. In some aspects, the targeting module is configured to detect pixel movement of the targeting field of view relative to the target and translate from pixel movement of the targeting field of view to motion of the targeting actuator.

[0018] In some aspects, the targeting system further comprises an inertial measurement unit configured to measure the acceleration of the targeting system and the rotation of the targeting system relative to the surface. In some aspects, the targeting module is configured to adjust the predicted location based on the amount of time since imaging, the acceleration of the targeting system, the rotation of the targeting system relative to the surface, or a combination thereof. In some aspects, the targeting system further comprises a second targeting module comprising a second targeting camera configured to image a second targeting field on the surface and identify a location of the target within the second targeting field, and a targeting module controller configured to convert the location of the target within the second targeting field to a position of the second targeting actuator. In some aspects, the predicted field of view comprises the targeting field of view.

[0019] In some aspects, the targeting system further comprises a vehicle carrying the predictive camera and the optical control module. In some aspects, the vehicle is an autonomous vehicle. In some aspects, the vehicle comprises multiple wheels.

[0020] In some aspects, the optical control module is enclosed within an enclosure, the enclosure being capable of transmitting the emitted and visible light and including an escape window positioned in the optical path between the first reflecting element and the surface. In some aspects, the optical control module is fully enclosed within the enclosure. In some aspects, the targeting system further includes an air source configured to direct an air flow from an opening in an exterior surface of the enclosure toward an exterior surface of the escape window. In some aspects, the enclosure further includes a wall opposite the opening configured to control the direction of the air flow and reduce turbulence without obstructing the beam.

[0021] In some aspects, the first reflective element is a mirror. In some aspects, the combining element transmits the beam and reflects visible light. In some aspects, the emitter is a laser emitter. In some aspects, the laser emitter is selected from the group consisting of an infrared laser, an ultraviolet laser, and a visible laser. In some aspects, the optical control module further comprises a second targeting actuator connected to the first reflective element and configured to rotate the first reflective element and deflect the beam toward the target. In some aspects, the optical control module further comprises a second reflective element positioned to intersect the optical path and deflect the beam deflected by the first reflective element, and a second targeting actuator connected to the second reflective element and configured to rotate the second reflective element and deflect the beam toward the target. In some aspects, the first targeting actuator deflects the beam along a first axis and the second targeting actuator deflects the beam along a second axis, the first axis and the second axis being orthogonal. In some aspects, the combining element is positioned after the emitter, the first reflective element is positioned after the combining element, and the second reflective element is positioned after the first reflective element, relative to the direction of the beam.

[0022] Additional aspects and advantages of the present disclosure will become readily apparent to those skilled in the art from the following detailed description, in which only illustrative embodiments of the present disclosure are shown and described. As will be realized, the present disclosure is capable of other and different embodiments, and its several details are capable of modification in various obvious respects, all without departing from the present disclosure. Thus, the drawings and description are to be regarded as illustrative in nature, and not restrictive. The present invention provides, for example, the following: (Item 1) 1. An optical control system, comprising: an emitter configured to emit a beam along an optical path toward a target location on a surface, the target location being determined by autonomously locating a target on the surface; a first reflective element positioned to intersect the optical path and deflect the beam; a first targeting actuator coupled to the first reflective element, the first targeting actuator configured to rotate the first reflective element to deflect the beam towards the target location; a combining element positioned in the optical path between the emitter and the first reflective element; Equipped with The combining element is configured to differentially deflect the beam and scattered light from the target location traveling along the optical path in a direction opposite to the beam. (Item 2) 2. The optical control system of claim 1, further comprising a targeting camera optically connected to the combining element, the targeting camera configured to receive the scattered light reflected from the first reflective element and image a targeted field of view including the target location. (Item 3) 3. The optical control system of claim 1 or 2, wherein the optical control system is configured to direct the beam towards the target location while the optical control system is moving relative to the surface. (Item 4) The optical control system of any one of items 1-3, further comprising a targeting system computer configured to detect pixel movement of the targeting field of view relative to the target location and convert from pixel movement of the targeting field of view to rotation of the first reflective element. (Item 5) 5. The optical control system of claim 4, wherein the conversion from the pixel movement to the rotation of the first reflective element includes referencing a calibration function. (Item 6) 6. The optical control system of item 5, wherein the calibration function is obtained by correlating the location of a reference marker on a calibration surface with camera pixel movement. (Item 7) 7. The optical control system of any one of items 1-6, further comprising an inertial measurement unit coupled to the optical control system, the inertial measurement unit configured to measure acceleration of the optical control system, rotation of the optical control system relative to the surface, or a combination thereof. (Item 8) 8. The optical control system of claim 7, wherein the targeting system computer is configured to adjust the target location based on an amount of time since imaging, an acceleration of the optical control system, a rotation of the optical control system relative to the surface, or a combination thereof. (Item 9) The optical control system of any one of items 1-8, wherein the optical control system is enclosed within an enclosure, the enclosure having an escape window capable of transmitting the emitted and visible light, the escape window being positioned within the optical path between the first reflective element and the surface. (Item 10) 10. The optical control system of claim 9, wherein the optical control system is completely enclosed within the enclosure. (Item 11) 11. The optical control system of claim 9 or 10, further comprising an air source configured to direct an air flow from an opening in an outer surface of the enclosure toward an exterior surface of the escape window. (Item 12) 12. The optical control system of any one of claims 9-11, wherein the enclosure further comprises a wall opposite the opening, the wall configured to control the direction of the air flow and reduce turbulence without obstructing the beam. (Item 13) 13. The optical control system of any one of claims 1-12, wherein the first reflective element is a mirror. (Item 14) 14. The optical control system of any one of claims 1-13, wherein the combining element transmits the beam and reflects the visible light. (Item 15) 15. The optical control system of any one of claims 1-14, wherein the emitter is a laser emitter. (Item 16) Item 16. The optical control system of item 15, wherein the laser emitter is selected from the group consisting of an infrared laser, an ultraviolet laser, and a visible laser. (Item 17) 17. The optical control system of any one of items 1-16, further comprising a second targeting actuator connected to the first reflective element, the second targeting actuator configured to rotate the first reflective element and deflect the beam toward the target location. (Item 18) 17. The optical control system of any one of items 1-16, further comprising a second reflective element positioned to intersect the optical path and deflect the beam deflected by the first reflective element, and a second targeting actuator connected to the second reflective element, the second targeting actuator configured to rotate the second reflective element and deflect the beam toward the target location. (Item 19) 19. The optical control system of claim 17 or 18, wherein the first targeting actuator deflects the beam along a first axis and the second targeting actuator deflects the beam along a second axis, the first axis and the second axis being orthogonal. (Item 20) 20. An optical control system according to any one of items 1-19, wherein, relative to the direction of the beam, the combining element is positioned after the emitter, the first reflective element is positioned after the combining element, and the second reflective element is positioned after the first reflective element. (Item 21) 21. The optical control system of any one of items 1-20, wherein weeds are positioned at the target location. (Item 22) A method for eradicating weeds, the method comprising: Capturing an image of the predicted field of view using a predictive camera; Locating a target within a predicted field of view; assigning the target to one of a plurality of targeting modules, the one including a targeting camera having a targeting field of view that overlaps with a location of the target; capturing an image of the targeted field of view with the targeted camera; Locating the target within the targeting field of view; directing a beam toward said target location; A method for eradicating weeds, comprising: (Item 23) 23. The weed eradication method according to claim 22, wherein locating the target within the predicted field of view further comprises identifying the location of the target within the predicted field of view. (Item 24) 24. The method of weed eradication according to claim 22 or 23, further comprising identifying an area containing the target, the area being defined by a polygon. (Item 25) 25. The weed eradication method according to any one of claims 22-24, further comprising converting the position to a predicted surface location. (Item 26) 26. The weed eradication method according to any one of claims 22-25, further comprising determining a desired movement within the targeted field of view. (Item 27) 27. The weed eradication method of claim 26, further comprising translating the desired movement into an actuator position change. (Item 28) 28. The method for eradicating weeds according to any one of claims 22-27, wherein locating the target comprises using a trained neural net to identify the target. (Item 29) 29. The weed eradication method according to claim 28, wherein the trained neural net is capable of providing a bounding box, a polygonal mask, or a combination thereof around the target. (Item 30) 30. The method for eradicating weeds according to claim 28 or 29, wherein the trained neural network is trained using images of fields. (Item 31) A weed eradication method according to any one of items 22-30, wherein identifying the position of the target within the targeting field of view further includes referencing a calibration function, the calibration function being obtained by correlating the location of a reference marker on a calibration surface with a camera pixel coordinate and correcting the location of the target. (Item 32) 32. The weed eradication method according to any one of claims 22-31, wherein assigning the target to one of the plurality of targeting modules comprises providing a location of the target to one of the plurality of targeting modules. (Item 33) 33. The weed eradication method of any one of claims 22-32, wherein directing the beam towards the target location further comprises referencing a calibration function, the calibration function being obtained by correlating pixel movements of a reference marker on a calibration surface with an actuator tilt value and correcting the actuator tilt value. (Item 34) 34. The weed eradication method according to any one of claims 22-33, further comprising deactivating the beam once the target has been damaged or killed. (Item 35) Item 35. The weed eradication method according to any one of items 22-34, wherein the capturing of an image of the targeting field of view with the targeting camera, the locating of the target within the targeting field of view, and the directing of a beam toward the location of the target are performed with a high degree of accuracy. (Item 36) 36. The method for eradicating weeds according to any one of items 22-35, wherein the target is a weed. (Item 37) 37. The method for eradicating weeds according to item 36, further comprising damaging or killing the weeds. (Item 38) Item 38. The method for eradicating weeds according to item 37, wherein damaging or killing the weeds comprises irradiating the weeds. (Item 39) Item 37. The method for eradicating weeds according to item 36, wherein damaging or killing the weeds comprises burning the weeds. (Item 40) 40. The method for eradicating weeds according to any one of items 36-39, wherein identifying the location of the target comprises distinguishing the weeds from desired plants. (Item 41) 1. A targeting system, comprising: a prediction module; a targeting module; and an optical control module; The prediction module includes: a predictive camera configured to image a predicted field of view on a surface and identify a location of a target within the predicted field of view; a prediction module controller configured to convert the location of the target within the predicted field of view to a predicted location on the surface and assign the target to the targeting module; Equipped with the targeting module comprises a targeting module controller configured to convert the predicted location into a position of a targeting actuator; The optical control module includes: an emitter configured to emit a beam along an optical path toward the target; a targeting actuator configured to receive position information from the targeting module controller and to deflect the beam towards the target; A targeting system comprising: (Item 42) 42. The targeting system of claim 41, further comprising a targeting camera configured to image a targeting field on the surface and identify a location of the target within the targeting field. (Item 43) The optical control module includes: a first reflective element controlled by the targeted actuator and positioned to intersect the optical path and deflect the beam; a combining element positioned in the optical path between the emitter and the first reflective element; Further equipped with 43. The targeting system of claim 41 or 42, wherein the combining element is configured to differentially deflect the beam and scattered light from the targeted field of view traveling along the optical path in an opposite direction to the beam. (Item 44) 44. The targeting system of any one of claims 41-43, wherein the optical control module is configured to direct the beam towards the target while the targeting system is moving relative to the surface. (Item 45) 45. The targeting system of any one of claims 41-44, wherein the targeting module is configured to detect pixel movement of the targeting field of view relative to the target and convert the pixel movement of the targeting field of view into movement of the targeting actuator. (Item 46) 46. ​​The targeting system of any one of claims 41-45, further comprising an inertial measurement unit configured to measure acceleration of the targeting system and rotation of the targeting system relative to the surface. (Item 47) 47. The targeting system of any one of claims 41-46, wherein the targeting module is configured to adjust the predicted location based on an amount of time since imaging, an acceleration of the targeting system, a rotation of the targeting system relative to the surface, or a combination thereof. (Item 48) and a second targeting module, the second targeting module comprising: a second targeting camera configured to image a second targeting field on the surface and identify a location of the target within the second targeting field; a targeting module controller configured to convert a location of the target within the second targeting field of view into a position of a second targeting actuator; 48. The targeting system of any one of items 41-47, comprising: (Item 49) 49. The targeting system of any one of claims 41-48, wherein the predicted field of view comprises the targeting field of view. (Item 50) 50. The targeting system of any one of claims 41-49, further comprising a vehicle carrying the predictive camera and the optical control module. (Item 51) 51. The targeting system of claim 50, wherein the vehicle is an autonomous vehicle. (Item 52) 52. The targeting system of claim 50 or 51, wherein the vehicle is equipped with a plurality of wheels. (Item 53) 53. The targeting system of claim 52, wherein the optical control module is enclosed within an enclosure, the enclosure having an escape window capable of transmitting the emitted and visible light, the escape window being positioned within the optical path between the first reflective element and the surface. (Item 54) Item 54. The targeting system of item 53, wherein the optical control module is completely enclosed within the enclosure. (Item 55) 55. The targeting system of claim 53 or 54, further comprising an air source configured to direct an air flow from an opening in an outer surface of the enclosure toward an outer surface of the escape window. (Item 56) 56. The targeting system of any one of claims 53-55, wherein the enclosure further comprises a wall opposite the opening, the wall configured to control the direction of the air flow and reduce turbulence without obstructing the beam. (Item 57) 57. The targeting system of any one of claims 43-56, wherein the first reflective element is a mirror. (Item 58) 58. The targeting system of any one of claims 43-57, wherein the combining element transmits the beam and reflects the visible light. (Item 59) 59. The targeting system of any one of claims 41-58, wherein the emitter is a laser emitter. (Item 60) 60. The targeting system of claim 59, wherein the laser emitter is selected from the group consisting of an infrared laser, an ultraviolet laser, and a visible laser. (Item 61) 61. The targeting system of any one of claims 43-60, wherein the optical control module further comprises a second targeting actuator connected to the first reflective element, the second targeting actuator configured to rotate the first reflective element and deflect the beam toward the target. (Item 62) 62. The targeting system of any one of claims 43-61, wherein the optical control module further comprises a second reflective element positioned to intersect the optical path and deflect the beam deflected by the first reflective element, and a second targeting actuator connected to the second reflective element, the second targeting actuator configured to rotate the second reflective element and deflect the beam toward the target. (Item 63) 63. The targeting system of claim 61 or 62, wherein the first targeting actuator deflects the beam along a first axis and the second targeting actuator deflects the beam along a second axis, the first axis and the second axis being orthogonal. (Item 64) Item 64. The targeting system of item 62 or 63, wherein, relative to the direction of the beam, the combining element is positioned behind the emitter, the first reflective element is positioned behind the combining element, and the second reflective element is positioned behind the first reflective element. (Incorporated by reference)

[0023] All publications, patents, and patent applications mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent, or patent application was specifically and individually indicated to be incorporated by reference. [Brief description of the drawings]

[0024] The novel features of the present disclosure are set forth with particularity in the appended claims. A better understanding of the features and advantages of the present disclosure will be obtained by reference to the following detailed description that sets forth illustrative embodiments, in which the principles of the disclosure are utilized, and the accompanying drawings.

[0025] [Figure 1A] FIG. 1A illustrates an isometric view of a laser targeting system according to one or more embodiments of the present disclosure.

[0026] [Figure 1B] FIG. 1B illustrates an isometric view of a laser targeting system with a laser path and a visible light path shown, according to one or more embodiments herein.

[0027] [Diagram 2]FIG. 2 illustrates a top view of a laser targeting system with the laser path and visible light path shown, according to one or more embodiments herein.

[0028] [Figure 3A] FIG. 3A illustrates a side view of a laser targeting system according to one or more embodiments herein.

[0029] [Figure 3B] FIG. 3B illustrates a side cutaway view of a laser targeting system with clean air paths shown, according to one or more embodiments herein.

[0030] [Figure 4] 4 illustrates a targeted laser and a targeted coverage area of ​​the targeted laser, according to one or more embodiments of the present disclosure. FIG. 4A illustrates a side view of the targeted laser and a targeted coverage area of ​​the targeted laser, according to one or more embodiments of the present disclosure. FIG. 4B illustrates a front view of the targeted laser and a targeted coverage area of ​​the targeted laser, according to one or more embodiments of the present disclosure.

[0031] [Diagram 5] FIG. 5 illustrates an isometric view of a predictive camera, multiple targeting lasers, the predicted viewing area of ​​the predictive camera, and the targeted coverage area of ​​the targeting lasers, in accordance with one or more embodiments herein.

[0032] [Figure 6] FIG. 6 illustrates a front view of the coverage area of ​​an autonomous laser weed eradication robot, a predictive camera, and multiple targeted lasers, in accordance with one or more embodiments herein.

[0033] [Figure 7] FIG. 7 illustrates an isometric view of the coverage area of ​​an autonomous laser weed eradication robot, a predictive camera, and multiple targeted lasers, in accordance with one or more embodiments herein.

[0034] [Figure 8] FIG. 8 depicts a method for identifying, assigning, and targeting targets according to one or more embodiments herein.

[0035] [Figure 9] FIG. 9 depicts a method for identifying, allocating, targeting, and eradicating weeds in a field according to one or more embodiments herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0036] Crop cultivation is essential for food and fiber production. One important component of crop management is the control or elimination of undesirable plant species, commonly referred to as weeds. Weeds can reduce crop yields by depriving desired plants of resources, including water, nutrients, sunlight, and space. Weeds can further impede crop growth by harboring pests or parasites that damage desired plants. Traditional weed control and eradication methods include manual cultivation or chemical herbicides. Manual cultivation is labor intensive, leading to increased costs of crop production and higher food and fiber prices. The use of chemical herbicides can have adverse environmental effects, including groundwater contamination, acute toxicity, or long-term health effects, such as cancer.

[0037] The development of environmentally responsible, low-cost weed control and eradication methods is important for higher crop yields, lower food prices, and long-term environmental stability. Reducing or eliminating the need for herbicides could reduce many of the harmful environmental side effects of crop production, including toxic runoff and groundwater contamination. Reducing the need for manual labor could substantially lower agricultural costs and improve labor standards.

[0038] The present disclosure provides various methods, devices, modules, and systems that can be employed for the automatic identification, maintenance, control, or targeting of plants. In some embodiments, the methods, devices, modules, and systems disclosed herein can be used to autonomously identify and eradicate weeds located within a field of crops. For example, certain methods are disclosed herein for autonomously locating, identifying, and targeting objects, such as weeds, using a beam comprising electromagnetic radiation. Also disclosed herein are devices configured to locate, identify, and autonomously target objects using a beam. The devices can be used, for example, to control or eliminate weeds. For example, the devices can be used to burn or irradiate weeds. The modules disclosed herein can be used to implement the methods disclosed herein and for autonomous control of the devices and systems disclosed herein for, for example, locating, identifying, targeting, and controlling or eliminating weeds. The systems disclosed herein may include devices, modules, and methods configured to autonomously control or eliminate objects, such as weeds, by locating, identifying, and targeting the objects using emissions. Sometimes, the methods, devices, modules, and systems may be used for agricultural crop management or for home weed control. The methods, devices, modules, and systems may be used as an alternative to manual cultivation or chemical herbicides. (Optical Control System)

[0039] Described herein is an optical control system for directing a beam (e.g., a light beam) towards a target location on a surface. FIG. 1A illustrates an isometric view of an embodiment of an optical control system 100 as disclosed herein. An emitter 101 is configured to direct a beam along an optical path 102. In some embodiments, the beam comprises electromagnetic radiation, e.g., light, radio waves, microwaves, or x-rays. In some embodiments, the light is visible light, infrared light, or ultraviolet light. The beam may be coherent. In a preferred embodiment, the emitter is a laser, such as an infrared laser. In some embodiments, the emitter emits a beam having a wavelength of about 1 m, about 100 mm, about 10 mm, about 1 mm, about 100 μm, about 10 μm, about 1.5 μm, about 1 μm, about 900 nm, about 800 nm, about 700 nm, about 600 nm, about 500 nm, about 400 nm, about 300 nm, about 100 nm, about 10 nm, or about 1 nm. In some embodiments, the emitter emits a beam having a wavelength of about 1 m to about 100 mm, about 100 mm to about 10 mm, about 10 mm to about 1 mm, about 1 mm to about 100 μm, about 100 μm to about 10 μm, about 10 μm to about 1.5 μm, about 1.5 μm to about 1 μm, about 1 μm to about 900 nm, about 900 nm to about 800 nm, about 800 nm to about 700 nm, about 700 nm to about 600 nm, about 600 nm to about 500 nm, about 500 nm to about 400 nm, about 400 nm to about 300 nm, about 300 nm to about 100 nm, about 100 nm to about 10 nm, or about 10 nm to about 1 nm. In some embodiments, the emitter may be capable of emitting electromagnetic radiation up to 10 mW, up to 100 mW, up to 1 W, up to 10 W, up to 100 W, up to 1 kW, or up to 10 kW. In some embodiments, the emitter may be capable of emitting electromagnetic radiation from 10 mW to 100 mW, 100 mW to 1 W, 1 W to 10 W, 10 W to 100 W, 100 W to 1 kW, or 1 kW to 10 kW.

[0040] FIG. 1B shows an isometric view of the embodiment of the optical control device 100 shown in FIG. 1A and further illustrates the position and direction of the beam path 102. Reference numbering is consistent between FIG. 1A and FIG. 1B. One or more optical elements may be positioned in the path of the beam. The optical elements may include one or more of a beam combiner 103, a first reflecting element 105, and a second reflecting element 106. The elements may be arranged in the following order in the direction of the beam path: the beam combiner 103, followed by the first reflecting element 105, followed by the second reflecting element 106. In another example, one or both of the first reflecting element or the second reflecting element may be arranged before the beam combiner in the order of the direction of the beam path. In another example, the optical elements may be arranged in the following order in the direction of the beam path: the beam combiner 103, followed by the first reflecting element 105. In another example, one or both of the first or second reflective elements may be configured before the beam combiner in the direction of the beam path. Any number of additional reflective elements may be positioned in the beam path.

[0041] The beam combiner may also be referred to as a beam combining element. In some embodiments, the beam combiner 103 may be a zinc selenide (ZnSe), zinc sulfide (ZnS), or germanium (Ge) beam combiner. For example, the beam combiner may be configured to transmit infrared light and reflect visible light. In some embodiments, the beam combiner 103 may be a dichroic. In some embodiments, the beam combiner may be configured to pass electromagnetic radiation having a wavelength longer than a cutoff wavelength and reflect electromagnetic radiation having a wavelength shorter than a cutoff wavelength. In some embodiments, the beam combiner may be configured to pass electromagnetic radiation having a wavelength shorter than a cutoff wavelength and reflect electromagnetic radiation having a wavelength longer than a cutoff wavelength. In some embodiments, the cutoff wavelength can be about 1 m, about 100 mm, about 10 mm, about 1 mm, about 100 μm, about 10 μm, about 1.5 μm, about 1 μm, about 900 nm, about 800 nm, about 700 nm, about 600 nm, about 500 nm, about 400 nm, about 300 nm, about 100 nm, about 10 nm, or about 1 nm. In some embodiments, the cutoff wavelength can be about 1 m to about 100 mm, about 100 mm to about 10 mm, about 10 mm to about 1 mm, about 1 mm to about 100 μm, about 100 μm to about 10 μm, about 10 μm to about 1.5 μm, about 1.5 μm to about 1 μm, about 1 μm to about 900 nm, about 900 nm to about 800 nm, about 800 nm to about 700 nm, about 700 nm to about 600 nm, about 600 nm to about 500 nm, about 500 nm to about 400 nm, about 400 nm to about 300 nm, about 300 nm to about 100 nm, about 100 nm to about 10 nm, or about 10 nm to about 1 nm. In other embodiments, the beam combiner can be a polarizing beam splitter, a long pass filter, a short pass filter, or a band pass filter.

[0042] The position and orientation of one or both of the first reflective element 105 and the second reflective element 106 may be controlled by an actuator. In some embodiments, the actuator may be a motor, a solenoid, a galvanometer, or a servo. For example, the position of the first reflective element may be controlled by a first actuator, and the position and orientation of the second reflective element may be controlled by a second actuator. In some embodiments, a single reflective element may be controlled by multiple actuators. For example, the first reflective element may be controlled by a first actuator along a first axis and a second actuator along a second axis. In some embodiments, a single actuator may control the reflective element along multiple axes. The actuator may change the position of the reflective element by rotating the reflective element, thereby changing the angle of incidence of the beam that encounters the reflective element. Changing the angle of incidence may cause a translation of the position where the beam encounters the surface. In some embodiments, the angle of incidence may be adjusted such that the position where the beam encounters the surface is maintained while the optical system moves relative to the surface. In some embodiments, the first actuator rotates the first reflective element about a first rotation axis, thereby translating the location where the beam meets the surface along the first translation axis, and the second actuator rotates the second reflective element about a second rotation axis, thereby translating the location where the beam meets the surface along the second translation axis. In some embodiments, the first actuator and the second actuator rotate the first reflective element about the first rotation axis and the second rotation axis, thereby translating the location where the beam meets the surface of the first reflective element along the first translation axis and the second translation axis. For example, a single reflective element may be controlled by the first actuator and the second actuator, with the single reflective element being controlled by two actuators to provide translation of the location where the beam meets the surface along the first translation axis and the second translation axis. The first translation axis and the second translation axis may be orthogonal.The coverage area on the surface may be defined by the maximum translation along the first translation axis and the maximum translation along the second translation axis. One or both of the first and second actuators may be servo-controlled, piezoelectrically actuated, piezoelectric inertia actuated, stepper motor controlled, galvanometer driven, linear actuator controlled, or any combination thereof. One or both of the first and second reflective elements may be mirrors, such as dichroic mirrors, or dielectric mirrors, prisms, beam splitters, or any combination thereof. In some embodiments, one or both of the first and second reflective elements may be any element capable of deflecting a beam.

[0043] FIG. 2 shows a top view of an embodiment of the optical control system 100 as shown in FIG. 1A and FIG. 1B. Reference numbers are consistent between FIG. 1A, FIG. 1B, and FIG. 2. The targeting camera 104 may be positioned to capture light 152, for example, visible light traveling along an optical path in a direction opposite to the beam path 151. The light may be scattered by a surface, such as a surface comprising a target. In some embodiments, the targeting camera is positioned such that it captures light reflected from the beam combiner 103. In other embodiments, the targeting camera is positioned such that it captures light transmitted through the beam combiner. The targeting camera may be configured to image a target field on a surface. The targeting camera may be coupled to the beam combiner or the targeting camera may be coupled to a support structure that supports the beam combiner. In a preferred embodiment, the targeting camera does not move relative to the beam combiner.

[0044] 3A and 3B show side views of an embodiment of an optical control device disclosed herein. Reference numbering is consistent between FIGS. 1-3. FIG. 3B illustrates a mechanism for preventing dust and residue buildup on the optical elements of the optical control device shown in FIGS. 1-3. In some embodiments, the optical elements may include hard stops 351 on the mirrors to prevent the beam from hitting an area of ​​the optical control device outside a predetermined boundary on the surface. One or both of the optical elements, e.g., the beam combining element and the reflective element, may be protected by an enclosure. The optical elements may be surrounded by an enclosure. In some embodiments, the enclosure is sealed to prevent dust, residue, water, or any combination thereof from contacting the optical elements. The enclosure may include a laser escape window 107, as shown in FIG. 3B. In some embodiments, the laser escape window is positioned to intersect the beam after the second reflective element in the beam path, or the laser escape window is positioned to intersect the beam after the first reflective element in the beam path. In some embodiments, the laser escape window is the last element in the beam path. The laser escape window may prevent dirt, debris, water, or any combination thereof from reaching the optical elements. In some embodiments, the laser escape window comprises a material that is substantially transparent to electromagnetic radiation, such as light. For example, the laser escape window may comprise glass, quartz, fused silica, zinc selenide, a transparent polymer, or a combination thereof.

[0045] The enclosure may further comprise a self-cleaning device configured to prevent accumulation of dust or residue on the surface of the laser escape window or to remove dust or residue accumulated on the surface of the laser escape window. In some embodiments, the self-cleaning device comprises an opening 352 in an exterior surface of the enclosure configured to exhaust clean air 353. The clean air may prevent residue from damaging the laser escape window. In some embodiments, the clean air may be filtered. The opening may be configured to direct the air flow towards the exterior surface of the escape window. The opening may be configured such that the clean air is directed across the surface of the laser escape window. In some embodiments, the enclosure is configured to direct the clean air without impeding the beam 102. For example, the enclosure may comprise an opening 354 after the laser escape window in the beam path with a gap so that the beam may pass unimpeded. In some embodiments, the opening comprises a wall opposite the opening. The wall may be configured to control the direction of the air flow and reduce turbulence without impeding the beam. The opening encompasses the laser escape window and the beam path, and may be configured such that the opening is narrower near the laser escape window and wider in the direction of the beam path further from the laser escape window, in some embodiments, the opening has smooth corners 355 to allow the passage of clean air while preventing turbulence.

[0046] After exiting the optical control system, the beam 102 may be directed towards a surface, as shown in Figures 4A and 4B. In some embodiments, the surface comprises a target, e.g., a weed. Rotational motion of one or both of the reflective elements 105 and 106 as shown in Figure 2 may generate a laser sweep along a first translation axis 401 and a laser sweep along a second translation axis 402, as shown in views 400 and 450 of Figures 4A and 4B, respectively. Rotational motion of one or both of the reflective elements may control where the beam meets the surface. For example, rotational motion of one or both of the reflective elements may move where the beam meets the surface to the location of the target on the surface. In some embodiments, the beam is configured to damage the target. For example, the beam may comprise electromagnetic radiation, and the beam may irradiate the target. In another example, the beam may comprise infrared light, and the beam may burn the target. In some embodiments, one or both of the reflective elements can be rotated so that the beam scans an area surrounding and including the target. (Complex System)

[0047] In some embodiments, multiple optical control systems can be combined to increase the coverage area on a surface. FIG. 5 illustrates a composite system 500 comprising multiple optical control systems 100. The multiple optical control systems are configured such that the laser sweep along the translation axis 402 of each optical control system overlaps with the laser sweep along the translation axis of an adjacent optical control system. The combined laser sweep defines a coverage area 503 that can be reached by at least one of the multiple beams from the multiple optical control systems. A predicted camera 501 can be positioned such that the predicted camera field of view 502 completely encompasses the coverage area 503.

[0048] The optical control systems may be configured on the vehicle 601 as shown in view 600 of FIG. 6 and view 700 of FIG. 7. For example, the vehicle may be an autonomous vehicle. The autonomous vehicle may be a robot. In some embodiments, the vehicle may be controlled by a human. For example, the vehicle may be driven by a human driver. In some embodiments, the vehicle may be coupled to a second vehicle driven by a human driver and may be towed behind or pushed by the second vehicle. The vehicle may be controlled remotely by a human, for example, by a remote control. In some embodiments, the vehicle may be controlled remotely via long wave signals, optical signals, satellite, or any other remote communication method. The optical control systems may be configured on the vehicle such that the coverage area overlaps with surfaces under, behind, in front of, or surrounding the vehicle. The vehicle may be configured to navigate a surface comprising multiple targets, for example, a crop field comprising multiple weeds. The vehicle may comprise one or more of multiple wheels, a power source, a motor, a predictive camera 501, or any combination thereof. In some embodiments, the vehicle has sufficient clearance above the surface to drive over plants, e.g., crops, without damaging the plants. In some embodiments, the space between the inner edge of the left wheel and the inner edge of the right wheel is wide enough to pass over one row of plants without damaging the plants. In some embodiments, the distance between the outer edge of the left wheel and the outer edge of the right wheel is narrow enough to allow the vehicle to pass over two rows of plants, e.g., two rows of crops, without damaging the plants. In a preferred embodiment, a vehicle with multiple wheels, multiple optical control systems, and a predictive camera can navigate the rows of crops and emit beams of multiple beams toward targets, e.g., weeds, thereby burning or irradiating the weeds. (Prediction module)

[0049] A prediction module configured to identify the location of a target on a surface is disclosed herein. FIG. 8 illustrates a prediction module 810 configured to identify, assign, and target targets. In some embodiments, the target prediction system 811 is configured to capture an image of a predicted field of view comprising the surface using a prediction camera 501, identify a target in the image, and identify the location of the target in the predicted field of view. A camera 812 for controlling a translation system may be configured to translate the location of the target in the predicted field of view to a position on the surface. For example, the camera for controlling a translation system may construct multiple interpolation functions that provide translation from a location in the predicted field of view to one or more actuator positions, such as the pan and tilt positions of one or more actuators that control one or more reflective elements 105 and 106 as shown in FIG. 1-3.

[0050] The prediction module 810 shown in FIG. 8 may further comprise an attitude and motion correction system 813. The attitude and motion correction system may comprise a position determination system, such as an inertial measurement unit (IMU), a global positioning system (GPS), or an internal navigation system (INS). The attitude and motion correction system may utilize an inertial measurement unit (IMU) that may be directly or indirectly coupled to the prediction camera. For example, the prediction camera and IMU may be mounted on a vehicle. The IMU may collect motion readings of the IMU and anything directly or indirectly coupled to the IMU, such as the prediction camera. For example, the IMU may collect readings comprising three-dimensional acceleration and three-dimensional rotation information, which may be used to determine the magnitude and direction of motion over time. The attitude and motion correction system may comprise a global positioning system (GPS). The GPS may be directly or indirectly coupled to the targeting camera. For example, the GPS may communicate with a satellite-based radio navigation system to measure a first position of the targeting camera at a first time and a second position of the targeting camera at a second time. The pose and motion correction system may include an internal navigation system (INS). The INS may be directly or indirectly coupled to the targeting camera. For example, the INS may include motion sensors, such as accelerometers, and rotation sensors, such as gyroscopes, to measure the position, orientation, and velocity of the targeting camera. The pose and motion correction system may use an external reference and may or may not determine the change in the position of the targeting camera. The pose and motion correction system may determine the change in the position of the targeting camera from the first position and the second position. In some embodiments, after the target prediction system identifies the location of the target in the image, the pose and motion correction system determines the amount of time that has elapsed since the image was captured, and the magnitude and direction of the predicted camera motion that occurred during the elapsed time. The pose and motion correction system may integrate the target location, the elapsed time, and the magnitude and direction of the motion to determine a corrected location of the target on the surface.

[0051] The prediction module may further comprise an image detection module. The image detection module may be configured to locate and identify a target in the image. For example, the image detection module may be configured to distinguish between two plants, such as a crop and a weed. In some embodiments, the image detection module includes using a convolutional neural net. The neural net may be trained with many images of a surface, such as images from a prediction camera or a targeting camera, with or without a target. For example, the neural net may be trained with images of a field, with or without weeds. Once trained, the neural net may be configured to identify a region in the image that comprises the target. The region may be defined by a polygon, e.g., a rectangle. In some embodiments, the region is a bounding box. In some embodiments, the region is a polygonal mask that covers the identified region.

[0052] Based on the target location, the target assignment system 814 may assign the target to a targeting module 820 of the multiple targeting modules. The target location may be corrected based on the magnitude and direction of movement during the elapsed time, or the location may be within an area defined by a polygon, or both. Future target locations may be determined based on the predicted magnitude and direction of movement during a future period of time. The target assignment module may assign the target to a targeting module that has a coverage area that overlaps with the target location, the corrected target location, or the future target location.

[0053] The prediction module may comprise a system controller, for example, a system computer having a storage device, a random access memory (RAM), a central processing unit (CPU), and a graphics processing unit (GPU). The system computer may comprise a tensor processing unit (TPU). The system computer should have sufficient RAM, storage space, CPU power, and GPU power to perform operations for detecting and identifying targets. The prediction camera should provide images of sufficient resolution to perform operations for detecting and identifying targets. (Targeting Module)

[0054] Disclosed herein is a targeting module configured to direct a beam toward a target location on a surface. FIG. 8 illustrates a targeting module 820 configured to predict a target location, move one or more optical elements, and direct a beam toward the target location. A plurality of targeting modules may communicate with the prediction module 810. The targeting module comprises an optical control system as described herein. For example, as shown in FIG. 1-3, the targeting module may comprise an emitter 101 that emits a beam 102 along an optical path, a beam combining element 103, and optionally a targeting camera 104, a first reflecting element 105 configured to deflect the beam controlled by a first actuator, and optionally a second reflecting element 106 configured to deflect the beam controlled by a second actuator positioned in the optical path. One or both of the actuators may be configured to rotate one or both of the reflective elements about a first rotation axis, optionally a second rotation axis, thereby varying the deflection of the beam path and translating the position where the beam meets the surface along the first translation axis, optionally along the second translation axis. In some embodiments, the first actuator and the second actuator may rotate a single reflective element about the first and second rotation axes to provide translation of the position of the point where the beam meets the surface along the first and second translation axes. The prediction camera should have a field of view wide enough to image the coverage area of ​​the beam path.

[0055] As shown in FIG. 8, a target prediction system 821 captures an image of an area on a surface. The area may be predicted to include a target as predicted by the prediction module 810. The target prediction system may identify a pixel location of the target in the image. A camera for controlling a translation system 822 may convert the pixel location of the target image to a position of a first reflective element, and optionally, a second reflective element. The positions of the reflective elements may be controlled by actuators as described herein. For example, a camera for controlling a translation system may convert the pixel location of the target into a pan or tilt value of one or both actuators (corresponding to a mirror position predicted to deflect the beam to the target location).

[0056] In some embodiments, the target prediction system further comprises an image detection module. The image detection module may be configured to locate and identify a target in an image. For example, the image detection module may be configured to distinguish between two plants, such as a crop and a weed. In some embodiments, the image detection module includes using a convolutional neural net. The neural net may be trained with many images of a surface, such as images from a prediction camera or a targeting camera, with or without a target. For example, the neural net may be trained with images of a field, with or without weeds. Once trained, the neural net may be configured to identify a region in the image that comprises a target. The region may be defined by a polygon, e.g., a rectangle. In some embodiments, the region is a bounding box. In some embodiments, the region is a polygonal mask that covers the identified region.

[0057] The target location may be further corrected using an attitude and motion correction system 823. The attitude and motion correction system may use a position determination system, such as an IMU, GPS, or INS, to determine the magnitude and direction of the targeting camera's motion. In some embodiments, acceleration and rotation readings from an IMU directly or indirectly coupled to the targeting camera are used to determine the magnitude and direction of the motion. For example, the prediction camera and IMU may be mounted on the vehicle. The IMU may collect motion readings of the IMU and anything directly or indirectly coupled to the IMU, such as the targeting camera. For example, the IMU may collect readings comprising three-dimensional acceleration and three-dimensional rotation information, which may be used to determine the magnitude and direction of the motion over time. In some embodiments, the attitude and motion correction system may use a GPS to determine the magnitude and direction of the targeting camera's motion. For example, the GPS may be mounted on the vehicle. The GPS may communicate with a satellite-based radio navigation system to measure a first position of the targeting camera at a first time and a second position of the targeting camera at a second time. In some embodiments, the pose and motion correction system may use an INS to determine the magnitude and direction of the targeting camera's motion. For example, the INS may measure the position, orientation, and speed of the targeting camera. In some embodiments, after the target prediction system 821 locates the target in the image, the pose and motion correction system determines the amount of time that has elapsed since the image was captured, and the magnitude and direction of the targeting camera's motion that occurred during the elapsed time. The pose and motion correction system may integrate the target location, the elapsed time, and the magnitude and direction of the motion to determine a corrected location of the target on the surface. In some embodiments, the position determination system used by the pose and motion correction system of the targeting module 823 and the position determination system used by the pose and motion correction system of the prediction module 813 are the same. Future target locations may be determined based on the predicted magnitude and direction of motion during a future period of time.In some embodiments, the position determination system used by the pose and motion correction system of the targeting module and the position determination system used by the pose and motion correction system of the prediction module are different.

[0058] The actuator control system 824 comprises software-driven electrical components capable of providing signals to the first actuator, and optionally the second actuator, to control the first reflective element, and optionally the second reflective element. For example, the actuator control system sends signals to the first actuator and the second actuator comprising actuator pan tilt values. The actuators adopt the signaled pan tilt positions and move the first reflective element and the second reflective element about the first and second rotation axes to positions such that the beam is deflected to the target location, the corrected target location, or a future target location.

[0059] The laser control system 825 comprises software-driven electrical components capable of controlling the activation and deactivation of the emitters. Activation or deactivation may depend on the presence or absence of a target as detected by the targeting camera 104. Activation or deactivation may depend on the position of the beam path directed towards the surface relative to the target location. In some embodiments, the laser control system may activate the emitter when the target is identified by the target prediction system. In some embodiments, the laser control system may activate the emitter when the beam path is positioned to overlap the target location. In some embodiments, the laser control system may fire the emitter when the beam path is within an area of ​​the surface comprising the target defined by a polygon, e.g., a bounding box, or a polygonal mask covering the identified area. The laser control system may deactivate the emitter when the target is removed, when the area comprising the target is scanned by the beam, when the target is no longer identified by the target prediction module, when a specified period of time has elapsed, or any combination thereof. For example, the laser control system may deactivate the emitter once an area on the surface comprising weeds has been scanned by the beam or once the weeds have been irradiated or burned.

[0060] The prediction module and the targeting module described herein may be used in combination to locate the target using the beam, to identify the target, and to target the target. The targeting control module may include an optical control system as described herein. The prediction module and the targeting module may be in communication, e.g., electrical or digital communication. In some embodiments, the prediction module and the targeting module are directly or indirectly coupled. For example, the prediction module and the targeting module may be coupled to a support structure. In some embodiments, the prediction module and the targeting module are configured on a vehicle, e.g., vehicle 601 as shown in FIG. 6 and FIG. 7.

[0061] The targeting module may include a system controller, such as a system computer having a storage device, a random access memory (RAM), a central processing unit (CPU), and a graphics processing unit (GPU). The system computer may include a tensor processing unit (TPU). The system computer should have sufficient RAM, storage space, CPU power, and GPU power to perform operations for detecting and identifying targets. The targeting camera should provide images of sufficient resolution to perform operations for detecting and identifying targets. (Calibration method)

[0062] The prediction module disclosed herein may further include a calibration step. In some embodiments, a camera for controlling the translation system of the prediction module 812 is calibrated. In some embodiments, a calibration surface is positioned within the field of view of the prediction camera. The calibration surface includes known marks at known locations. The prediction camera may collect multiple images of the calibration surface at different locations relative to the calibration surface. The prediction module may then correlate pixel locations of the known marks with known locations on the surface. An interpolation function may be constructed from the multiple correlated pixel locations and the known surface location. In some embodiments, the interpolation function may be stored on a hard drive and loaded from the hard drive by the prediction module.

[0063] The targeting module disclosed herein may further include a calibration step. In some embodiments, a camera for controlling the translation system of the targeting module 812 is calibrated. In some embodiments, a calibration surface is positioned within the field of view of the targeting camera. The calibration surface comprises a known mark at a known location. The targeting module may collect multiple images of the calibration surface and multiple actuator positions such that the multiple images comprise different fields of view. For example, the targeting module may collect multiple images at multiple randomly selected pan tilt values ​​of the first actuator and the second actuator. A calibration map may be constructed from multiple sample points. Each sample point may be collected by identifying a pixel location of a known mark in an image collected at a known actuator position and correlating the known location with the actuator position and pixel location. In some embodiments, the map is fitted with a spline smoothing algorithm to construct a smooth curve and allow accurate estimation of the location between the sample points. In some embodiments, the spline smoothing algorithm may be stored on a hard drive and loaded from the hard drive by the targeting module. (Weed eradication system)

[0064] FIG. 9 illustrates a process 900 for an embodiment of the device and method disclosed herein. The following example is illustrative and non-limiting to the scope of the device, system, and method described herein. The process includes identifying, allocating, targeting, and eradicating weeds in a field. In this example, the weed eradication system includes a prediction module 810 that communicates with a plurality of targeting modules 820. The prediction module and the targeting module are controlled by a system controller, e.g., a computer that includes storage, RAM, CPU, and GPU. Each targeting module includes an optical control system 100, as shown in FIG. 1-3. The prediction module and the targeting module are coupled to a solid support. The solid support is positioned on a vehicle 601, as shown in FIG. 6 and FIG. 7.

[0065] As shown in FIG. 9, operations 920, 930, 940, 950, and 960 are repeated 910 until the field of interest is completely scanned. First, the prediction module initiates operation 920. The prediction camera collects images of the field surface in the area surrounding or in front of the vehicle. The system controller processes the images and identifies weeds in the images. In step 921, the prediction model predicts the location of one or more weeds identified in the image. In step 922, the camera for controlling the system converts the pixel coordinates of the weeds in the image to ground locations. The system controller commands the vehicle to adjust its position and speed 923 based on the motion of the vehicle measured by the IMU in 922. Each of the one or more weeds is assigned to a targeting module 924 based on the ground location of the weed and the coverage area of ​​the targeting module.

[0066] Operations 930, 940, 950, and 960 are repeated for each target module 925. Operations 940, 950, and 960 are repeated for each weed. A targeting module of the multiple targeting modules initiates operation 940. A targeting camera captures a target image of the field, and a system controller identifies the weed in the target image 941. The system controller converts the pixel location of the weed in the target image into pan and tilt values ​​for each actuator that controls each reflective element in the optical control system controlled by the targeting module 942. The system controller applies attitude and motion corrections to the actuator pan and tilt values ​​based on the vehicle motion measured by the IMU at 943, and plans a route for the emitted beam path controlled by the actuator pan and tilt positions 944. When the actuator reaches the determined position, the emitter is activated 945.

[0067] Operations 950 are repeated 946 while the planned route is implemented. Weeds are identified in images collected by the targeting camera and the route plan is updated 952 based on the observed positions of the weeds. The system controller applies attitude and motion corrections to the actuator pan and tilt values ​​based on the vehicle motion measured by the IMU at 953. The actuators are moved into position 954 based on the updated route plan. Once the planned route is completed, the emitters are deactivated 960.

[0068] While preferred embodiments of the present disclosure have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Numerous variations, changes, and substitutions will now occur to those skilled in the art without departing from the present disclosure. It is understood that various alternatives to the embodiments of the present disclosure described herein may be employed in practicing the present disclosure. The following claims define the scope of the disclosure, and it is intended that methods and structures within the scope of these claims and their equivalents be covered thereby.

Claims

1. 1. A targeting system for targeting a target on a surface, the targeting system comprising: a prediction module; and a targeting module; The prediction module includes: a predictive camera configured to image a predictive camera field of view on the surface and identify a location of the target within the predictive camera field of view; a prediction module controller configured to convert a location of the target within the predicted camera field of view to a predicted location of the target on the surface, assign the target to the targeting module, and provide the predicted location to the targeting module; Equipped with The targeting module comprises: a targeting module controller configured to receive the predicted location from the prediction module and convert the predicted location into an actuator position; an optical control system comprising the actuator, the optical control system configured to point a targeting camera towards the predicted location; a targeting camera configured to image a target field of view at a first time that overlaps with the predicted location, the targeting module controller being further configured to determine, at a second time after the first time, a location of the target within the target field of view that compensates for movement of the targeting camera relative to the surface during an elapsed time between the first time and the second time; an emitter configured to emit a beam along an optical path, the optical control system further configured to receive position information from the targeting module controller and direct the optical path towards the target based on a location of the target within the target field of view; A targeting system comprising:

2. 2. The targeting system of claim 1, wherein the optical control system is further configured to activate the emitter to emit the beam when the optical path is aligned with the target.

3. The optical control system includes: a first reflective element controlled by the actuator and positioned to intersect the optical path and deflect the beam; a beam combining device positioned in the optical path between the emitter and the first reflective element, the beam combining device configured to differentially deflect the beams and visible light from the target field of view traveling along the optical path in a direction opposite to the beams; 3. The targeting system of claim 1 or claim 2, further comprising:

4. 4. The targeting system of claim 3, wherein the optical control system further comprises an additional actuator connected to the first reflective element, the additional actuator configured to rotate the first reflective element and deflect the beam towards the target.

5. 5. The targeting system of claim 3 or claim 4, wherein the optical control system further comprises a second reflective element positioned to intersect the optical path and deflect the beam deflected by the first reflective element, and a second actuator connected to the second reflective element, the second actuator configured to rotate the second reflective element and deflect the beam towards the target.

6. 6. The targeting system of claim 5, wherein the actuator deflects the beam along a first translation axis and the second actuator deflects the beam along a second translation axis, the first translation axis and the second translation axis being orthogonal.

7. 7. The targeting system of claim 5 or claim 6, wherein, relative to the direction of the beams, the beam combiner is positioned after the emitters, the first reflective element is positioned after the beam combiner, and the second reflective element is positioned after the first reflective element, and the beam combiner transmits the beams and reflects the visible light.

8. 8. The targeting system of claim 1, wherein the optical control system is configured to direct the beam towards the target while the targeting system is moving relative to the surface.

9. The targeting system of any one of claims 1 to 8, wherein the targeting module is configured to detect movement of the target field of view relative to the target and translate the movement of the target field of view into movement of the actuator.

10. 10. The targeting system of any one of claims 1 to 9, wherein the targeting module is configured to adjust the predicted location based on an amount of time since imaging, an acceleration of the targeting system, a rotation of the targeting system relative to the surface, or a combination thereof.

11. Further comprising a second targeting module, the second targeting module comprising: a second targeting camera configured to image a second target field on the surface and identify a location of the target within the second target field; a second targeting module controller configured to translate a location of the target within the second target field of view into a position of a second targeting module actuator; The targeting system of any one of claims 1 to 10, comprising:

12. The targeting system of any one of claims 1 to 11, wherein the target field of view is contained within the predicted camera field of view.

13. 13. The targeting system of any one of claims 1 to 12, wherein the optical control system is enclosed within an enclosure, the enclosure comprising a laser escape window capable of transmitting the emitted and visible light, the escape window being positioned in the optical path between a first reflective element and the surface.

14. 14. The targeting system of claim 13, further comprising an opening in an exterior surface of the enclosure configured to direct air from the opening towards an exterior surface of the laser escape window.

15. The targeting system of any one of claims 1 to 14, wherein the actuator is configured to control a direction of the targeting camera and the optical path of the beam.

16. The targeting system of any one of claims 1 to 15, wherein the predictive camera, the optical control system, the actuator position, or a combination thereof, are calibrated relative to the surface.

17. A targeting system comprising: a first camera configured to image the surface; an emitter configured to emit a beam towards the surface; a computing system in communication with the first camera and the emitter, the computing system comprising: receiving a target image of the surface from the first camera at a first time, the target image depicting weeds on the surface; identifying an area in the target image that includes the weed; predicting a target location of the weed on the surface at a second time based on the identified region in the target image, the predicted target location being corrected for motion of the first camera relative to the surface during an elapsed time between the first time and the second time; aligning an optical path of the emitter with the predicted target location of the weed; causing the emitter to emit the beam towards the weed when the optical path is aligned with the weed; a computing system configured to perform operations including: A targeting system comprising:

18. 20. The targeting system of claim 17, further comprising a second camera in communication with the computing system.

19. The operation includes: receiving a predicted image from the second camera, the predicted image including the surface; and identifying an area in the predicted image that contains the weed; predicting a predicted location of the weeds based on the predicted image and the identified regions within the predicted image; aligning the first camera with the predicted predicted location; 20. The targeting system of claim 18, further comprising:

20. 20. The targeting system of claim 19, wherein the target image includes the predicted predicted location.

21. 21. The targeting system of any one of claims 17 to 20, further comprising an actuator configured to align the optical path of the emitter with the predicted target location, the actuator further configured to align the first camera with the predicted predicted location, control a mirror in the optical path, or both.

22. The targeting system of any one of claims 1 to 21, wherein the targeting system is located on a vehicle.

23. 23. The targeting system of claim 22, wherein the vehicle is an autonomous vehicle.

24. The targeting system of any one of claims 1 to 23, wherein the surface is an agricultural surface and the target is a weed.

25. 25. The targeting system of any one of claims 1 to 24, further comprising an inertial measurement unit configured to measure acceleration of the targeting system and rotation of the targeting system relative to the surface.

26. The targeting system of any one of claims 1 to 25, wherein the emitter is a laser.

27. 27. The targeting system of claim 26, wherein the laser is selected from the group consisting of an infrared laser, an ultraviolet laser, and a visible laser.

28. A method, comprising: receiving a target image of a surface at a first time, the target image depicting weeds on the surface; identifying an area in the target image that includes the weed; predicting a target location of the weed on the surface at a second time based on the identified region in the target image, the predicted target location being corrected for motion of a first camera relative to the surface during an elapsed time between the first time and the second time; aligning an optical path of an emitter with the predicted target location of the weed; causing the emitter to emit a beam towards the weed when the optical path is aligned with the weed; A method comprising:

29. receiving a predicted image of the surface; identifying an area in the predicted image that contains the weed; predicting a predicted location of the weed based on the predicted image and the identified region within the predicted image; Further comprising: The method of claim 28 , wherein the target image includes the predicted predicted location.

30. 30. The method of claim 28 or claim 29, wherein the emitter is moving relative to the surface.

31. A method according to any one of claims 28 to 30, further comprising causing the emitter to deactivate the beam after the weeds have been damaged or killed.

32. A method according to any one of claims 28 to 31, wherein aligning the optical path with the expected target location comprises referencing a calibration feature.

33. A system comprising: A processor; A memory that stores programming instructions; Equipped with The programming instructions, when executed by the processor, receiving a target image of a surface at a first time, the target image depicting weeds on the surface; identifying an area in the target image that includes the weed; predicting a target location of the weed on the surface at a second time based on the identified region in the target image, the predicted target location being corrected for motion of a first camera relative to the surface during an elapsed time between the first time and the second time; providing instructions for aligning an optical path of an emitter with the predicted target location of the weed; providing instructions to the emitter to emit a beam towards the weed when the optical path is aligned with the weed; A system that causes the system to perform operations including:

34. The operation includes: receiving a predicted image of the surface; identifying an area in the predicted image that contains the weed; predicting a predicted location of the weed based on the predicted image and the identified region within the predicted image; Further comprising:

34. The system of claim 33, wherein the target image includes the predicted predicted location.

35. 35. The system of claim 34, wherein the operations further include providing instructions to the emitter to deactivate the beam once the weeds are damaged or killed.

Images