Systems and methods for using backscatter imaging in precision agriculture

Abstract

Systems and methods are provided for determining the quality of a crop by using at least one X-ray scanner. The method includes obtaining at least two scan images of the crop, wherein a first of the at least two images is obtained along a first plane relative to the crop and a second of the at least two images is obtained along a second plane relative to the crop, and wherein the first plane is angularly displaced relative to the second plane, registering the first and second images, correcting the registered first and second images, and determining the quality of the crop from the corrected first and second images.

Description

[0001] Cross-references to related applications

[0002] This application is a continuation-in-part of U.S. Patent Application No. 15 / 758134, entitled “Backscatter Imaging for Precision Agriculture,” filed March 7, 2018, which is a 371 national phase application of PCT / US2016 / 050545 with the same title, filed September 7, 2016. Its priority relies on U.S. Provisional Application No. 62 / 337971, filed May 18, 2016, and U.S. Provisional Application No. 62 / 215456, filed September 8, 2015.

[0003] Furthermore, this application relates to U.S. Patent Application No. 16 / 656965, filed October 18, 2019, entitled “Backscatter Imaging for Precision Agriculture” and granted July 14, 2020, as U.S. Patent No. 10,712,293, which is a continuation-to-file of U.S. Patent Application No. 15 / 758134, filed March 7, 2018, which is a 371 national phase application of PCT / US2016 / 050545, filed September 7, 2016, with priority dependent on U.S. Provisional Application No. 62 / 337971, filed May 18, 2016, and U.S. Provisional Application No. 62 / 215456, filed September 8, 2015. The contents of all the foregoing referenced applications, including their attachments, are incorporated herein by reference. Technical Field

[0004] This invention relates to apparatus and methods for obtaining agriculturally important crop data from X-ray backscatter imaging and characterization. Background Technology

[0005] For the past 10,000 years, crops have been grown based on unsystematically collected data. Now that humans have utilized electromagnetic radiation that penetrates vegetation, it should be possible to more systematically assess the internal condition of crops, but the obstacles described below have hindered their application to date.

[0006] Precision agriculture describes a management technique based on crop and soil data measured in the field as a function of location and time. Over time, the correlation between the data and its location in the field is used to make farm management decisions to maximize overall returns. Data collection can include information on four main aspects: farm environment, soil, plants, or the final crop. Significant value can be gained in terms of crop yield and quality if a method can provide specific and accurate data on many key areas for a wide variety of crops, especially specialty crops. Specialty crops include high-value fruits, vegetables, and nuts that are used as food or medicine sold directly to consumers and therefore require extremely high aesthetic quality. For these crops, accurate yield forecasting and plant health are of great value. Early and accurate measurement of crop yields enables the preparation of equipment and resources for harvesting, packing, and storage, and allows for accurate pricing of the crop. For tree- and vine-based specialty crops, year-after-year plant maintenance and health are also crucial. Collecting yield data and plant health data year after year will enable forecasting tools to be used for fertilization, irrigation, and harvest planning.

[0007] Crop yield estimation is an important task in the management of many crops, including orchards such as apples. Fruit crops such as apples, citrus, and grapes consist of starch-rich, relatively sparse plants (leaves, branches, and stems) and dense, water-saturated fruits. Current estimation techniques rely on statistical sampling using humans to provide yield estimates, which is time-consuming, labor-intensive, and inaccurate.

[0008] According to Wang et al., “Automated Crop Yield Estimation for Apple Orchards,” 13th International Symposium on Experimental Robotics (ISER 2012), which is incorporated herein by reference.

[0009] Accurate yield forecasting helps growers make better decisions regarding thinning intensity and harvest labor scale, thereby improving fruit quality and reducing operating costs. This also benefits the packaging industry, as managers can use the assessment results to optimize packaging and storage capacity. Typical yield estimates are based on historical data, weather conditions, and manual calculations of fruit (e.g., apple) yields by workers at multiple sampling points. This process is time-consuming and labor-intensive, and the limited sample size is often insufficient to reflect the yield distribution across the entire orchard, especially in orchards with high spatial variability. Therefore, current yield estimation practices are inaccurate and inefficient, and improving current practices will be a significant outcome for the industry. (Ibid., p. 1)

[0010] Although X-ray scattering has been observed for some time, the mechanism by which X-ray quanta are scattered by electrons was first described by Compton in “On the Mechanism of X-Ray Scattering,” Proc. Nat. Acad. Sci., vol. 11, pp. 303-06 (1925), which is incorporated herein by reference and has since been referred to as “Compton scattering.” Previous suggestions for using X-ray backscattering to characterize plant material were limited to applications where agricultural products had been harvested and processed under controlled conditions. These include actual handling of food during processing, as discussed by Cruvinel et al. in “Compton ScatterTomography for Agricultural Measurements,” Eng. Aric. Jaboticabal, vol. 26, pp. 151-60 (2006) and U.S. Patent No. 7734012 to Boyden et al., both of which are incorporated herein by reference.

[0011] Fruit growers desire automated systems for crop yield estimation. Current technology focuses on visual imaging systems. Visual environment-based estimation is challenging due to variable lighting, occlusion caused by foliage, and multiple counts. Occlusion from foliage can lead to multiple counts because it is difficult to view the entire crop. Visual imaging processing can also be advantageously computationally intensive.

[0012] According to the present invention, applying X-ray backscattering to unharvested crops that are still alive in the field offers particular benefits, but requires specialized techniques unknown to date in the art. These specialized techniques and benefits will be described in detail below. Summary of the Invention

[0013] The following embodiments and aspects thereof are described and illustrated in conjunction with systems, tools, and methods. These embodiments and aspects are exemplary and illustrative, and not intended to limit the scope. Numerous embodiments are disclosed in this application.

[0014] In some embodiments, this specification discloses a method for estimating crop weight, wherein the crop comprises at least one row of plants, and wherein the at least one row of plants comprises at least one resulting vine and / or branch, the method comprising: irradiating a predetermined region of the crop with X-rays from at least both sides; obtaining scanned images of the at least one row of plants; performing contrast enhancement and denoising on each collected scanned image; performing global registration of all contrast-enhanced and denoised images; obtaining segmented images representing individual vines and / or branches by separating and discarding the edges of at least one resulting vine and / or branch in the at least one row of plants; performing local registration of the obtained segmented images; performing a clustering piecewise function on each segmented image; processing the obtained segmented images using distance calibration; and estimating the weight of the crop by using the distance-calibrated images.

[0015] This specification discloses a method for estimating crop weight, wherein the crop comprises at least one row of plants, and wherein the at least one row of plants comprises at least one resulting vine and / or branch, the method comprising: irradiating a predetermined region of the crop with X-rays from at least both sides; obtaining scanned images of the at least one row of plants; performing contrast enhancement and denoising on each collected scanned image; performing global registration of all contrast-enhanced and denoised images; obtaining segmented images representing individual vines and / or branches by separating the vines and / or branches from the images; performing local registration of the obtained segmented images; performing a clustering piecewise function on each segmented image; processing the obtained segmented images using distance calibration; and estimating the weight of the crop by using the distance-calibrated images.

[0016] Optionally, the step of collecting at least one row of scanned image data of the plant includes flipping the scanned image generated by irradiating a predetermined area of ​​the crop with X-rays in the same predetermined direction.

[0017] Optionally, separating vines and / or branches from the image includes cutting off the separated vines and / or branches from the image by discarding the edges of at least one resulting vine and / or branch in at least one row of plants.

[0018] Optionally, performing local registration includes obtaining alignment between pairs of images showing different views of the same vine and / or branch.

[0019] Optionally, the method also includes using distance-calibrated images to predict crop yield.

[0020] Optionally, obtaining scanned images of at least one row of plants includes: extracting image data from a raw data file; creating a schematic diagram of each predefined region; plotting the movement of a vehicle carrying a scanning device for irradiating the crop with X-rays on the schematic diagram; and determining the GPS coordinates of a point on the schematic diagram by correlating at least one timestamp of one or more GPS coordinates with at least one timestamp when the point on the schematic diagram was captured. Optionally, the method further includes locating the plant row along a corresponding direction based on the vehicle's direction of movement and flipping the located plant row in a predetermined direction to obtain a consistent plant sequence in each obtained scanned image. Optionally, the method further includes normalizing each obtained scanned image using a predefined normalization bar. Optionally, the method further includes scanning the GPS coordinates of each predetermined region to obtain the distance between plant rows within the region.

[0021] Optionally, the method also includes identifying and annotating segmented images.

[0022] Optionally, the method also includes determining a clustering technique for processing each segmented image.

[0023] Optionally, the method also includes processing the segmented image by using a coarse clustering segmentation method.

[0024] Optionally, the clustering segmentation function can be a classic clustering segmentation function or a deep learning clustering segmentation function.

[0025] Optionally, the estimated weight of fruit hanging on the plant is determined by determining the change in X-ray signal backscattered from the fruit over a predetermined time period. Optionally, the fruit includes one of grapes, berries, citrus fruits, apples, melons, and tomatoes. Optionally, the X-ray signal backscattered from the fruit is proportional to the fruit's mass and the distance from the fruit to a scanning system that generates X-rays for irradiating the fruit. Optionally, the X-ray signal backscattered from the fruit is proportional to the square of the distance from the fruit to the scanning system. Optionally, the total mass of the fruit is determined by integrating the signal intensity of the X-ray signal backscattered from the fruit through the crop. Optionally, the method further includes performing dual-view data acquisition by scanning the fruit simultaneously using two X-ray scanners. Optionally, the method further includes performing dual-view data acquisition by scanning the fruit using a single X-ray scanner with multiple acquisitions. Optionally, the X-ray scanner is located outside the field fruiting area, with the scanner positioned on opposite sides of a row of fruit plants. Optionally, the method further includes collecting images of the fruit and analyzing the images using a distance normalization process at the pixel or feature level.

[0026] This specification also discloses a method for determining crop quality using at least one X-ray scanner, the method comprising: obtaining at least two scan images of the crop, wherein the first of the at least two scan images is obtained along a first plane relative to the crop, and the second of the at least two scan images is obtained along a second plane relative to the crop, and wherein the first plane is angularly shifted relative to the second plane; registering the first scan image and the second scan image; correcting the registered first and second scan images; and determining the quality of the crop from the corrected first and second scan images.

[0027] Optionally, the first plane is angularly shifted relative to the second plane by an angle between 90 and 270 degrees.

[0028] Optionally, the first plane and the second plane are parallel to each other.

[0029] Optionally, registering the first scan image and the second scan image includes matching the first scan image and the second scan image by flipping and translating the other relative to at least one of the first scan image and the second scan image.

[0030] Optionally, obtaining at least two scanned images of the crop includes scanning the crop simultaneously using two X-ray scanners.

[0031] Optionally, obtaining at least two scanned images of the crop includes scanning the crop using a single X-ray scanner and performing multiple scans.

[0032] Optionally, the first and second scan images for calibration registration include calibrating the scan images for a plurality of predefined parameters.

[0033] Optionally, the calibration of the first and second scan images includes correcting one or more of the contrast, brightness, intensity, or scale of the scan images.

[0034] Optionally, determining crop quality from the corrected first and second scanned images includes identifying one or more fruit clusters in the scanned images and analyzing the strength of the clusters pixel by pixel. Optionally, the method further includes summing and correlating the analyzed strengths of the clusters over a predetermined time period.

[0035] This specification also discloses a system for determining crop quality, comprising: at least one X-ray scanner for acquiring at least two scan images of the crop, wherein a first of the at least two scan images is acquired along a first plane relative to the crop, and a second of the at least two scan images is acquired along a second plane relative to the crop, wherein the first plane is angularly shifted relative to the second plane; and a controller coupled to the X-ray scanner, wherein the controller is adapted to: register the first and second images; correct the registered first and second images; and determine the quality of the crop from the corrected first and second scan images.

[0036] Optionally, the first plane is angularly shifted relative to the second plane by an angle between 90 and 270 degrees.

[0037] Optionally, the first plane and the second plane are parallel to each other.

[0038] Optionally, registering the first scanned image and the second scanned image includes matching the first image and the second image by flipping and translating the other relative to at least one of the first image and the second image.

[0039] Optionally, the system includes two X-ray scanners for simultaneously acquiring at least two scanned images of the crop.

[0040] Optionally, at least one X-ray scanner is used to scan the crop at least twice to obtain at least two scan images of the crop.

[0041] Optionally, the first and second scanned images for calibration registration include correcting the images for a plurality of predefined parameters.

[0042] Optionally, the first and second scanned images for calibration registration include images calibrated for one or more of contrast, brightness, intensity, or scale.

[0043] Optionally, determining crop quality from the corrected first and second scanned images includes identifying one or more fruit clusters in the images and analyzing the strength of the clusters pixel by pixel.

[0044] Optionally, the system also includes summing and correlating the analytical strengths of clusters over a predetermined time period.

[0045] This specification also discloses a system for estimating crop weight, wherein the crop comprises at least one row of plants, and wherein the at least one row of plants comprises resulting vines and / or branches. The system comprises: at least one X-ray scanner for irradiating a predetermined area of ​​the crop with X-rays from at least both sides; and a controller coupled to the at least one X-ray scanner, wherein the controller is adapted to: acquire scanned images of at least one row of plants; perform contrast enhancement and denoising on each acquired scanned image; perform global registration of the contrast-enhanced and denoised images; acquire segmented images by separating the vines and / or branches from the images; perform local registration of the acquired segmented images; perform a clustering segmentation function on a predefined set of segmented images; process the segmented images using distance calibration; and estimate the weight of the crop by using the distance-calibrated images.

[0046] Optionally, collecting at least one row of scanned image data of the plant includes flipping scanned images generated by irradiating a predetermined area of ​​the crop with X-rays in the same predetermined direction.

[0047] Optionally, obtaining a segmented image includes cutting off the separated vines and / or branches from the image by discarding the edges of the vines and / or branches.

[0048] Optionally, performing local registration includes obtaining alignment between pairs of images showing different views of the same plant.

[0049] Optionally, the system can be used to predict crop yield using distance-calibrated images.

[0050] Optionally, obtaining scanned images of at least one row of plants includes: extracting image data from a raw data file; creating a schematic diagram of each predefined region; plotting the movement of a vehicle carrying a scanning device for irradiating the crop with X-rays on the schematic diagram; and determining the GPS coordinates of a point on the schematic diagram by correlating at least one timestamp of one or more GPS coordinates with at least one timestamp when the point on the schematic diagram was captured.

[0051] Optionally, the controller positions the plant rows along the corresponding direction based on the vehicle's direction of motion; and flips all the positioned plant rows in the predetermined direction to obtain a consistent plant sequence in each obtained scan image.

[0052] Alternatively, the controller normalizes each acquired scan image by using a predefined normalization bar.

[0053] Optionally, the controller instructs the X-ray scanner to scan the GPS coordinates of each predetermined area to obtain the distance between plant rows in the area.

[0054] Optionally, the controller identifies and annotates segmented images.

[0055] Alternatively, the controller processes the segmented image using a coarse clustering segmentation method.

[0056] Optionally, the controller executes a clustering segmentation function, either a classic clustering segmentation function or a deep learning clustering segmentation function.

[0057] Optionally, the controller determines the clustering processing technique used to process each segmented image.

[0058] Optionally, the controller determines the weight of the fruit hanging on the plant by determining the change in the X-ray signal backscattered by the fruit over a predetermined time period.

[0059] Alternatively, the fruit may include one of the following: grapes, berries, citrus fruits, apples, melons, and tomatoes.

[0060] Optionally, the X-ray signal backscattered from the fruit is proportional to the mass of the fruit and the distance of the fruit from the scanning system that generates the X-rays used to irradiate the fruit. Optionally, the X-ray signal backscattered from the fruit is proportional to the square of the distance of the fruit from the scanning system.

[0061] Alternatively, the total mass of the fruit can be determined by integrating the signal intensity of the X-ray signal backscattered from the fruit through the crop.

[0062] According to an embodiment of the present invention, a method for remotely characterizing a living plant is provided. The method comprises the following steps: a) generating a first beam of penetrating radiation; b) scanning the beam through a living plant; c) detecting Compton scattering from the living plant from the first beam of penetrating radiation to generate a first scattering signal; and d) processing the scattering signal to obtain one or more features of the living plant.

[0063] In the embodiments, it should be noted that Compton backscattered X-ray characterization, especially imaging, is sensitive to materials with low effective atomic numbers, such as water and organic materials.

[0064] Some embodiments of the invention may use a beam collimated in one dimension. Other embodiments may use a beam collimated in two dimensions, referred to as a pencil beam. Some embodiments of the invention may derive features such as water content, root structure, branch structure, xylem size, fruit size, fruit shape, fruit aggregate volume, cluster size and shape, fruit maturity, and images of a portion of a living plant. In some embodiments, penetrating radiation may include X-rays. In other embodiments, X-rays may include photons ranging from 50 keV to 220 keV.

[0065] Other embodiments of the invention may use parallel sensing modes to acquire data. In some embodiments, the sensing mode may include at least one of visible light, microwave, terahertz, and ultrasound. Other embodiments may use data acquired using parallel sensing modes to register an image of a living plant relative to a reference frame of the living plant. In some embodiments of the invention, the living plant may be illuminated with a pencil beam emitted from a transport vehicle. In other embodiments, the living plant may be illuminated from above. Further embodiments may include illuminating the living plant from a position horizontally displaced relative to the living plant. In other embodiments, the beam may be generated in a scanning head deployed on an articulated arm. Other embodiments may further include a gripper disposed on a robotic arm and an X-ray source with pencil beam collimation. Further embodiments may include obtaining characteristics of the roots of the living plant. In other embodiments of the invention, penetrating radiation may be emitted simultaneously into two half-spaces. In other embodiments, scanning may include electronically guiding the penetrating radiation beam.

[0066] In other embodiments of the invention, radiation can pass through a defined aperture. In a further embodiment, the defined aperture can be adjusted during the scanning of the beam. In other embodiments, scattering can be spectrally resolved. This resolution can be achieved by modulating the spectral content of the first penetrating radiation or by detection that is differentially sensitive to different spectral features.

[0067] In other embodiments of the invention, organic features can be distinguished based on unique spectral characteristics. In a further embodiment of the invention, at least one of the position and orientation of the pencil beam in the reference frame of the living plant can be monitored, and the pencil beam can be manipulated in a closed loop to maintain a specified path of the pencil beam in the reference frame of the living plant. In other embodiments of the invention, an image of the scattered signal can be registered relative to the reference frame of the living plant. In other embodiments of the invention, both the first and second beams of penetrating radiation can be used to generate a second scattered signal, and the second scattered signal can be processed to derive a second feature of the living plant. In other embodiments of the invention, at least one derived feature is related to water absorption. In a further embodiment of the invention, the first and second scattered signals can be used to derive three-dimensional features of the plant. The derived three-dimensional feature can be a stereo image. In other embodiments of the invention, the first and second scattered signals can be used to generate spatial coordinates of the living plant relative to other living plants as a whole, spatial coordinates of the living plant relative to the base of the living plant, or spatial coordinates of the fruit relative to another part of the living plant. In a further embodiment, a gripper can be mounted on a robotic arm, and a closed-loop feedback control system can be used to position the gripper. In some embodiments of the invention, the first and second beams can be scanned from a transport vehicle. In other embodiments of the invention, the first and second beams can each scan relative to corresponding first and second central rays and can be shifted relative to each other by a certain angle. This angle can be in the range of 45 degrees and 135 degrees. In a further embodiment of the invention, the first and second scattering signals can be used to generate spatial coordinates of at least one object other than the living plant located between the transport vehicle and the living plant, and using these coordinates, a topographic map can be generated.

[0068] According to another embodiment of the present invention, a method for measuring ground water content is provided. The method comprises the steps of: a) generating a penetrating radiation beam; b) scanning the beam on the ground; c) detecting Compton scattering from the ground from the penetrating radiation beam; and d) processing the scattering signal from the Compton scattering relative to a reference sample to derive the ground water content.

[0069] According to another embodiment of the present invention, a method for guiding robot movement is provided. The method comprises the following steps: a) generating a first beam of penetrating radiation; b) scanning the first beam through a living plant; c) detecting Compton scattering from the living plant originating from the first beam of penetrating radiation to generate a scattering signal; d) processing the scattering signal to obtain features of the living plant; and e) using the derived features of the plant to guide a gripper of the robot to grasp a portion of the living plant.

[0070] Further embodiments may include harvesting plants using a gripper. In other embodiments of the invention, the gripper may be mounted on a robotic arm, and the system may use a closed-loop feedback control system to position the gripper.

[0071] The above and other embodiments of this specification will be described in more detail in the accompanying drawings and detailed description provided below. Attached Figure Description

[0072] These and other features and advantages of this specification will be better understood when considered in conjunction with the accompanying drawings and with reference to the detailed description, and will be further understood when considering these features and advantages.

[0073] Figure 1 The graph shows the relative absorption of electromagnetic radiation by water as a function of wavelength or photon energy.

[0074] Figure 2 A rear view of a horizontal backscatter scanning apparatus according to an embodiment of this specification is shown;

[0075] Figure 3 A duplex scanning device according to an embodiment of this specification is shown;

[0076] Figure 4 A top-down backscatter scanning apparatus according to an embodiment of this specification is shown;

[0077] Figure 5 An exemplary dual-energy detector scanning apparatus according to an embodiment of this specification is shown;

[0078] Figure 6 An exemplary X-ray backscattered image of an agricultural scene according to an embodiment of this specification is shown;

[0079] Figure 7 A flowchart illustrating the steps of a scanning method according to an embodiment of this specification is shown;

[0080] Figure 8 A second flowchart illustrating the steps of a method for guiding robot motion according to an embodiment of this specification is shown;

[0081] Figure 9 A fan-beam scanning apparatus using fan-beam illumination according to an embodiment of this specification is shown;

[0082] Figure 10 A graph illustrating the use of volume information from the scanning process to quantify the fruit according to an embodiment of this specification is shown.

[0083] Figure 11 This is a table listing exemplary types of characteristic crops to which backscatter imaging can be applied, according to embodiments of this specification;

[0084] Figure 12A This is a flowchart illustrating a method for estimating crop weight and predicting crop yield according to embodiments of this specification;

[0085] Figure 12B This is a flowchart illustrating the steps of collecting image data of a row of plants according to an embodiment of this specification;

[0086] Figure 12C A graph showing image data before and after normalization according to an embodiment of this specification is provided.

[0087] Figure 13A These are scanned images before and after contrast enhancement according to embodiments of this specification;

[0088] Figure 13B These are contrast-enhanced images before and after denoising according to embodiments of this specification;

[0089] Figure 14A It is an image on which global registration is performed;

[0090] Figure 14B An image illustrating the global registration result according to an embodiment of this specification is shown;

[0091] Figure 15A Successful segmentation images according to embodiments of this specification are shown;

[0092] Figure 15B A segmentation image illustrating a false positive is shown according to an embodiment of this specification;

[0093] Figure 16A The image shown is a segmented image after local registration has been applied, according to an embodiment of this specification;

[0094] Figure 16B Coarse feature annotations are shown on the segmented image according to an embodiment of this specification;

[0095] Figure 16C Registration of segmented images using coarse feature annotations is shown according to an embodiment of this specification;

[0096] Figure 16D A close-up illustration of the side of a segmented image with coarse feature annotations is shown according to an embodiment of this specification;

[0097] Figure 17A Clustering annotation images according to embodiments of this specification are shown;

[0098] Figure 17B A coarse segmented network according to an embodiment of this specification is shown;

[0099] Figure 17C A graph showing the training loss for each epoch in the coarse segmentation network when training is performed on 50 pre-harvest data images;

[0100] Figure 17D The diagram shows the validation loss for each epoch in the coarse segmentation network when training is performed on 50 pre-harvest data images with 5 validation sets;

[0101] Figure 17E This is a block diagram illustrating the application of distance calibration to an image obtained by scanning an object from two opposite sides, according to an embodiment of this specification;

[0102] Figure 17F A graph showing the correction factor C versus intensity ratio according to an embodiment of this specification is provided.

[0103] Figure 18A The illustrations show photographs of fruits from the pre-flowering season to the harvest season, along with corresponding X-ray backscattered images of the fruits, according to embodiments of this specification.

[0104] Figure 18B The embodiments according to this specification are shown. Figure 18A Backscattered X-ray scan image of the fruit photograph shown;

[0105] Figure 18C This is a table, according to embodiments of this specification, for tracking the weight of individual fruit vines and the correlation of corresponding scanned images over a predetermined time period.

[0106] Figure 18D The curves of vine weight versus scan data corresponding to multiple fruit vines over a period of time are shown according to embodiments of this specification.

[0107] Figure 18E The embodiments according to this specification are shown. Figure 18D The curve shown is a normalized curve of vine weight against scan data;

[0108] Figure 19A This is a graph illustrating the relationship between scattering intensity and cluster size according to embodiments of this specification;

[0109] Figure 19B This is a graph illustrating the relationship between the scattered signal and the distance between the scanner and the fruit according to an embodiment of this specification;

[0110] Figure 20A A region analysis method according to an embodiment of this specification is illustrated;

[0111] Figure 20BThis is a graph showing the correlation between the ground true cluster weight plotted on the y-axis 2020 and the X-ray backscattering intensity plotted on the X-axis 2021 under different application correction factor values ​​according to embodiments of this specification;

[0112] Figure 21A A scanning system for scanning crops to predict the weight of hanging fruit, according to an embodiment of this specification, is shown;

[0113] Figure 21B The embodiments of this specification are shown in Figure 21A The contrast block used in the scanning system;

[0114] Figure 21C The scanned image obtained by using a contrast block according to an embodiment of this specification is shown;

[0115] Figure 21D A graph illustrating the variation of contrasting blocks across different time periods of the day, according to embodiments of this specification, is shown; and

[0116] Figure 22 This is a flowchart illustrating a method for determining the hanging weight of fruits / vegetables on plants / vines / branches according to embodiments of this specification. Detailed Implementation

[0117] As the inventors first recognized, Compton-scattered X-ray radiation in the energy range of approximately 50-220 keV is an ideal probe for crop size and yield because this radiation is effectively scattered from water and appears with bright contrast in Compton X-ray backscatter imaging. X-rays penetrate low-density objects, such as leaves or branches, and consequently show less backscattered signal. This ability to penetrate plants and produce a signal sensitive to water content allows backscattered X-ray imaging to provide data with higher precision than other techniques such as visible light, infrared, or radar imaging.

[0118] The following description provides detailed instruction on the various challenges of applying backscatter imaging to precision agriculture, including yield estimation, crop and plant growth in a season and throughout the year, water management, robotic harvesting, and data fusion and management.

[0119] It should be noted that backscatter imaging of crop yield can be advantageously applied to any crop with isolated or clustered fruits, including trees, vines, or plants. According to some embodiments of the invention, the type, size, and shape of the crop may differ, but the scanning method can remain the same, including linear translation of the imager over the plant.

[0120] Larger apple and citrus trees are typically grown outdoors, while smaller plants such as tomatoes, pumpkins, and melons can be grown in greenhouses or outdoors. In embodiments of the invention, the same basic backscattering system can be used for a variety of applications, and parameters such as beam spectral content or dwell time can be modified for specific applications. Furthermore, crop type applications often differ in the amount of scan time and power required. For example, scanning unmanaged apple or citrus trees may require more power and a slower scan speed than managed trees. Unmanaged trees have greater height and diameter, requiring imaging of trees with greater height and diameter than managed trees, thus requiring higher power and energy. This corresponds to higher power and energy. Managed orchards have managed trees with branches attached to trellises. Unmanaged orchards have unmanaged trees without trellis structures.

[0121] This specification relates to several embodiments. The following disclosure is provided to enable those skilled in the art to practice this specification. The language used in this specification should not be construed as a general denial of any particular embodiment, nor should it be used to limit the claims beyond the meaning of the terms used herein. The general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of this specification. Furthermore, the terminology and wording used are for the purpose of describing exemplary embodiments and should not be considered restrictive. Therefore, this specification is to be accorded the widest scope, including numerous alternatives, modifications, and equivalents consistent with the disclosed principles and features. For clarity, details relating to technical materials known in the art related to this specification have not been described in detail so as not to unnecessarily obscure this specification.

[0122] In the specification and claims of this application, each of the words “comprising,” “including,” and “having,” and their forms, is not necessarily limited to members of the list that the word may be associated with. It should be noted here that, unless explicitly stated otherwise, any feature or component described in connection with a particular embodiment may be used and implemented with any other embodiment.

[0123] While X-ray backscattering has been used to characterize many materials, characterizing crops in vivo using backscattering techniques has long been considered impossible due to the inherent geometric irregularities and variations caused by motion. This is unacceptable in current applications because users lack control over the sample's position and stability. Without control and regularity, accurate crop yield estimates are impossible. Crop yield estimation is a crucial task in the management of many crops, such as apples, grapes, and cucumbers. Accurate yield forecasts help growers make better decisions regarding thinning intensity and harvest labor scale, thereby improving fruit quality and reducing operating costs. Managers can use the estimates to optimize packing and storage capacity. Scanners capable of assessing product size and quantity are suitable for total crop yield estimation; spatial mapping of water, fertilizer, and pesticide distribution; detection, damage assessment, and mapping of diseases, insects, or other causes of crop loss; temporal analysis of crop growth rates and development; and testing the efficacy of fertilizers or pesticides.

[0124] Fruit crops such as apples, citrus fruits, grapes, and others consist of low-density features rich in starch (leaves, branches, and stems) and dense, water-saturated fruit. X-ray radiation in the energy range of approximately 100 keV is effectively scattered from water and appears as bright contrast in Compton X-ray backscattering imaging. X-rays penetrate low-density objects, such as leaves or branches, and accordingly show less backscattered signal.

[0125] In addition to the number of crops in the image, 3D location also helps prevent double counting and better understand the target location of pesticides and fertilizers. 3D location can be estimated by including multiple X-ray irradiation angles. Furthermore, the precise location of the X-ray source can be recorded. To correct for unevenness in ground elevation, it is preferable to record the sensing of the source's location and orientation. In some embodiments, a Global Positioning System (GPS) can also be used for field positioning, optionally combined with GPS for fertilization and pesticide application.

[0126] Embodiments of the present invention can integrate multiple sensing systems onto a mobile transport vehicle to achieve an accurate, rapid, and automated detection system for crop yield estimation. Embodiments may include a combination of an X-ray system with position and orientation sensing, which prevents double counting when scanning from both sides of the crop row.

[0127] Further embodiments may include an X-ray system comprising multiple viewpoints that can be used to calculate the 3D position of the crop. In some embodiments, the X-ray imaging system is optimized in terms of source and detector design to maximize sensitivity, thereby increasing inspection speed. Specifically, a fast scan rate can be achieved using an coded aperture configuration of the source and area detectors. A complete description of X-ray backscattering detection techniques using coded apertures can be found in U.S. Patent No. 5,940,469 to Huang et al., which is incorporated herein by reference.

[0128] Determining crop yield requires measuring scattering intensity within a specific spectral range and at a specific spatial resolution, all of which vary depending on the crop involved. An example is viticulture, where yield can be determined by measuring scattering variations in the 140-220 keV range at a resolution of 1-4 mm. To screen growing crops from a distance of approximately 1 meter at a speed of 1 km / h, a detection sensitivity of 500-600 photons / steradian / second / root-hertz is required. This was considered impossible, and so was the invention's unique ability to penetrate the intervening growth between the fruit and the scanner, as described in detail below, before the inventors recognized that a pencil-beam X-ray source for generating Compton backscatter signals from a moving inspection platform could achieve the necessary sensitivity.

[0129] For large areas that typically need to be covered (usually square miles), scanning pencil beams may not be feasible. For example, obtaining 1 mm from a 1 m interval. 2 The resolution means 10 -6 The angular resolution of SR, or the aperture of 0.06° in the rotating ring. Based on 5 × 10⁸ photons / second / cm. 2 Calculations of the incident flux of / mA, the spacing distance of 1 meter, and the signal-to-noise ratio of a representative beam spectrum indicate that for a pencil beam generated by a rotating aperture, there is a dwell time of at least 2.5 μs per pixel. This implies that the aperture ring rotates at a speed of 3000 revolutions per second.

[0130] A scene containing 1-meter-tall fruits (filling a 10-acre square with rows spaced 3 meters apart) contains approximately 10^10 pixels, thus requiring 24 hours of scanning with a pencil beam. In many cases, such scanning fails to provide time-scale data useful for agricultural insights. Fan beams, or alternatively conical beams, using coded aperture technology can advantageously increase throughput, thereby increasing the scan rate by more than 100 times. In the case of a fan beam, this simply stems from the simultaneous detection of approximately 100 collinear pixels.

[0131] For example, in the field of viticulture, using a 160keV X-ray source with a dissipation of 1000W, a chopper wheel with eight spokes rotating at 1800rpm can scan simultaneously in two directions to obtain bilateral data, moving along the rows at a speed of 2.18mph, and can scan a 1-acre square vineyard with 130-inch row spacing in 0.3 hours.

[0132] Definition: "Transport vehicle" refers to any device characterized by a platform supported on a ground contact member, such as wheels, tracks, pedals, slides, etc., used to transport equipment from one location to another.

[0133] The term “trailer” as used herein and in any of the appended claims means a means of transport suitable for being towed on a surface by a motor vehicle or means of transport, and may be referred to herein as a “tractor.”

[0134] The term "image" as used herein and in any appended claims means any multidimensional representation, whether tangible or otherwise perceptible, or otherwise, in which values ​​of some properties (amplitude, phase, etc.) are associated with each of multiple locations corresponding to the dimensional coordinates of an object in physical space, although not necessarily mapped one-to-one. Thus, for example, a graphical display of the spatial distribution of some fields, or scalar or vector quantities such as brightness or color, or X-ray scattering intensity, constitutes an image. The same applies to a set of digital data in computer memory or a holographic medium, such as a 3D holographic dataset. Similarly, "imaging" means presenting a specified physical feature in the form of one or more images.

[0135] The term “X-ray source” should refer to an apparatus that generates X-rays, including but not limited to an X-ray tube or a bremsstrahlung target bombarded by high-energy particles, without regard to the mechanism used to generate X-rays, including but not limited to linear accelerators.

[0136] "Half-space" refers to each of the two parts in which a three-dimensional space is divided by an imaginary plane.

[0137] "Living plant" refers to an unharvested plant, that is, a living plant still attached to the source that provides it with nutrients for growth. Furthermore, for the purposes of this instruction manual, if plant matter consumes energy during its growth, then it should be referred to as "living."

[0138] A "gripper," also known in the art as an "end-of-arm tool," is a remotely operated device for holding and manipulating objects. It can be a claw, a bag, a suction device, etc., as is well known in the art and is included within the scope of this invention.

[0139] A "robotic arm" is an actuating mechanical linkage device that connects a gripper to a base, which can be a vehicle or other means of transportation.

[0140] As used in this article, the term "part" refers to an object and should include a portion or all of the object.

[0141] A "scanner" is a device for moving the direction of a beam of light propagation. Examples of scanners that can be used with an X-ray beam are described, for instance, in U.S. Patent No. 9,014,339 ("Versatile X-Ray Beam Scanner" by Grodzins et al.), and include mechanical scanners such as rotating rings, X-ray tubes with rotating anode apertures as described in U.S. Patent No. 9,099,279 ("X-Ray Tube with Rotating Anode Aperture" by Rommel et al.), or electronic scanners such as those described in U.S. Patent No. 6,282,260 ("Unilateral Hand-held X-Ray Inspection Apparatus" by Grodzins). All of these scanners are within the scope of this invention.

[0142] The term "scanning head" is used more generally to refer to the hardware that guides the light beam, and may include, for example, a radiation source scanned by a scanner.

[0143] A “scanning device” refers to a system used to characterize a scanned object or scene based on radiation (such as X-rays) scanned across the scene. A scanning device may include detectors for detecting scattered radiation, and may also include a transport vehicle for transporting the scanning head and detectors, as well as auxiliary equipment such as power supplies and sensors.

[0144] A "fan beam" is a beam collimated in one dimension transverse to the direction of beam propagation.

[0145] A "pencil beam" is a beam collimated in two dimensions transverse to the direction of beam propagation.

[0146] "Organic characteristics" are the features of organic matter (such as plants) that distinguish organic matter from inorganic matter. Organic characteristics may include, but are not limited to, water content, root structure, branch structure, xylem size, fruit size, fruit shape, fruit aggregate volume, cluster size, cluster shape, and fruit maturity.

[0147] "Ground real vines" are vines monitored over a predetermined period of time, and imaged before and after harvesting and weighing.

[0148] In various embodiments, the system of this specification includes a computing device having one or more processors or a central processing unit, one or more computer-readable storage media such as RAM, a hard disk, or any other optical or magnetic medium, a controller such as an input / output controller, at least one communication interface, and system memory. The system memory includes at least one random access memory (RAM) and at least one read-only memory (ROM). In embodiments, the memory includes a database for storing raw X-ray data, scanned images, processed images, and data associated with these images. Multiple functional and operational elements communicate with the central processing unit (CPU) to enable the computing device to operate. In various embodiments, the computing device may be a conventional standalone computer, or alternatively, the functionality of the computing device may be distributed across a network and / or cloud computing system of multiple computer systems and architectures. In some embodiments, the execution of a plurality of program instructions or code sequences stored in one or more non-volatile memories enables the CPU of a computing device to perform various functions and processes, such as receiving raw X-ray data, such as image data, and applying various processing steps (e.g., but not limited to contrast enhancement, denoising, global registration, local registration, clustering, cluster segmentation, deep learning processes, calibration, and weight estimation) to provide images and other data, such as but not limited to crop weight or yield information, for display on a screen. In alternative embodiments, hardwired circuitry may be used in place of or in combination with software instructions for implementing the systems and methods described herein. Therefore, the described systems and methods are not limited to any particular combination of hardware and software.

[0149] Backscattering imaging utilizes a pencil beam of X-ray radiation irradiating an object and detects Compton scattered radiation from the object. The contrast in backscattering imaging depends on the material composition. Biomaterials composed of low atomic number materials, including hydrogen, carbon, nitrogen, and oxygen, exhibit strong contrast in backscattering X-ray imaging due to two competing effects: Compton scattering generation and X-ray absorption. The Compton scattering cross-section increases proportionally to the atomic number (Z), so materials with higher Z scatter more. However, photoelectric absorption is proportional to Z⁴ / E³, so for higher Z, especially at lower energies, photoelectric absorption prevents Compton scattered X-rays from leaving the target material. Compton scattered X-ray radiation in the energy range of approximately 50–220 keV is an ideal probe for crop size and yield because this radiation is effectively scattered from water and exhibits bright contrast in Compton X-ray backscattering imaging. This energy is used for imaging deep soil penetration and / or plant roots. While operation in the 50–220 keV range is described, this range is provided by way of example and is not intended to be limiting.

[0150] The absorption spectrum of water vapor is Figure 1Plotted as a function of wavelength (in meters) or photon energy (in eV), it highlights the significant attenuation in the short-wavelength region of the visible spectrum, dominated by the photoelectric effect in the X-ray region. Therefore, shorter wavelengths of radiation are not suitable for detecting, imaging, or characterizing crops from any considerable distance as required in agricultural environments. Because imaging performance deteriorates at shorter wavelengths, it is counterintuitive that Compton scattering X-ray imaging meets all the criteria for good yield estimation based on radiation penetration, scattering, and imaging performance.

[0151] Backscatter imaging of crop yield can be advantageously applied to any crop with isolated or clustered fruits, including trees, vines, or plants. According to some embodiments of the invention, the type, size, and shape of the crop may differ, but the scanning method can remain the same, including linear translation of the imager over the plant.

[0152] Larger apple and citrus trees are typically grown outdoors, while smaller plants such as tomatoes, pumpkins, and melons can be grown in greenhouses or outdoors. In embodiments of the invention, the same basic backscattering system can be used for a variety of applications, and parameters such as beam spectral content or dwell time can be modified for specific applications.

[0153] According to certain embodiments of the present invention, in Figure 2 The scanning device, typically designated by the numeral 101, has a scanning head located at the end of the arm assembly. The scanning device 101 can be used for horizontal scanning or top-down scanning.

[0154] Figure 2 A rear view of a horizontal backscatter scanning apparatus 101 according to an embodiment of the present invention is shown. In this embodiment, a scanning head 100 is mounted on the arm 102 of a robot 104. Furthermore, a gripper 106 is also coupled to the arm 102 of the robot 104. The robot 104 may be carried on a transport vehicle 108, and may be self-propelled, autonomous, or guided. In some embodiments of the invention, as described above, the transport vehicle 108 is a tractor, truck, or motorized handcart. Furthermore, the transport vehicle may have scanning electronics 113 and one or more sensors 110. In some embodiments, the horizontal backscatter scanning apparatus 101 may advantageously utilize Compton backscatter X-ray imaging, which is sensitive to low atomic numbers, including water and organic materials. Compton backscatter X-ray imaging may advantageously be used to detect fruit 112, which typically contains a concentrated amount of water.

[0155] In some embodiments, for position sensing, the positions of multiple data streams, such as X-rays, visible light, microwaves, submillimeter waves, and infrared light, can be co-registered. In some embodiments, the position sensing device 116 can determine the position of the transport vehicle 108 at any given time. This position is preferably determined with an accuracy of 1 / 5 of the minimum feature size required for imaging. In some embodiments, the orientation of the transport vehicle 108 can also be recorded to maintain the relative irradiation angle and position of the X-ray source 114 relative to the crop.

[0156] In some embodiments of the invention, image resolution is proportional to X-ray source power and inversely proportional to scan speed. Crop type applications typically vary in terms of the required scan time and power. For example, scanning unmanaged apple or citrus trees may require more power and a slower scan speed than managed trees. Unmanaged trees have greater height and diameter, which requires imaging of trees with greater height and diameter than managed trees, thus requiring higher power and energy. This corresponds to higher power and energy. Managed orchards have managed trees where branches are attached to trellises. Unmanaged orchards have unmanaged trees that do not have trellises.

[0157] An X-ray beam produced by bremsstrahlung contains photons with energies up to the endpoint energy, which is equal to the energy of the highest-energy electron striking the bremsstrahlung target. When an X-ray beam is characterized solely by its energy in this document, that energy refers to the endpoint energy of the beam. Therefore, if the endpoint energy of a first X-ray beam exceeds the endpoint energy of a second X-ray beam, the first X-ray beam is said to have a higher energy than the second X-ray beam.

[0158] Compared to the surrounding twigs and leaves, mature crops consist of relatively high concentrations of water, carbon, and nitrogen, referred to here as "interventional material." Further reference Figure 2 For the pencil beam 120 impacting the fruit 112, the Compton scattering signal will increase with the radius of the fruit for some, but not all, of the incident X-ray spectrum, because, as can be readily calculated, multiple scattering and absorption processes occur in the crop. The pencil beam 120 may also be referred to herein as the “input beam” or the “scanning beam.” As the scanning beam 120 passes through the thickness of the plant 127 constituting the crop, a portion of the input beam 120 is scattered from the plant 127. The plant 127 may also be referred to herein as the “object” in relation to its scattered X-rays, and as to the passage of the input beam 120 through the plant 127, it may also be referred to as the “material.”

[0159] The radiation scattered by the fruit 112 into the input beam 120, also referred to herein as scattered radiation 130 (or “backscattered X-rays”), must now return through the same intermediate material to reach the detector 125. Each photon of the backscattered X-rays 130 has a lower energy than the photons of the incident beam 120 that caused the scattering and will be attenuated by the outgoing intervening material. “Outgoing intervening material” refers to the path of the scattered radiation 130 through the plant 127 on its way to the detector 125. Compton scattering causes an energy transfer effect that is a function of the scattering angle and energy. For leaves and small-diameter crops such as citrus or apples, the scattered radiation is not strongly attenuated by the thickness of the fruit itself; however, for very large crops such as melons, significant attenuation of the backscattered X-rays may occur within the fruit itself. In either case, the high scattering of the fruit relative to the low scattering of the intervening material emphasizes the fruit in the detected scattered signal.

[0160] In some embodiments, the horizontal backscatter scanning device 101 can scan plants, fruits, or trees by scanning from the side of the transport vehicle 108. This configuration can have one scanner 100 or dual scanners, such as... Figure 3 As shown.

[0161] Figure 3 A dual-sided horizontal backscatter scanning apparatus, generally designated by the numeral 201, is described according to an embodiment of the invention. In some embodiments of the invention, multiple backscatter X-ray scanning heads 203, 205 are positioned very close to the plant 127, allowing for a panoramic view of the area where the crop can grow. An X-ray source 228 generates X-rays scanned by scanners 203 and 205 to produce a scanning beam 222 for scanning multiple rows at once (e.g., each side of the scanner). In the case of a multi-directional beam, a shield 235 prevents the backscatter detector 203 from detecting the scattering.

[0162] The scanning device used for X-ray imaging (e.g., a dual-sided horizontal backscatter scanning device 201) may also be referred to herein as an "X-ray imager". The X-ray imager 201 can operate continuously or intermittently as the transport vehicle 108 moves. For example, within the scope of the invention, the imaging system may employ flying-spot / pencil beam 222, fan-beam imaging, or coded aperture imaging. According to various embodiments of the invention, three-dimensional information can be collected in a dual-scanner configuration 201. In other embodiments, three-dimensional information can be collected by including a single system that acquires images at variable angles by rotating the imaging system 201 at periodic intervals. In some embodiments, multiple imaging scanners 203, 205 with different viewing angles are included on the transport vehicle 108, wherein the transport vehicle 108 continuously records backscatter signals or coded aperture imaging at multiple angles, which can be used to acquire 3D information. In addition to the X-ray detector 230 (also referred to herein as an X-ray sensor), additional sensing may be included in some embodiments to provide accurate information about the position and orientation of the transport vehicle 108 within a reference frame of the crop 127. This provides control over the orientation of beam 222, or allows for the use of the tracked position to record the backscattered signal within the acquired image (e.g., Figure 6 (As shown). This can advantageously improve the ability to measure the size and / or volume of crop 127. GHz / Thz radiation, thermal / IR, and ultrasound techniques can also be combined with X-ray data to produce 3D images of the scanned scene.

[0163] Figure 4 A view of a top-down backscatter scanning device, generally designated by the numeral 301, is shown according to an embodiment of the invention. The top-down backscatter scanning device 301 has a field of view 310 including a crop 127. In some embodiments, an adjustable-height stand 303 can be provided for crops of various heights. Top-down scanning can be used for plants such as strawberries. According to embodiments of the invention, if the height of the scanning device 301 is increased, multiple results can be obtained simultaneously. In some embodiments of the invention, the scanning head 305, coupled to a transport vehicle 108, is coupled to existing spraying equipment to reduce implementation costs for growers.

[0164] The scanning head 100 can emit X-rays covering a width of approximately 4-5 feet. The large detection area of ​​the X-ray detector 230 will increase the detected flux, thereby increasing the scan rate. Crops such as cotton and melons may not have enough space for side illumination from the top. Increasing the X-ray flux can improve the scan rate, but if additional flux is obtained by increasing the aperture size, it comes at the cost of image resolution. For a typical device 301, beam apertures up to 4 mm can be used. In some embodiments, an coded aperture imaging system can allow for more efficient imaging of point targets with repetitive shapes. In the case of fruit crops, where objects are identical in shape but vary in size and position, an coded aperture X-ray imaging system can be used to efficiently determine the position and size of objects in the field of view. Because the imaging system allows for illumination of the entire area, the flux delivered to the crop in an coded aperture system can be much higher. This can advantageously increase the scan rate of the system.

[0165] Figure 5 An exemplary dual-scanner scanning apparatus according to an embodiment of the present invention is illustrated, generally indicated by the numeral 401. First and second beams 403 and 406 can be scanned from a transport vehicle 108. In other embodiments of the invention, the first and second beams can each be scanned relative to corresponding first and second central rays 405 and 407, and can be displaced relative to each other by an angle. Although the first and second central rays 405 and 407 are parallel in the illustrated embodiment, in other embodiments, the angle between the first and second central rays 405 and 407 can be in the range of 45 degrees and 135 degrees. The emission of beams 403 and 406 is appropriately phased such that the source of the scattered photons arriving at detector 230 is clearly defined.

[0166] Figure 6 An exemplary X-ray backscatter image of an agricultural scene generated according to embodiments of the present invention is shown, typically represented by the numeral 501. In some embodiments of the invention, crop yield mapping may consist of four pieces of information: fruit size, fruit quantity, planting location, and date / time. Fruits 503 may be highlighted in image 501 to aid in obtaining the aforementioned characteristic data. To facilitate historical data, in some embodiments, the scanning device may be coupled to a GPS unit to monitor the location of fruits or plants. Data may be collected, correlated, and stored in a database. Reports may be generated for growers to reflect the collected data. According to embodiments of the present invention, crop health can be advantageously tracked year by year using historical data from the same plants / trees.

[0167] According to some embodiments, in addition to backscatter imaging data used for yield and plant health, other data sources can be combined to build a comprehensive understanding of the local environment. These can advantageously include temperature, humidity, soil conditions, etc.

[0168] In some embodiments, the software may have an easy-to-understand graphical user interface that allows growers to access and display various information in an easy-to-read interface. In some embodiments, reports and graphical representations of historical plant and crop data may be included. This data may include plant health, crop health, crop yield, individual plant health and growth, and various other data.

[0169] Now for reference Figure 7 and 8 The flowchart describes a method according to an embodiment of the present invention. Figure 7 A flowchart illustrating an embodiment of the invention is shown, describing the steps of a method for scanning living plants, generally denoted by the number 601. Specifically, the scanning method 601 can identify characteristics of the living plants, such as organic characteristics, wherein characteristics considered organic are as described above. In some embodiments of the invention, software algorithms can improve noise suppression, dual-count resolution, and crop size / distribution estimation. Furthermore, the algorithm can be used to indicate which crops require more or less water / fertilizer by mapping the field.

[0170] In the first step 602, a penetrating radiation beam is generated. It should be understood that, within the scope of any embodiment of the invention, the source can be time-modulated. In process 604, the generated beam scans across the living plant. In process 606, at least one scattering detector is used to perform Compton scattering detection. In some embodiments of the invention, multiple scattering detectors 125 (e.g., ...) are used. Figure 2 The detection is performed in a manner that (as shown) or otherwise, such that one signal preferentially records high-energy scattering, while a second signal preferentially records low-energy scattering. High-energy scattering and low-energy are terms defined relative to each other and are used herein to refer to providing energy resolution by altering the energy distribution of the incident X-rays or by the energy resolution of the detected scattering, all according to known multi-energy techniques. In the case of energy resolution, the ratio of the backscattered signal from the low-energy scattering detection to that from the high-energy scattering detection is calculated in process 608. This calculation is further explained in U.S. Patent No. 8,442,186, which is incorporated herein by reference. It should be understood that the term "ratio" as used herein includes weighted or corrected ratios, logarithmic differences, etc. Once the ratio of the low-energy to high-energy scattering signals is calculated, it is compared to a threshold to determine the characteristics of the living plant (608).

[0171] In a preferred embodiment of the invention, the ratio R between detections of different energy spectra is compared to a specified threshold pixel-by-pixel in the image, allowing for determination of each pixel in the image. However, in practical systems used for plant inspection, the X-ray intensity is often insufficient to allow for the calculation of the R value of a single pixel with the required accuracy, due to the limited number of scattered photons detectable within the integration time of a single pixel. Therefore, a sub-region of the image consisting of many pixels, such as a 10×10 region, can be analyzed, and the value of R can be calculated based on the total backscattered signal from all pixels in that sub-region.

[0172] In other embodiments of the invention, software algorithms can be used for size estimation, processing and eliminating duplicate counts, and determining crop locations in 3D. In some embodiments, the size estimation algorithm is used to determine features in a scanned image.

[0173] In some embodiments, software algorithms can be used for size estimation. Backscattered X-rays produce high-contrast, isolated scattering features in the image. Using a known aperture response function, the fruit size can be deconvolved using a software algorithm. The size of the fruit can then be estimated using the deconvolved image.

[0174] Furthermore, the algorithm can be used to reject features that are not compact and isolated. These features may include branches, stems, or leaves. In a further embodiment, the image processing software can be combined with location data to utilize data from scans on both sides of a line to eliminate duplicate counts. Embodiments may also use 3D reconstruction algorithms, utilizing the location and orientation information from the X-ray-irradiated transport vehicle and multi-angle views.

[0175] In other embodiments of the invention, plant health is determined by monitoring the water content in the plant trunk. By varying the energy level of the backscattering source, the plant's water content can be observed beneath the bark in the cambium. Through comparisons between trees, imaging processing algorithms can calculate which tree has less water and which is healthy. Similarly, according to embodiments of the invention, using higher energy and directing the scan towards the root system, backscattering can penetrate less dense soil to identify some root structures. The scan can also determine whether the soil is sufficiently moist for plant growth. The less dense the soil, the deeper the backscattered signal can penetrate for better examination of the root system.

[0176] Furthermore, in other embodiments, the algorithm can be used to track root structure and overall plant health. For plants like strawberries, the root system is close to the ground and is prominent because it contains more water than the soil, which can be similarly mapped onto the branches. In some embodiments, the length and thickness of the roots are measured and compared to historical reference values ​​for plant maturity to determine the plant's health status.

[0177] Other embodiments of the invention can scan soil moisture content. In some embodiments, the algorithm may have historical reference levels of moisture content relative to local neighborhoods of wet and dry soil samples. From images of the soil scan, moisture content signal levels can be monitored to determine whether the ground is sufficiently moist for plants.

[0178] In the precision agriculture industry, determining the health status of crops as they grow is crucial. According to some examples, backscatter scanning can be performed along the developmental process to aid crop health.

[0179] In some embodiments, scanning can be used after bud break. For apple and citrus crops, post-bud break typically occurs when the fruit is only a few millimeters in diameter. At this stage, if the fruit is diseased (too small or irregularly shaped) or clustered, it can be removed. If the fruit is clustered, it can be removed to reduce the number of fruits in the cluster, thereby preventing fruit damage later in growth. In some embodiments, scanning at the post-bud break stage can be used to determine fruit counts and locate clusters.

[0180] Scanning can locate fruit clusters and count newly appearing fruits. In smaller plants, root structures can be scanned to determine proper plant development. Image processing programs can determine if the roots are strong and developed enough for fruit ripening.

[0181] In other embodiments, scanning can be used during the mid-growth stage. Shortly after budding, trees tend to select fruits for their own survival. After this self-selection process, trees typically retain the remaining fruits until maturity. Scanning crops, such as apples or citrus, when the fruits are between 30 and 40 mm in size will provide growers with an accurate yield estimate at maturity. At this scanning stage, growers can accurately know how many fruits were lost during the selection phase.

[0182] In other embodiments, scanning is used to determine crop health. Since fruits are primarily composed of water and have a strong backscattered signal, crop health can be monitored during fruit growth. Furthermore, branch structure can be monitored as another mechanism for determining tree health. The canopy can be assessed by changing the energy from higher energies (e.g., >100 keV) to lower energies (e.g., 50 keV) because, in other embodiments, softer X-rays can result in more signal returning to the leaves. Moreover, in further embodiments, by fusing backscattering with optical imaging and using signal processing, the canopy can be assessed more precisely from water content (scanned with backscattered X-rays) and optical angles to determine color and size.

[0183] In some embodiments, backscatter scanning can also be used to determine crop maturity. Just before maturity (e.g., a few weeks before harvest), backscatter scanning can provide actual crop yield, including calculable quantity and size, so growers can determine the labor required for harvesting (or automate it via backscatter), the amount of packaging material needed, determine market prices, and know profits and losses before harvest.

[0184] As the fruit ripens, its water content increases significantly, reaching 85%-90% of its volume. This excess water allows backscattered signals to provide a more accurate representation of the fruit. With this updated information, size, shape, and count can be determined more precisely, thus providing growers with highly accurate information about crop yield (e.g., crop output).

[0185] This yield information enables growers to accurately plan and determine the labor required for harvesting, the packaging materials needed, set market prices, and profits and losses.

[0186] Embodiments of the invention can also be advantageously used for disease detection. Disease is a major problem for growers because it reduces the overall profitability of crops. Disease can also refer to damage / infection by parasites or insects. By performing backscatter scanning on the crop over multiple seasons, healthy plants will have sufficient moisture in their fruits and branches. The water content of the canopy can also be examined by varying the energy level. In some embodiments, some larvae and insects can be detected, but they need to be large enough and dense enough for backscatter pickup.

[0187] Embodiments of the present invention can also be used to monitor and track plant development and health. This may include, but is not limited to, branch structure and mapping, leaf mapping, root mapping, and trunk structure.

[0188] In some embodiments of the invention, image processing can map the branch growth structure throughout a season and a year by using backscatter scanning. This is a good indicator of plant health because branches typically grow at different rates throughout the season, depending on the plant / tree variety. Understanding year-on-year growth can provide growers with valuable information to determine whether a plant / tree needs replacement, or whether it requires more or less fertilizer and / or water.

[0189] As plants mature during the season, their canopies begin to thin, typically thickening early in the season and potentially reaching full size around mid-season. During this period, in some embodiments, leaves can be mapped using backscatter scanning to ensure normal canopy growth. The plant canopy can be compared to neighboring plants and other plants in the field to identify any patterns that might be caused by plant disease, insects, parasites, watering problems, or fertilizer issues. Backscatter images can be processed for leaf size and water content (signal return amplitude).

[0190] Plant roots are crucial to plant health. In some embodiments, by using high energy (e.g., >140 KeV), a scanner can scan the ground around a plant such as a strawberry plant and obtain images of the near-surface root structure. Roots, rich in water, will stand out. By processing the images and providing root information to the grower, the grower can decide whether to replace the plant early in the growing season so that the plant still has time to bear fruit. Without embodiments of the present invention, growers may have to wait until later in the growing season to determine root health, at which point noticing stunted plant growth may be too late.

[0191] The interior of a plant trunk contains heartwood, which is the dead material from the previous growth of xylem and does not contain water transported within the plant. According to some embodiments, the heartwood and xylem can be distinguished in trunk imaging because of their different water contents and densities. The size and relative intensity of the backscattered signals in the heartwood and xylem can be used as a measure of tree health.

[0192] Embodiments of the present invention can be used to detect surface water and water absorption. In some embodiments of the invention, the X-ray backscatter signal is modulated by the water content in the soil. The backscatter signal may also be attenuated by the soil, making the system unable to detect water content at depths greater than >3-4 inches. According to some embodiments, the depth sensitivity of the X-ray backscatter signal will depend on soil composition and density. The backscatter imaging system can plot the relative changes in soil water content. Soils with low water content will have weak backscatter signals, indicating that the area is dry. An example method for measuring water content is as follows: a reference sample scan is performed on soils with different moisture conditions. The reference scan can be used to construct a correlation or lookup table between the backscatter signal and water content. With the soil composition remaining constant, the reference scan can be used to measure the relative water content.

[0193] Furthermore, according to some embodiments, X-ray backscattering signals can be used as a measure of plant water content. For example, the trunk and branches of a plant can be imaged using a backscattering X-ray system to determine whether the plant is receiving enough water and also to indicate disease conditions that prevent water absorption. Water transport in a tree occurs in the sapwood or xylem of the outer diameter of the trunk. A decrease in relative scattering intensity and xylem size may indicate insufficient water or disease. For small plants like tomatoes, water is transported throughout the stem. For healthier plants, this signal level will be higher than for plants that are not receiving the same amount of water through the trunk.

[0194] In other embodiments, algorithms can be used to track the water content of the trunk and branches. In some embodiments, the algorithm can segment and detect the branch and trunk system of a plant. The algorithm can then examine both sides of the trunk and determine the brightness of the edges to calculate the water content compared to the central portion of the trunk. In some embodiments, the algorithm and scanner can work to track the consistency of the xylem around the trunk relative to the water content. According to some embodiments of the invention, since the xylem cross-section at the center of the trunk image may be thinner than the xylem cross-section at the image edges, where the xylem is deeper due to the curvature of the trunk, the edges may be brighter in the signal return. This will indicate the humidity level of the trunk. By comparing with reference trunk images that are wet and dry, the health of the trunk can be determined.

[0195] Figure 8 A second flowchart 701, according to an embodiment of the present invention, is depicted, which includes steps of a method for guiding robot movement. This is related to... Figure 3 The flowchart illustrates a process similar to the one shown, except that in this case, the method uses a processed scattering signal to guide the robot's gripper to grasp a live plant 710. A penetrating radiation beam is generated (602) and scanned (604) the live plant. Compton scattering from the live plant is detected (606), and the resulting scattering signal is processed (608) to derive characteristics of the live plant, and may be combined with inputs derived from other sensing modalities to guide the robot's gripper to grasp the plant or a portion thereof.

[0196] Figure 11 A table is provided listing exemplary types of crops for which backscatter imaging can be applied. Plant height and width are used to estimate the required imaging conditions. The reference power is the power consumed in the source, i.e., the product of the electron flow (mA) towards the target and the target potential (kV).

[0197] Apple or citrus fruit can grow either as a single fruit or in clusters of two or more fruits within a small area. Clustering often leads to inaccurate counting, and in the worst cases, damages the fruit, preventing it from being sold at maturity. In early stages of using backscatter technology, in some embodiments, growers could precisely locate these clusters and prune or remove them, allowing for the harvest of ripe fruit. In some embodiments, precise location can be achieved by performing two orthogonal scans of the tree in stereo mode using two simultaneous backscatter sources. According to some embodiments, to use two orthogonal sources and obtain correct imaging, the source scans are staggered, so that when one source emits X-rays, the other does not. For example, X-ray source staggering technology is described in, for example, U.S. Patent No. 7400701 (Cason's "Backscatter Inspection Portal"), which is incorporated herein by reference. In some embodiments, each scan line uses a pencil beam source to generate one image line. To generate the scanning pencil beam, a precollimator and a chopper are used in some embodiments. According to some embodiments, each chopper has multiple slits called spokes, and each spoke generates one line of image data. Image processing allows each system to collect temporal and spatial synchronization data for each tree across each source data source. This allows for more accurate fruit location and also allows the system to indicate to the operator, via a user interface and database, where the clusters are located. In some embodiments, the robotic harvesting system can be precisely informed of the cluster locations, and then the robotic harvesting arm can harvest unwanted fruit through a closed-loop control system.

[0198] In some embodiments, for fruits that grow in clusters, such as grapes, the backscattering system can obtain volume information, allowing growers to determine the volume of the crop, rather than just counting individual fruits. This information can help growers understand crop yield, better plan harvesting, packaging, and determine market prices.

[0199] According to some embodiments, the fruit size returned by the backscattered image generated by the scanning system is proportional to the signal strength on the display and the size of the fruit image. Assuming the fruit is symmetrical, its geometry and size can be determined through image processing, and thus its volume. In some embodiments, the software can also track the year-to-year cumulative changes in the location and measurements of a specific plant or crop.

[0200] When the fruit ripens, it contains a large amount of water, allowing it to emit a strong scattered signal to the scanning system. In the image, the scattered signal appears as bright spheres. Branches become darker, and the echoes from the canopy layer are irrelevant signal echoes. Using image processing, the trunk and branches can be segmented, so only the fruit is retained. From this image, the size, volume, and count of the fruit can be collected and stored to report statistics to growers.

[0201] In some embodiments, image processing is required for fruit detection and localization. In some embodiments, the algorithm may receive images of immature fruit to determine fruit health and fruit clustering by removing branches and leaves using various segmentation methods. Algorithms such as cell mapping algorithms can be used to track and segment the fruit. In some embodiments, the algorithm may focus on a bright, round sphere that represents the fruit. The roundness of the size can be monitored, and the size can be determined to ensure that the fruits differ from each other by less than 20%. The algorithm may also record which fruits and the location of clustered fruits. The algorithm may further report any issues, such as fruits exceeding size requirements, deformed fruits, and clustered fruits.

[0202] In some embodiments, the algorithm can also be used for canopy detection. In some embodiments, the canopy detection algorithm uses a two-energy method to pull the canopy out from the rest of the plant. The leaf return level (liquid content), size, and shape will depend on the canopy density. Furthermore, in some embodiments, a visual camera can be used to assist in determining the size, shape, and color of the leaves.

[0203] Embodiments of the present invention can also be used for robot guidance 710 (e.g.) Figure 8 (As shown). This may include initial target mapping in 2D or 3D, closed-loop feedback guidance, and robotic harvesting. Closed-loop feedback guidance can include oscillating beam guidance and image guidance.

[0204] To date, two key challenges in robotic harvesting of specialty crops have been guidance and non-destructive grasping of the target crop. Visual guidance methods are ineffective due to visual clutter caused by plant leaves. Visual targeting is particularly challenging for crops that are the same color as the background foliage. Because specialty crops are sold directly to customers, robotic harvesting must avoid bruising or damaging the fruit. Shaking trees has been used in the past, but this can cause bruising. Optical methods for crop imaging for manipulator control are limited by obstacles from leaves.

[0205] In embodiments of the invention, backscatter imaging is used to improve guidance and manipulator control. First, an initial 2D image of the complete plant can be generated to produce a target map for harvesting. In some embodiments, imaging from multiple views can be used, similar to tomography, to generate a 3D map of the crop. Using the 2D or 3D coordinates of the crop on each plant, the system can plan an effective movement trajectory for harvesting the crop. Next, in some embodiments of the invention, a closed-loop feedback system can be used to generate rapid guidance to the target crop. Finally, a backscatter imaging system can be used to guide the end effector to grasp the fruit for harvesting. Embodiments of the invention using backscatter signals for trajectory planning and final guidance are not distorted by clutter from leaves. Data analysis of backscatter images is simplified compared to the analysis of visual images. Furthermore, embodiments of the invention are unaffected by environmental changes and can operate under daytime or nighttime lighting, variable temperature conditions, fog, or rain. Backscatter signals generated by moisture on leaves can increase the uniform background of scattering from leaves but do not prevent the generation of backscatter images.

[0206] Embodiments of the present invention can be used for initial target mapping, particularly 2D or 3D position detection. Typical crops such as apples, oranges, and strawberries appear as isolated fruits. As a result, they behave as single isolated point emitters of backscattered signals. In this case, according to certain embodiments of the present invention, a 3D map of the crop can be generated before or during harvesting by using multiple backscattered images taken at various angles associated with the plant. As previously mentioned, a 3D map can be generated by mounting two or more scanners on a vehicle with a tracking system enabled. The spatial coordinates of the crop, for individual plants or trees, can then be used to generate a target map for robotic harvesting.

[0207] According to some embodiments, initial image data can be used to avoid tree branches. The vehicle may also include means for robotic manipulation of an end effector suitable for harvesting crops. Customized end effectors can be tailored to the shape and size of each crop to be harvested. Optionally, embodiments of the invention may include two or more backscatter imaging systems mounted at different angles relative to the crop. In some embodiments of the invention, the backscatter imaging systems may be positioned at different angles, with at least 45 degrees, preferably 90 degrees, between views to generate a 3D perspective view of the crop.

[0208] Embodiments of the invention may also provide closed-loop feedback guidance, including image guidance. In some embodiments, after generating target coordinates, the system may use a closed-loop feedback control system employing one of two methods to position the manipulator near the fruit. One method employed in some embodiments of the invention can utilize rapid 2D or 3D imaging of the crop. The robotic harvesting system can be used to create a rapid, localized backscattered image of the plant from one of multiple views.

[0209] Embodiments of the invention can also utilize oscillating beam guidance to move the robotic manipulator near a target using 3D positional information. In some embodiments, it can be guided within a closed-loop feedback system as the arm approaches the target. The time required for image generation and processing is critical for rapid and accurate final guidance to a specific selected target. The robotic arm can be equipped with an X-ray source with pencil-beam collimation or fan-beam collimation.

[0210] In some embodiments of the invention, the output beam strikes a target and generates a backscattered signal. By changing the collimation position or angle, the position of the beam can be oscillated during the motion shown to generate a time-varying backscattered signal. This position can vary in one or more dimensions. The signal amplitude generated by the oscillating input beam is related to the beam's position on the target.

[0211] In some embodiments, in order to maintain optimal proximity to the target, when the transport vehicle 108 ( Figure 2 As the robot approaches the target (as shown), the backscattered signal amplitude is actively maintained at its maximum value by serving the position and orientation of the transport vehicle 108. In some embodiments, by maintaining the signal at its maximum amplitude, the robot manipulator will approach the target in a straight line. Image-based and static pencil beam data can be used in combination based on the robot's movement speed and proximity to the target. According to some embodiments of the invention, as the picking arm moves closer to the fruit to be picked, it can continuously update its position and provide the picking arm with new control data regarding the position of the picking arm and the fruit.

[0212] In some embodiments, backscattered images can be used to control the gripper's movement when the arm is very close to the fruit (almost touching it). Based on the fruit's shape, size, and orientation in the backscattered data, it can open the picking mechanism to surround the fruit. Once the fruit is picked, the robot can gently place it into a container to avoid damaging it. Sensing mechanisms such as sound or IR range sensors can tell the robot how deep the fruit is in the container so that it can be gently placed in without falling and bumping into other fruits.

[0213] In some embodiments, utilizing 3D coordinate mapping, the system can be adapted to selectively harvest only the side of the plant closest to the robot, preventing plant damage caused by the manipulator passing through the plant. This is based on determining the position of the central stem of the plant and harvesting only those plants whose distance relative to the robot is shorter than the distance to the central stem at each specific height. Using 3D coordinate mapping, plants can be harvested continuously without interruption. The system can advantageously operate continuously, scanning 24 / 7, requiring minimal operator intervention or supervision. Cameras can advantageously be mounted on the robot to allow the operator to monitor progress and intervene if problems arise.

[0214] Figure 9 A fan-beam scanning apparatus, typically denoted by numeral 901, is illustrated for accelerating yield estimation and other crop characterization. In some embodiments of the invention, the speed of backscatter imaging may be limited by the flux of the pencil-beam collimated emission. In some embodiments, to increase the flux to the irradiated object, the backscatter imaging system may advantageously use fan-beam irradiation. In some embodiments, the scanning apparatus 901 may estimate the total volume of crop 112 in the field. In some embodiments, the irradiating fan beam 910 is advantageously positioned vertically and scans the object being examined. The total amount of X-ray backscattering along the irradiation line (which is the cross-section of the fan beam) is immediately detected and generates a scattered signal. According to some embodiments, the data obtained from the scan may be a 1D curve of the total backscattered position versus signal along the path of the transport vehicle 108.

[0215] Figure 10 Curve 1001 is shown, representing the quantification of fruit using volume information from the scanning process. This is achieved by integrating the area 1005 under curve 1001 and multiplying it by an adjustable threshold 1010 used for detecting fruit 112 (e.g., ...). Figure 2 By comparing the backscattered signal (as shown), volume information can be inferred from the 1D curve 1001 of the location. According to some embodiments, the volume derived from area 1005 can be correlated with the total mass of the fruit. In some embodiments, an adjustable threshold 1010 can advantageously reject signals from objects such as the trunk, canopy, and branches.

[0216] In embodiments, this specification provides a method for quantitatively measuring the weight of a portion of a crop, such as hanging fruit, located on at least one plant, branch, and / or vine. While the embodiments described herein refer to hanging fruit, it should be noted that this specification is not limited to such embodiments, and the methods provided herein can be used to weigh any crop. In embodiments, a backscattered X-ray scanning system, such as those described above, can be used to scan a plant or any part thereof, such as a branch and / or vine, to obtain scanned images, which can be analyzed to determine the weight of fruit hanging in a field of plant, branch, and / or vine. In various embodiments, the backscattered signal generated from the hanging fruit is proportional to the mass of the fruit and the distance of the fruit from the scanning system. The backscattered signal generated by the fruit irradiated by incident X-rays is approximately proportional to the square of the distance of the fruit from the scanning system. In embodiments, the intensity of the backscattered signal is integrated over the imaged fruit as a measure of the total mass of the fruit. In embodiments, the weight of fruit can be estimated using the methods of this specification, for example, but not limited to, grapes, berries, citrus fruits, apples, melons, and tomatoes. In some embodiments, the methods described herein can be used to determine the weight of any trellis fruit or vine fruit, which may be scanned from both sides. In others embodiments, the methods described herein can be used to determine the weight of any non-trellis fruit or vine fruit, which may be scanned from both sides.

[0217] The following description interchangeably describes methods for collecting crop data and image processing tasks for analyzing the data, the image processing tasks being performed by a controller coupled to one or more X-ray scanners for collecting crop data by irradiating the crop with X-rays from multiple directions and obtaining scanned images.

[0218] In various embodiments, the controller is a computing device including an input / output unit, at least one communication interface, and system memory. The system memory includes at least one random access memory (RAM) and at least one read-only memory (ROM). These components communicate with a central processing unit (CPU) to enable the operation of the controller. In various embodiments, the controller may be a conventional standalone computer, or alternatively, the controller's functionality may be distributed across a network of multiple computer systems and architectures.

[0219] In some embodiments, the execution of a sequence of multiple program instructions for quantitatively measuring the weight of a portion of a crop enables or causes the controller's CPU to perform various functions and processes, such as performing tomographic image reconstruction for display on a screen. In alternative embodiments, hardwired circuitry may be used in place of or in combination with software instructions for implementing the systems and methods described herein. Therefore, the described systems and methods are not limited to any particular combination of hardware and software.

[0220] In one embodiment, this specification provides a method for performing dual-view data acquisition by simultaneously scanning plants, branches, vines, and / or fruits from two different sides of a crop using two X-ray scanners. In another embodiment, dual-view data acquisition can be performed by scanning fruits on two different sides using a single X-ray scanner operated to acquire multiple scans. More specifically, the X-ray scanner is positioned outside the resulting area of ​​the field (the area of ​​the crop where the fruit is growing or is expected to grow), on opposite sides of a row of plants. In one embodiment, scan data from each viewpoint (X-ray scanner location) is registered using a position sensor or an auxiliary camera view. Since the backscattered signal generated by the hanging fruit is proportional to the mass of the fruit and the distance between the fruit and the scanning system, knowing the position of the X-ray scanner at any given time is useful. The orientation and position of the scanner can be recorded to preserve the relative irradiation angle and position of the X-ray source relative to the fruit. In one embodiment, once scanned images of the fruit have been collected, the images can be analyzed using a distance normalization process at the pixel or feature level. In one embodiment, identical pixels from two images can be compared and used to correct for variations in distance. In another embodiment, when performing feature-level distance correction, the scanned fruit can be combined with intensity before distance correction is performed.

[0221] In various embodiments, scanned fruit data is analyzed by using image segmentation to estimate weight and predict fruit yield in the field. Figure 12A This is a flowchart illustrating a method for estimating crop weight and predicting crop yield according to an embodiment of this specification. In step 1202, the crop in a predetermined area / block is irradiated with X-rays, and at least one row of image data of the plant is collected, with all images flipped in a predetermined direction. In this embodiment, the crop is a fruit crop, including vines bearing fruit growing on the vines, such as grapes. The following figures will describe the process further. Figure 12A In this embodiment, data such as, but not limited to, image data, GPS coordinate data, scan speed data, and / or tilt / inclinometer data are stored in a raw data file.

[0222] Figure 12BThis is a flowchart illustrating the steps of collecting image data of a row of plants according to an embodiment of this specification. In step 1242, image data is extracted from the raw data file. In step 1244, GPS coordinates of each scanned block, including the distance between rows of plants within the block, are extracted from the raw data file. In step 1246, a schematic diagram of each block is created. In step 1248, the movement or transport of a vehicle (e.g., but not limited to a tractor) carrying scanning equipment for irradiating the crop with X-rays is plotted on the schematic diagram. In step 1250, the GPS coordinates of a point having a timestamp closest to the image timestamp obtained after distance correction from the raw image data are determined. More specifically, the GPS coordinates of a point on the schematic map are determined by relating at least one timestamp of one or more GPS coordinates to at least one timestamp at the time the point was captured on the schematic map.

[0223] In one embodiment, once scanned images of the fruit are collected, the images can be analyzed using a pixel-level or feature-level distance normalization process. In another embodiment, identical pixels from two images can be compared and used to correct for variations in distance. In yet another embodiment, when performing feature-level distance correction, the scanned fruit can be combined with intensity before distance correction. In another embodiment, feature-level distance correction allows for changes in registration and magnification to match individual features of the image; while pixel-level distance correction does not allow for changes in perspective, registration, or magnification of the image. In another embodiment, to perform feature-level distance correction, predefined features are segmented from the image, and the segmented features are matched with respect to the image's driving direction, registration, and magnification, resulting in pixel-by-pixel distance correction of the image.

[0224] In step 1252, the plant / vine rows are positioned along a corresponding direction based on the vehicle's direction of motion. In step 1254, the image of the positioned plant / twig / vine rows is flipped in a predetermined direction to obtain the same / consistent sequence of plants / twigs / vines in each row of each image. In step 1256, each image is normalized using a predefined normalization bar. In an embodiment, a sheet of HDPE plastic (plastic component) is positioned at the top of the scanner's field of view, such that the X-ray scanning beam strikes the plastic sheet and generates a signal for calibration / normalization for each image. In an embodiment, the HDPE plastic component can be a contrast block, such as... Figures 21A-21B As shown and described below. Figure 12C Curves of image data before and after normalization according to embodiments of this specification are shown. Curves 1260 and 1262 show measured intensity data of the scanned image corresponding to each pixel row of the image before and after normalization, wherein the normalization is performed to correct for any variations that may occur relative to the source intensity or detector response of the scanner.

[0225] Return to reference Figure 12A In step 1204, contrast enhancement and denoising are performed relative to each collected scanned image to achieve better registration and segmentation. In embodiments, the image is processed to enhance contrast using any known contrast enhancement technique. In one embodiment, image contrast enhancement is performed using one or more known image processing techniques, such as, but not limited to, histogram equalization, where the contrast of the image is enhanced by transforming values ​​in the intensity image such that the histogram of the output image approximately matches a specified histogram. In another embodiment, a contrast-limited adaptive histogram equalization technique may be employed, where the contrast of small data regions (blocks) of the image is enhanced such that the histogram of each output region approximately matches a specified histogram. Contrast enhancement may be limited to avoid amplifying noise that may be present in the image. In one embodiment, the specified histogram is the final data histogram of all pixels in the complete image.

[0226] Figure 13A Scanned images before and after contrast enhancement according to an embodiment of this specification are shown. When scanned image 1302 is processed using a contrast enhancement algorithm, a contrast-enhanced image 1304 is obtained. As can be seen from the decomposed image 1306 of a portion 1308 of image 1304, noise in the image also increases with the increase in contrast level. In this embodiment, noise in the contrast-enhanced image is eliminated by removing all image pixels located below a predetermined threshold. In this embodiment, all image pixels having fewer than three neighboring pixels greater than the threshold are also removed. In this embodiment, the denoising threshold is selected to include relevant features in the image, such as, but not limited to, leaves, clusters, and warning lines.

[0227] Figure 13B The images shown are contrast-enhanced images before and after denoising, according to an embodiment of this specification. Image 1310 is a contrast-enhanced image, which is denoised to obtain image 1312. Figure 13B The image 1314 before contrast enhancement and denoising is also shown. After applying contrast enhancement technology to image 1314, a contrast-enhanced image 1316 is obtained, which produces image 1318 during denoising.

[0228] Return to reference Figure 12A In step 1206, global registration of all collected scanned images is performed. In an embodiment, global registration of each image is performed by selecting a predefined portion from a first image and finding the best correlation of that portion in a second image. In an embodiment, the best correlation is the location that provides the desired registration between the two images and can be obtained using techniques such as, but not limited to, least-squares fitting. In an embodiment, horizontal scaling of the images is corrected during global registration. Figure 14AAn image that has been globally registered is shown. In this embodiment, the left portion of image 1402 is selected and associated with a second image obtained by flipping image 1402. Points or regions 1404 represent points or regions where the two images have the best or optimal overlap. This figure illustrates the registration quality of the two images when horizontal and vertical shifts are applied.

[0229] In some embodiments, the global registration 1106 step is not performed. Figure 14B An image 1410 illustrating the result of global registration according to an embodiment of this specification is shown. In some embodiments, global registration provides the desired result in the central region 1412 of image 1410; however, the edges / sides 1414 of the image show an accumulation of offset data. Therefore, in some embodiments, a local registration process is performed. In embodiments, global registration is performed simultaneously on the entire row of images (including multiple plants, branches, and / or vines); while after segmentation, local registration is performed one at a time for each plant, branch, or vine. In embodiments, global registration is performed simultaneously on the entire row of images (including multiple vines); while local registration is performed one at a time on each vine after segmentation. Therefore, local registration is more accurate because it allows for speed variations, such as... Figure 14B As shown.

[0230] Return to reference Figure 12AIn step 1208, individual plants / branches / vines are cut from the collected scanned images, and the edges of the plants / branches / vines are discarded to obtain a segmented image. In embodiments, such as in viticulture, segmentation is performed when data on real vines on the ground are collected individually. In embodiments, individual plants / vines / branches from at least one row of plants are separated using features such as stems or trunks. For example, vines can be identified by the trunk of each plant. In embodiments, false positives can be eliminated by enforcing a minimum distance between vines. In embodiments, known methods for plant / branchine / vine segmentation, such as, but not limited to, identifying vertical columns of pixels and applying various available filters to them, do not produce the desired results. This is due to a variety of reasons, such as: poles and stumps in the image and long overhangs are identified as trunks; tilted trunks in the image are not identified; missing vines in the image are not identified every time; and there are large variations in ground topology and vertical magnification in the image. In various embodiments, manual identification, extensive manual inspection, and correction of the starting points of plants / branches / vines in an image produce the desired results when the distance between plants (e.g., but not limited to vines) is greater than or less than a predetermined value. In embodiments, the benchmark for the desired result is the actual distance between plants / branches / vines, which is known from when the crop was planted (in the example, when the vines were planted). Therefore, the measured or determined distance between plants / branches / vines cannot be less than the actual distance at the time of crop planting. Thus, in the case of grape cultivation, the measured or determined distance between vines cannot be less than the actual distance at the time of vine planting. Figure 15A A successful vine segmentation image 1510 according to an embodiment of this specification is shown. Figure 15B A segmented image 1520 of a vine plant showing a false positive is shown according to an embodiment of this specification.

[0231] Return to reference Figure 12A In step 1210, local registration of all segmented images is performed (and in this example, local registration of the vine segmented images). In this embodiment, local registration of the segmented images is performed to obtain precise alignment between pairs of images showing different plant views. This is achieved by using images showing relative plant views for distance calibration, subtracting the warning line from the corresponding image previously taken in the pre-flowering season, and using images showing plant views from earlier in the season to aid in clustering segmentation. The subtraction removes any strong background caused by the warning line from the registered images. In various embodiments, a predefined registration algorithm (including steps for image shifting and magnification, or one or more) is used for the local registration of the segmented images to obtain distortion-free results.

[0232] Figure 16AThe images shown are segmented images (e.g., vine segmentation images) after local registration has been applied according to embodiments of this specification. Image 1602 shows local registration applied to pre-harvest and post-harvest images. Image 1604 shows local registration applied to pre-flowering and post-harvest images. Image 1606 shows local registration applied to fruit set and post-harvest images.

[0233] In this embodiment, instead of registering paired images, all images are modified to be consistent in order to create a consistent vertical scaling for performing data analysis, thereby improving the efficiency of the image registration process. In this embodiment, a "coarse annotation" method is used, which involves manually annotating warning lines, trunks, ground boundaries, and water pipes, and may include straightening warning lines, horizontally aligning the image using the trunk, vertically aligning the image using the straightened warning lines, and vertically scaling the image by keeping the distance between the warning lines and the ground constant. Figure 16B Coarse feature annotations are shown on segmented images (e.g., vine segmented images) according to embodiments of this specification. Image 1610 shows a vine segmented image with the trunk identified before the application of coarse feature annotations. Image 1612 shows the same vine segmented image after the application of coarse feature annotations. Image 1614 shows the same vine segmented image registered using coarse feature annotations, where the height of the warning line has been identified. Figure 16C The illustration shows the registration of vine segmentation images using coarse feature annotations according to embodiments of this specification. Images 1616 and 1618 show the regularization of vines that flatten warning lines. Figure 16D The images show close-up views of the sides of a vine segmentation image with coarse feature annotations according to embodiments of this specification. Images 1620 and 1622 show close-up views of vine clusters from a first direction / side and a second opposite direction / side, respectively.

[0234] Return to reference Figure 12A In step 1212, a clustering processing technique for processing each segmented image is determined. In step 1214, a clustering segmentation function is performed on a predefined set of registered segmented images. In an embodiment, a classical clustering segmentation function is performed on a predefined set of registered segmented images. In an alternative embodiment, a deep learning clustering segmentation function is performed on a predefined set of registered segmented images. In an embodiment, a classical clustering segmentation algorithm based on a set of predefined rules is used to process the predefined set of registered segmented images. In an embodiment, the segmented images are clustered to enable cluster-based data analysis, and a neural network is also trained to identify clusters during data analysis. In an embodiment, at least one plant cluster within a predetermined distance is identified and annotated. In an embodiment, at least one vine cluster within a predetermined distance is identified and annotated.

[0235] In an embodiment, when using a deep learning clustering segmentation function, identification and annotation are performed by a deep learning algorithm trained to identify desired features in manually selected clusters. Figure 17A A clustering annotation image according to an embodiment of this specification is shown. Image 1710 is processed using a clustering piecewise function to produce a clustering annotation image 1720. The different shading shown in image 1720 represents various manual annotations.

[0236] In one embodiment, a “coarse segmentation analysis” method is used to process segmented images (e.g., vine segmentation images), in which a coarse segmentation network is used to perform coarse segmentation analysis of the image. Figure 17B A coarse segmented network according to an embodiment of this specification is illustrated. Image 1730 is an input scanned image of vines in a field, while image 1732 depicts a desired output image including predefined multiple classes of segments. In embodiments, examples of segment classes include posts, warning grounding wires, pipes, and background. Image 1734 depicts the segment classes applied to image 1730. Figure 17C A graph is shown representing the training loss for each epoch in the coarse segmentation network when training is performed on 50 pre-harvest vine images. Figure 17D A graph is shown representing the validation loss in each epoch of the coarse segmentation network when training is performed on 50 pre-harvest vine images with 5 validation sets. In various embodiments, coarse segmentation identifies key features of the fruit vine field, such as, but not limited to, the ground, trunk, warning lines, and / or water pipes. Figure 17C and 17D The diagrams depicted in this specification demonstrate that the method described herein allows for the identification of these features.

[0237] Refer back Figure 12A In step 1218, distance calibration is used to process the clustered images. In step 1220, crop weight estimates and yield predictions are determined. In this embodiment, the intensity of individual pixels in the clustered image is measured from scans of crop features from both sides. Figure 17E This is a block diagram illustrating distance calibration applied to images obtained by scanning an object from two opposite sides, according to an embodiment of this specification. As shown, a first X-ray system 1750 scans crop cluster 1752 from a first side, while a second X-ray system 1754 scans crop cluster 1752 from a second opposite side. The scanning systems 1750 and 1754 are at the same distance from the center of crop cluster 1752, and... Figure 17EThe distance from the first scanning system 1750 to the first side 1756 of the cluster 1752 is marked as "L" in the figure. After measuring the intensity of each pixel in the clustered image by scanning the cluster 1752 from both sides, a correction is applied based on the ratio of the intensity I1 measured from the first side to the intensity I2 measured from the second side. In the embodiment:

[0238] I1 = I0 / (d / L) 2 (1)

[0239] I2=I0 / ((2L–d) / L) 2 (2)

[0240]

[0241] In this embodiment, when I1 > I2, the distance correction factor 'C' is determined as:

[0242]

[0243] Wherein, “R” represents the strength ratio and is equal to I1 / I2.

[0244] Figure 17F The curve of the correction factor C versus the intensity ratio according to an embodiment of this specification is shown. From Figure 17F As can be seen, the curve of correction factor C, 1758, decreases as the intensity ratio increases.

[0245] In one embodiment, the weight of fruit hanging on vines in the field is estimated by determining changes in the X-ray signal backscattered by the fruit over a predetermined time period, such as the pre-flowering season (i.e., from late April) to the harvest season (i.e., late October) for any given crop in any given year. Time-based crop image comparison enables crop yield prediction. Accurately measuring the mass of fruit / vegetables growing on plants / branches / vines (often referred to as hanging weight) in the early season (before harvest) can predict harvest yield. In various embodiments, yield estimation is performed using measurements of early-season hanging weight and growth rate.

[0246] Figure 18A These are photographs of fruits from the pre-flowering season (i.e., late April) to the harvest season (i.e., late October) according to embodiments of this specification, along with corresponding X-ray backscattered images of the fruits. Photograph 1802 shows a pre-flowering view of a fruit cluster taken in late April; Photograph 1804 shows a fruit cluster in late May; and Photograph 1806 shows a fruit cluster four weeks later; Photograph 1808 shows a fruit cluster in late July; and Photograph 1810 shows a fruit cluster near harvest time in late October. The corresponding backscattered X-ray scan images are shown in… Figure 18B In the images, image 1812 corresponds to the fruit cluster shown in photograph 1810; image 1814 corresponds to the fruit cluster shown in photograph 1808; image 1816 corresponds to the fruit cluster shown in photograph 1806; image 1818 corresponds to the fruit cluster shown in photograph 1804; and image 1820 corresponds to the fruit cluster shown in photograph 1802. In this embodiment, the exemplary correlation between images and corresponding fruit clusters ranges from 80% between scanned image 1812 and the fruit clusters shown in photograph 1810 to 64% between scanned image 1818 and the fruit clusters shown in photograph 1804.

[0247] Figure 18C This is a table, according to embodiments of this specification, that tracks the weight of individual vines and the correlation of corresponding scanned images over a predetermined time period. Table 1830 includes column 1832 for recording the date on which the weight and scanned image of the fruit vine / cluster were obtained; column 1834 for recording the harvest stage of the fruit vine / cluster; column 1836 for recording the corresponding average weight of the fruit vine / cluster; and column 1838 for recording the correlation with the corresponding scanned image of the fruit vine / cluster. It can be observed from Table 1830 that the weight of the fruit vine / cluster increases from May to July, and then remains almost unchanged until the November harvest. Furthermore, the correlation between the corresponding scanned images and the fruit vine / cluster is maintained between the final harvest in May and November. Figure 18D The diagram shows a curve of vine weight versus scan data corresponding to multiple fruit vines over a period from May to October, according to an embodiment of this specification. Figure 18E The embodiments according to this specification are shown. Figure 18D The curve shown is a normalized curve of vine weight against scan data.

[0248] Those skilled in the art will understand that, in some embodiments, data corresponding to X-ray scan images of crops, such as but not limited to precipitation, time of year, temperature, geographic coordinates, wind speed, and sunshine, are stored in a database coupled to a controller and used, through the application of processing techniques such as (e.g.) artificial intelligence and big data analytics, to determine the weight of hanging fruits and / or to make continuous predictions of crop yield at different times / months of the year.

[0249] In this embodiment, the backscattered signal increases with the increase of the cluster size of the scanned fruits. Figure 19A This is a graph illustrating the relationship between scattering intensity and cluster size according to embodiments of this specification. As shown in image 1910, curve 1912 depicts the increase in scattering intensity plotted on the y-axis 1914 as the thickness of the scanned fruit cluster plotted on the x-axis 1916 increases.

[0250] In this embodiment, the backscattered signal decreases as the distance between the X-ray scanner and the scanned fruit cluster increases. Figure 19B This is a graph illustrating the relationship between the backscattered signal and the distance between the scanner and the fruit according to embodiments of this specification. As seen in image 1920, curve 1922 depicts the decrease in backscattered signal intensity plotted on the y-axis 1924 and on the x-axis 1926 as the distance between the X-ray scanner and the scanned fruit cluster increases. It can be seen in image 1920 that when scanning the fruit cluster at the furthest distance (approximately 3 feet from the scanner), the backscattered signal intensity drops to about 10% of its maximum intensity. It is observed that the distance from the scanner and the size of the scanned fruit cluster add noise to the backscattered signal. In various embodiments, it is assumed that the fruit cluster size is randomly distributed in the scanned area, and the average cluster size can eliminate the noise in the backscattered signal caused by the increased distance between the scanner and the cluster. In various embodiments, the harvest yield data corresponding to the crop cluster is based on the total intensity of the backscattered signal from that cluster. In this embodiment, the harvest yield data plotted relative to the backscattering intensity varies with the crop because the composition of fruit clusters varies with the crop, and the backscattering signal intensity is different from that of other substances in the fruit when X-rays are backscattered by water.

[0251] In this embodiment, because the backscattered signal intensity decreases with increasing distance between the scanner and the fruit, and the signal intensity is not strongly correlated with fruit weight, alternative image features with improved correlation to fruit weight are used on a predetermined number of real ground vines. In this embodiment, a row of vines comprises approximately 120 vines. The real ground vines are those monitored over a predetermined time period, and these vines are imaged before and after harvesting and weighing. In this embodiment, a region analysis method is used to provide feature size, digital shape, and orientation. Figure 20A A region analysis method according to an embodiment of this specification is illustrated. In such... Figure 20A In the illustrated embodiment, image 2010 was acquired before the actual ground harvest; image 2012 was acquired after the actual ground harvest; image 2014 was obtained by performing image normalization on images 2012 and 2010, and then subtracting normalized image 2012 from normalized image 2010; image 2016 depicts threshold features, while image 2018 depicts the results of region of interest analysis on features such as, but not limited to, area, perimeter, orientation, intensity, and quantity.

[0252] Figure 20BThe diagram illustrates the correlation between ground fact cluster weight plotted on the y-axis 2020 and X-ray backscattering intensity plotted on the x-axis 2021 at different values ​​of the applied correction factor, according to embodiments of this specification. It can be seen that curve 2022 exhibits the best correlation at a correction factor "R" = 0.8048. Curve 2204 exhibits the worst correlation at a correction factor "R" = 0.5668, while curve 2206 exhibits a moderate correlation at a correction factor "R" = 0.4353. In various embodiments, variations in the processing steps involved produce less desirable correlation results; these steps include, but are not limited to, pre-flowering image subtraction, intensity correction corresponding to a given magnification factor, filling the warning line region from below and above using simple interpolation, multiplying pixel rows by a factor based on the distance from the warning line, and raising the intensity of each pixel to a power.

[0253] Figure 21A A scanning system for scanning crops to predict the weight of hanging fruit, according to an embodiment of this specification, is illustrated. In some embodiments, it has been observed that when using, such as... Figure 21A When the scanning system 2102 is shown, the signal strength changes by approximately 20%. Because the performance of the X-ray scanning system varies with operating temperature, in this embodiment, a contrast block is inserted in region 2104 of the scanning system 2102, since the contrast block can correct the X-ray beam output intensity and detector sensitivity as the X-ray scanning system temperature changes. Figure 21B The embodiments of this specification are shown in Figure 21A The contrast block used in the scanning system. In an embodiment, the contrast block 2106 is a plastic (HDPE) block that is inserted into the field of view of the scanning system 2102. Figure 21C A scanned image obtained by using a contrast block according to an embodiment of this specification is shown. Figure 21D A graph showing the intensity variation of a contrast block region across the field of view relative to different times of day, according to an embodiment of this specification, is illustrated.

[0254] In various embodiments, this specification provides a method for accurately assessing the quality of fruits / vegetables growing on plants / branches / vines, which is commonly referred to as hanging weight. Figure 22 This is a flowchart illustrating a method for determining the hanging weight of fruits / vegetables on a plant / branch / vine according to an embodiment of this specification.

[0255] In step 2202, at least two X-ray scan images of the plant / branch / vine, including clusters of fruits / vegetables, are obtained. In this embodiment, the images are obtained from at least two opposing sides / directions, such as... Figure 17EAs shown. In one embodiment, this specification provides a method for performing dual-view data acquisition by simultaneously scanning a cluster using two X-ray scanners. In another embodiment, dual-view data acquisition can be performed by scanning the cluster using a single X-ray scanner with multiple acquisitions.

[0256] In an embodiment, the two acquired scanned images are offset from each other by an angle of 90 to 270 degrees. This can be achieved by capturing a first image from a first side or plane and a second image from a second side or plane. In an embodiment, the preferred offset between the images is 180 degrees, such that the sides of the captured images are substantially parallel to each other.

[0257] In step 2204, the two images are registered, which includes combining / matching the images. In an embodiment, the two images are registered by flipping and translating one image relative to the other, as referenced above. Figure 12A The image registration method described in this specification is performed under the assumption that the two obtained images are parallel to each other. In embodiments where the obtained scanned images corresponding to both sides of a cluster are not parallel to each other, various different registration methods can be used to correctly match the images.

[0258] In step 2206, the matched image is corrected for one or more predefined parameters, such as contrast or scale. Conventional backscatter X-ray scanning applications are used to detect hidden threat elements, such as, but not limited to, firearms and other weapons. In this case, the scanned image needs to be sharp enough to approximate the shape of the weapon. However, in this specification, the scanned image needs to be sharper than in conventional applications because the accurate calculation of the clustering's suspension weight depends on the image's sharpness. Therefore, the registered image is corrected for at least contrast, brightness, intensity, and scale. In an embodiment, feature level correction and distance correction are performed on the matched image, as referenced... Figure 12A and 12B As stated above.

[0259] In step 2208, the weight of the fruit / vegetable clusters captured in the corrected image is calculated by identifying the clusters in the image and analyzing the cluster strength pixel by pixel. The analyzed strengths are summed over a predefined time period and correlated to obtain the weight. In an embodiment, as shown in the reference... Figure 12A , 18AThe -18D description describes an estimation of the weight of fruit / vegetable clusters by determining changes in the X-ray signal backscattered by the clusters over a predetermined time period, such as from the pre-flowering season in late April to the harvest season in late October of any given year. Because fruits / vegetables may overlap, making volume determination difficult, volume estimation of imaged clusters often leads to inaccurate results. Therefore, according to the method of this specification, the mass / weight of the imaged fruit / vegetable clusters is determined using the pixel intensity of the image.

[0260] The examples above are merely illustrative of many applications of the systems and methods described herein. Although only a few embodiments of this specification have been described herein, it should be understood that this specification may be embodied in many other specific forms without departing from the spirit or scope of this specification. Therefore, these examples and embodiments are to be considered illustrative rather than restrictive, and the specification may be modified within the scope of the appended claims.

Claims

1. A method for estimating crop weight, wherein, The crop comprises at least one row of plants, and wherein the at least one row of plants comprises at least one fruiting vine and / or branch, the method comprising: Irradiate a predetermined area of ​​the crop with X-rays from at least both sides; Obtain scanned images of at least one row of plants; Perform contrast enhancement and noise reduction on each collected scanned image; Perform global registration on all contrast-enhanced denoised images; A segmented image representing individual vines and / or branches is obtained by separating the vines and / or branches from the image; Perform local registration on the obtained segmented image; Perform a clustering piecewise function on each segmented image; Segmented images obtained using distance calibration processing; and The weight of the crop is estimated by using distance-calibrated images.

2. The method according to claim 1, wherein, The step of collecting scanned image data of at least one row of plants includes flipping the scanned image produced by irradiating a predetermined area of ​​the crop with X-rays in the same predetermined direction.

3. The method according to claim 1, wherein, Separating vines and / or branches from the image includes cutting the separated vines and / or branches from the image by discarding the edges of at least one resulting vine and / or branch from the at least one row of plants.

4. The method according to claim 1, wherein, Performing local registration involves aligning image pairs that show different views of the same vine and / or branch.

5. The method of claim 1, further comprising using the distance-calibrated image to predict crop yield.

6. The method according to claim 1, wherein, Obtaining scanned images of at least one row of plants includes: Extract image data from the original data file; Create a schematic diagram for each predefined region; The schematic diagram depicts the movement of a vehicle carrying scanning equipment used to irradiate crops with X-rays; and The GPS coordinates of a point on a schematic diagram are determined by relating at least one timestamp of one or more GPS coordinates to at least one timestamp when the point was captured.

7. The method of claim 6 further comprises positioning the plant row along a corresponding direction based on the direction of movement of the vehicle, and flipping the positioned plant row in a predefined direction to obtain a consistent plant sequence in a row in each acquired scan image.

8. The method of claim 6 further comprises normalizing each acquired scan image by using a predefined normalization bar.

9. The method of claim 6, further comprising scanning the GPS coordinates of each predetermined area to obtain the distance between plant rows in the area.

10. The method of claim 1, further comprising identifying and annotating the segmented image.

11. The method of claim 1, further comprising determining a clustering processing technique for processing each segmented image.

12. The method of claim 1, further comprising processing the segmented image using a coarse clustering segmentation method.

13. The method according to claim 1, wherein, The clustering segmentation function is either a classic clustering segmentation function or a deep learning clustering segmentation function.

14. The method according to claim 1, wherein, The estimated weight of fruit hanging on a plant is determined by measuring the changes in the X-ray signal backscattered by the fruit over a predetermined time period.

15. The method according to claim 14, wherein, The fruit includes one of grapes, berries, citrus fruits, apples, melons, and tomatoes.

16. The method of claim 14, wherein, The X-ray signal backscattered from the fruit is proportional to the fruit's mass and the distance between the fruit and the scanning system that generates the X-rays used to irradiate the fruit.

17. The method according to claim 16, wherein, The X-ray signal backscattered from the fruit is proportional to the square of the distance between the fruit and the scanning system.

18. The method according to claim 16, wherein, The total mass of the fruit is determined by integrating the signal intensity of the X-ray signal backscattered from the fruit through the crop.

19. The method of claim 14 further comprises performing dual-view data acquisition by simultaneously scanning the fruit with two X-ray scanners.

20. The method of claim 14, further comprising performing dual-view data acquisition by scanning the fruit using a single X-ray scanner with multiple acquisitions.

21. The method according to claim 14, wherein, The X-ray scanner is located outside the fruiting area of ​​the field, on the opposite side of a row of fruit plants.

22. The method of claim 14 further comprises collecting images of the fruit and analyzing the images using a distance normalization process at the pixel or feature level.

23. A method for determining crop quality using at least one X-ray scanner, the method comprising: At least two scanned images of the crop are obtained, wherein the first of the at least two scanned images is obtained along a first plane relative to the crop, and the second of the at least two scanned images is obtained along a second plane relative to the crop, wherein the first plane is angularly shifted relative to the second plane; Register the first scan image and the second scan image; The first and second scan images were corrected and registered; and The quality of the crop is determined from the corrected first and second scan images.

24. The method according to claim 23, wherein, The first plane is angularly shifted relative to the second plane by an angle between 90 degrees and 270 degrees.

25. The method according to claim 23, wherein, The first plane and the second plane are parallel to each other.

26. The method according to claim 23, wherein, Registering the first scan image and the second scan image includes matching the first scan image and the second scan image by flipping and translating the other relative to at least one of the first scan image and the second scan image.

27. The method according to claim 23, wherein, Obtaining at least two scanned images of the crop involves scanning the crop simultaneously using two X-ray scanners.

28. The method according to claim 23, wherein, Obtaining at least two scanned images of the crop involves scanning the crop using a single X-ray scanner and performing multiple scans.

29. The method according to claim 23, wherein, The first and second scan images for registration correction include correcting the scan images for a number of predefined parameters.

30. The method according to claim 23, wherein, The first and second scan images for calibration registration include calibrating the scan images for one or more of contrast, brightness, intensity, or scale.

31. The method according to claim 23, wherein, Determining crop quality from the corrected first and second scan images involves identifying one or more fruit clusters in the scan images and analyzing the strength of the clusters pixel by pixel.

32. The method of claim 31 further includes summing and correlating the analysis intensities of clusters over a predetermined time period.

33. A system for determining crop quality, comprising: At least one X-ray scanner for obtaining at least two scanned images of a crop, wherein the first of the at least two scanned images is obtained along a first plane relative to the crop, and the second of the at least two scanned images is obtained along a second plane relative to the crop, wherein the first plane is angularly shifted relative to the second plane; and A controller coupled to an X-ray scanner, wherein the controller is adapted to: Register the first and second images; Correct and register the first and second images; and The quality of the crop is determined from the corrected first and second scan images.

34. The system according to claim 33, wherein, The first plane is angularly shifted relative to the second plane by an angle between 90 degrees and 270 degrees.

35. The system according to claim 33, wherein, The first plane and the second plane are parallel to each other.

36. The system according to claim 33, wherein, Registering the first scanned image and the second scanned image includes matching the first image and the second image by flipping and translating the other relative to at least one of the first image and the second image.

37. The system of claim 33, comprising two X-ray scanners for simultaneously acquiring at least two scanned images of the crop.

38. The system according to claim 33, wherein, The at least one X-ray scanner is used to scan the crop at least twice to obtain at least two scan images of the crop.

39. The system according to claim 33, wherein, The first and second scan images for calibration and registration include images that have been calibrated for a number of predefined parameters.

40. The system according to claim 33, wherein, The first and second scanned images for calibration registration include images that are calibrated for one or more of contrast, brightness, intensity, or scale.

41. The system according to claim 33, wherein, Determining crop quality based on the corrected first and second scan images involves identifying one or more fruit clusters in the images and analyzing the strength of the clusters pixel by pixel.

42. The system of claim 33 further includes summing and correlating the analysis intensities of clusters over a predetermined time period.

43. A system for estimating crop weight, wherein, The crop comprises at least one row of plants, and wherein the at least one row of plants comprises fruiting vines and / or branches, the system comprising: At least one X-ray scanner for irradiating a predetermined area of ​​a crop with X-rays from at least both sides; and A controller coupled to at least one X-ray scanner, wherein the controller is adapted to: Obtain scanned images of at least one row of plants; Perform contrast enhancement and noise reduction on each collected scanned image; Perform global registration of contrast-enhanced and denoised images; A segmented image is obtained by separating vines and / or branches from the image; Perform local registration on the obtained segmented image; Perform a clustering piecewise function on a predefined set of segmented images; Segmented images are processed using distance calibration; and The weight of the crop is estimated by using distance-calibrated images.

44. The system according to claim 43, wherein, Collecting scanned image data of at least one row of plants includes flipping scanned images generated by irradiating a predefined area of ​​the crop with X-rays in the same predefined orientation.

45. The system according to claim 43, wherein, Obtaining a segmented image involves cutting off the separated vines and / or branches from the image by discarding the edges of the vines and / or branches.

46. ​​The system according to claim 43, wherein, Performing local registration involves obtaining alignment between pairs of images showing different views of the same plant.

47. The system of claim 43, for predicting crop yield using the distance-calibrated image.

48. The system according to claim 43, wherein, Obtaining scanned images of at least one row of plants includes: Extract image data from the original data file; Create a schematic diagram for each predefined region; The schematic diagram depicts the movement of a vehicle carrying scanning equipment used to irradiate crops with X-rays; and The GPS coordinates of a point on a schematic diagram are determined by relating at least one timestamp of one or more GPS coordinates to at least one timestamp when the point was captured.

49. The system according to claim 43, wherein, The controller positions the plant rows along the corresponding direction based on the vehicle's direction of motion; and flips all the positioned plant rows in a predetermined direction to obtain a consistent plant sequence in each obtained scan image.

50. The system according to claim 43, wherein, The controller normalizes each acquired scanned image by using a predefined normalization bar.

51. The system according to claim 43, wherein, The controller instructs the X-ray scanner to scan the GPS coordinates of each predetermined area to obtain the distance between plant rows in the area.

52. The system according to claim 43, wherein, The controller identifies and annotates the segmented image.

53. The system according to claim 43, wherein, The controller processes the segmented image using a coarse clustering segmentation method.

54. The system according to claim 43, wherein, The controller executes the clustering segmentation function, which can be either a classic clustering segmentation function or a deep learning clustering segmentation function.

55. The system according to claim 43, wherein, The controller determines the clustering processing technique used to process each segmented image.

56. The system according to claim 43, wherein, The controller determines the weight of the fruit hanging on the plant by measuring changes in the X-ray signal backscattered by the fruit over a predetermined time period.

57. The system according to claim 56, wherein, The fruit includes one of grapes, berries, citrus fruits, apples, melons, and tomatoes.

58. The system according to claim 43, wherein, The X-ray signal backscattered from the fruit is proportional to the fruit's mass and the distance between the fruit and the scanning system that generates the X-rays used to irradiate the fruit.

59. The system according to claim 58, wherein, The X-ray signal backscattered from the fruit is proportional to the square of the distance between the fruit and the scanning system.

60. The system according to claim 43, wherein, The total mass of the fruit is determined by integrating the signal intensity of the X-ray signal backscattered from the fruit through the crop.

Images