Sorting apparatus, detection system for sorting apparatus, and related methods

Abstract

A mechanism for identifying one or more unwanted elements within a set of multiple elements. A first image of the set of multiple elements is generated in response to fluorescence emanating from light in a first wavelength band. A second image of the set is generated in response to reflected / transmitted light in a second wavelength band. The first and second images are processed to identify the one or more unwanted elements.

Description

Cross-reference to related applications

[0001] None applicable.

Technical Field

[0002] The present embodiment generally relates to an optical bulk product sorting apparatus and a method of operating an optical sorting apparatus.

Background Art

[0003] Automatic sorting apparatuses have conventionally been used to sort bulk products. In particular, sorting apparatuses are typically used to separate specific individual pieces (e.g., grains) from a bulk product and sort and / or discard them. A sorting apparatus generally includes a conveyor system that generates a flow of the bulk product. The bulk product can include nuts, grains, seeds, or plastic pieces. A sorting apparatus typically includes an optical detection system arranged to acquire and analyze an image of the flow of the bulk product. The detection system is typically configured to provide image data to a control device. The control device removes selected individual pieces (e.g., grains) from the bulk product by transmitting a control signal to an ejection device based on information determined from the acquired image. The ejection device typically includes an air nozzle that generates an air jet and / or a pneumatic ejection device.

Summary of the Invention

Problems to be Solved by the Invention

[0004] One or more embodiments include a method of identifying unwanted elements within a set of one or more elements.

[0005] The method comprises the steps of: irradiating the assembly with first light in a first wavelength band, wherein the first light causes fluorescence at a level in at least one first unwanted element of the assembly, and the level in the at least one first unwanted element is different from the fluorescence level in a desired element of the assembly; irradiating the assembly with second light in a second wavelength band, wherein the second light transmits or reflects the second light to at least one second unwanted element of the assembly in a manner different from that of the desired element of the assembly; generating first image data that changes according to the amount of fluorescence generated by each element of the assembly in response to the first light; generating second image data that changes according to the amount of second light reflected or transmitted by each element in the assembly; and identifying unwanted elements in the assembly by processing at least the first image data and the second image data.

[0006] In some embodiments, the step of irradiating the collection with a first light is performed at a different time than the step of irradiating the collection with a second light, and the step of processing at least the first image data and the second image data includes the step of generating a first combined image data by combining the first image data and the second image data, and the step of identifying unwanted elements in the collection by processing the first combined image data.

[0007] The step of combining the first image data and the second image data includes the step of overlaying the second image data on the first image data, or the step of overlaying the first image data on the second image data.

[0008] In some cases, the first wavelength band is the ultraviolet wavelength band.

[0009] In some cases, the second wavelength band is the visible light wavelength band.

[0010] The method may further include the steps of: irradiating the collection with third light in a third wavelength band different from the second wavelength band, wherein the third light transmits or reflects the third light to at least one third unwanted element of the collection different from a desired element of the collection; and generating third image data which changes in proportion to the amount of third light reflected or transmitted by each element in the collection. The steps of processing at least the first and second image data include identifying unwanted elements in the collection by processing at least the first, second, and third image data.

[0011] Optionally, the step of irradiating the collection with a first light is performed at a different time than the step of irradiating the collection with a second light, and the step of irradiating the collection with a third light is performed at a different time than both the step of irradiating the collection with a first light and the step of irradiating the collection with a second light, and the step of processing at least the first image data and the second image data includes the step of generating a second combined image data by combining the first image data, the second image data and the third image data, and the step of identifying unnecessary elements in the collection by processing the second combined image data.

[0012] The step of combining the first image data, the second image data, and the third image data may include the step of generating the combined image data by overlapping the first image data, the second image data, and the third image data on top of each other.

[0013] The third wavelength band may be an infrared wavelength band. In some examples, the third wavelength band is a short-wave infrared wavelength band.

[0014] In some examples, the first image data includes a first two-dimensional image, and the second image data includes a second two-dimensional image.

[0015] The step of identifying unnecessary elements in the set by processing at least the first image data and the second image data may include the step of identifying a representation of any element in a set of one or more elements in the first image data and the second image data by processing at least the first image data and the second image data, and the step of determining whether each of the identified representations of any element is an unnecessary element by processing the first image data and the second image data.

[0016] The step of generating the first image data may include the step of capturing light, as a first capture image, which includes at least a portion of the fluorescence emitted by any element in the collection as a result of irradiating the collection with first light, and the step of generating first image data representing the first capture image.

[0017] The step of generating the second image data may include the step of capturing light as a second capture image, which includes at least a portion of the second light that is reflected or transmitted by any element in the collection as a result of irradiating the collection with the second light, and the step of generating second image data representing the second capture image.

[0018] A detection system is also provided, comprising a collection support configured to support a collection of one or more elements, at least one light source, at least one image sensor, and a computer device. The at least one light source is configured to irradiate the collection supported by the collection support with first light in a first wavelength band, and to irradiate the collection supported by the collection support with second light in a second wavelength band, wherein the first light causes fluorescence at a level in at least one first unwanted element of the collection, the level in the at least one first unwanted element being different from the fluorescence level in a desired element of the collection, and the second light causes at least one second unwanted element of the collection to transmit or reflect the second light in a way different from that of the desired element of the collection. The at least one image sensor is configured to generate first image data that changes according to the amount of fluorescence generated by each element of the collection in response to the first light, and to generate second image data that changes according to the amount of second light reflected or transmitted by each element in the collection. The computer device comprises at least one processing unit and at least one non-temporary computer-readable storage medium that stores instructions, when executed by the at least one processing unit, causing the at least one processing unit to process at least the first image data and the second image data to identify unnecessary elements in the set.

[0019] The at least one light source may include a first light source configured to generate the first light and a second light source configured to generate the second light and separated from the first light source.

[0020] The at least one image sensor may include a first image sensor configured to generate the first image data and a second image sensor configured to generate the second image data and separated from the first image sensor.

[0021] The at least one image sensor may include an image sensor positioned to receive reflection of the second light by each element in the set.

[0022] The at least one image sensor may include only at least one image sensor that functions in the visible light spectrum.

[0023] There is also provided a sorting device including the detection system disclosed in the present application, an element exclusion device configured to control and exclude a plurality of elements from a set of one or more elements, and a system control device configured to control the operation of the element exclusion device to exclude unnecessary elements from the set.

[0024] Other technical features may be readily apparent to those skilled in the art from the following drawings, description, and claims.

[0025] It should be understood that within the scope of the present application, the various aspects, embodiments, examples, alternatives, and their individual features described herein can be adopted independently or in any possible and suitable combination. When a feature is described in relation to a single aspect or embodiment, it should be understood that the feature is applicable to all aspects and embodiments, unless otherwise specified or the feature is not suitable.

Brief Description of the Drawings

[0026] While the specification is concluded with claims that particularly identify and clearly claim what is regarded as embodiments of the present specification, the following description of exemplary embodiments may more readily facilitate an understanding of various features and advantages when read in conjunction with the accompanying drawings.

[0027] [Figure 1] A sorting device according to one or more embodiments of the present specification is shown. [Figure 2] A proposed method for identifying unnecessary elements within a set of one or more elements according to one or more embodiments of the present specification is shown. [Figure 3] Another proposed method for excluding unnecessary elements from a set of one or more elements is shown. [Figure 4] An example of a false-color image generated according to one or more embodiments of the present specification is shown. [Figure 5]A graph showing the capture wavelengths of colors in image data according to one or more embodiments of this specification is shown. [Figure 6] This document shows a computer device according to an embodiment of this specification. [Modes for carrying out the invention]

[0028] The figures shown herein are not intended to represent the actual appearance of any specific sorting device, detection system, elimination device, component, or system, but are idealized representations used to illustrate embodiments of the disclosure. Furthermore, elements common to multiple drawings may retain the same numerical designations for convenience and clarity.

[0029] The following description provides specific details of the embodiments. However, those skilled in the art will understand that the disclosed embodiments can be implemented without many such specific details. In fact, the disclosed embodiments can also be implemented in combination with the prior art used in the industry. In addition, the description provided below does not include all elements that constitute a complete structure or assembly. Only the processes and structures necessary to understand the disclosed embodiments are described in detail below. Additional prior art processes and structures are also available. The drawings accompanying this application are for illustrative purposes only and are not necessarily drawn to scale.

[0030] In this specification, “equipment,” “includes,” “possess,” “characterize,” and their grammatical equivalents are inclusive or open terms that do not exclude additional, undescribed elements or process steps, and also include the more restrictive “consist of only,” “essentially consist of only,” and their grammatical equivalents.

[0031] In this specification, singular forms following definite and indefinite articles are intended to include plural forms unless the context clearly indicates otherwise.

[0032] In this specification, terms such as “may,” “may,” and “may be” are used with respect to materials, structures, features, or method processes to indicate that such are intended to be used in the implementation of embodiments of this disclosure, and are preferred over the more restrictive “is” to avoid any implication that other compatible materials, structures, features, and methods should or must not be used in combination with them.

[0033] In this specification, “configured” means that the size, shape, material composition, or arrangement of one or more structures and devices facilitates the operation of the structures and devices in a predetermined manner.

[0034] In this specification, terms such as "First," "Second," etc., are used for clarity and convenience in understanding the disclosure and accompanying drawings, and do not imply any particular order or sequence, nor do they have such meaning unless the context clearly indicates otherwise.

[0035] In this specification, the term “substantially” means, and includes, the degree to which a given parameter, characteristic, or condition is understood to be met with small variation, such as within acceptable manufacturing tolerances, by a person skilled in the art. For example, depending on the particular parameter, characteristic, or condition to be met, that parameter, characteristic, or condition may be met by at least 90.0%, at least 95.0%, at least 99.0%, or at least 99.9%.

[0036] In this specification, the term "approximately" includes the stated values ​​with respect to a given parameter and its meaning is determined by the context (for example, including variations due to measurement errors or manufacturing tolerances associated with a given parameter).

[0037] In this specification, "and / or" includes any combination of one or more of the relevant enumerated items.

[0038] In this specification, when the term "intensity" is used in relation to light, it refers to radiant radiation measured in watts per steradian (W / sr), luminous intensity measured in lumens per steradian (lm / sr) or candela (cd), or watts per square meter (W / m²). 2 This refers to one or more radiation levels measured by ).

[0039] In the context of this disclosure, ultraviolet light may be defined as light having wavelengths from 1 nm to 380 nm, visible light as light having wavelengths from 380 nm to 780 nm, and infrared light as light having wavelengths from 780 nm to 106 nm. However, other appropriate definitions will be apparent to those skilled in the art.

[0040] Multiple embodiments include a sorting device having one or more light sources and one or more image sensors to identify the presence and / or location of unwanted elements in a collection of elements (transported by the sorting device). Embodiments include performing such identification using image data generated by the image sensors. This identification can be used, for example, for control to remove unwanted elements from the collection of elements.

[0041] The sorting apparatus and methods described herein may offer advantages over conventional sorting apparatus and methods. In particular, conventional sorting apparatuses aim to distinguish unwanted elements from desired elements. However, some unwanted elements are very similar to desired elements in terms of color, shape, and other characteristics, making it difficult to distinguish between unwanted and desired elements. This can lead to missorting of element sets, for example, mislabeling unwanted elements as desired elements. The proposed techniques can enable more accurate identification of unwanted elements or increase the number of types of unwanted elements that can be identified within a single sorting apparatus. This can reduce sorting errors and the risk of contamination of sets (e.g., containing desired elements) with unwanted elements. This is particularly advantageous when the unwanted elements are contaminated or infected elements (e.g., contaminated grains).

[0042] Figure 1 shows a sorting apparatus 102 to which one or more proposed embodiments can be applied.

[0043] The sorting device 102 may include a support frame 104, an input system 106, at least one detection system 108, at least one element removal device 110, and a plurality of collection containers 112. The input system 106 may function as a collection support for supporting and / or moving a collection of one or more elements (to be sorted).

[0044] As will be described later, the sorting device 102 may be used to sort a collection of elements 116 (i.e., bulk products or granular products) such as nuts, seeds, grains, and plastic pieces. The sorting device 102 may be configured to sort a collection of elements based on one or more of the size, shape, color, type, chemical properties, or material of the elements. In particular, the sorting device 102 can sort the elements of the collection 116 according to pre-selected sorting criteria. As an unspecified example, the sorting device 102 may be used to sort grains based on the quality of the grains (which may be determined by the color of the grains). As another unspecified example, the sorting device 102 may be used to sort a collection of plastic pieces based on the type of plastic piece.

[0045] The feeding system 106 of the sorting device 102 may include a hopper 118 and a chute and / or belt 120. The hopper 118 defines a path to the chute and / or belt 120, and in some embodiments, the hopper 118 may include one or more vibrators (e.g., vibrator feeders), augers, or other feeding devices that can feed the collection 116 of elements from the hopper 118 to the chute and / or belt 120. The chute and / or belt 120 may be sized, shaped, and arranged such that the elements of the collection 116 descend by gravity and / or a conveyor belt and pass in front of at least one detection system 108. For example, the chute and / or belt 120 may be configured to allow the flow 122 of elements in the collection to pass in front of at least one detection system 108 at a selected speed (e.g., velocity).

[0046] The detection system 108 may include at least one light source 124 and at least one image sensor 128. The detection system 108 may further include a computer device (not shown). The detection system may include one or more background units 126 and / or one or more reference units 134.

[0047] At least one light source is configured to illuminate the collection of elements 116 as they pass in front of the detection system 108. For example, at least one light source 124 may include one or more light-emitting diodes (LEDs) for emitting light.

[0048] At least one light source 124 is configured to emit at least two different types of light, i.e., light belonging to two different wavelength bands. Thus, the light source can emit two or more of the following: visible light, short-wavelength infrared light (SWIR light), near-infrared light (NIR light), infrared light (IR light), or ultraviolet light (UV light). In one or more embodiments, at least one light source is configured to emit light individually within two or more specific (e.g., selected) spectral bands of the electromagnetic spectrum.

[0049] One or more image sensors 128 may include one or more charge-coupled device (CCD) cameras, IR cameras, UV cameras, or RGB cameras. When in use, one or more image sensors 128 may be oriented and configured to detect (e.g., image) reflected light, light transmitted through the collection 116, and / or fluorescent light emitted by the collection. For example, the field of view of one or more image sensors 128 may include at least a portion of the flow 122 of the collection 116.

[0050] In one or more embodiments, the detection system 108 further includes one or more optical filters that can filter (e.g., narrow the wavelength band) the light detected (e.g., captured) by one or more image sensors 128. In some embodiments, one or more optical filters can narrow the reflected light to a specific (e.g., selected) wavelength to highlight sorting criteria (e.g., criteria for distinguishing grades or types of bulk products).

[0051] The computer device of the detection system 108 is configured to process image data generated by the image sensor 128 to identify unwanted elements. In particular, the computer device can receive image data from at least one image sensor 128 and analyze the image data to identify unwanted elements in the set 116.

[0052] The computer device may include at least one processor and at least one non-temporary computer-readable storage medium that stores instructions for processing image data to identify unwanted elements when executed by at least one processing unit.

[0053] A suitable example of the computer device for the detection system 108 is described in more detail in connection with Figure 6, which is described later in this disclosure.

[0054] As described above, the detection system 108 may include one or more background units 126. The one or more background units 126 may be positioned and oriented behind the assembly 116 with respect to one or more image sensors 128, i.e., behind the assembly from the viewpoint of the image sensors. The background units 126 are intended to provide better detection and imaging of individual grains of the bulk product 116. The one or more background units 126 may include any of the known background units used in sorting devices.

[0055] As described above, at least one detection system 108 may include one or more reference units 134. Each reference unit 134 may be positioned within the field of view of at least one image sensor 128. In some embodiments, each reference unit 134 may have or exhibit at least substantially constant color. In a non-limiting example, each reference unit 134 may include a colored piston positioned in a transparent cylinder, the center of the colored piston (e.g., a reference area) may be protected from contamination (e.g., dust or discoloration) by one or more gaskets. The sorting device 102 can utilize multiple reference units 134 as reference points for light recognized by the detection system 108.

[0056] At least one elimination device 110 may be connected to and at least partially controlled by a system control device 114. In particular, the system control device 114 may generate control signals to at least one element elimination device 110, at least partially based on the analysis of image data by the computer device of the detection system 108.

[0057] In some cases, the input system 106 may also be connected to the system control unit 114 in a way that makes it operable and at least partially controlled. For example, the system control unit 114 may send control signals to the input system 106 to cause the input system 106 to feed the element assembly 116 to the chute and / or belt 120 of the sorting device 102 and to generate a flow of bulk product 116 on the chute and / or belt 120 of the sorting device 102.

[0058] The computer units for the system control and detection systems may be integrated into a single component. In practice, this means that a single computer unit can perform the functions of both the system control and detection system computer units.

[0059] This disclosure proposes a novel technique for identifying unwanted elements within a set of elements. This approach is useful, for example, for identifying which elements in a set of elements contained within a sorting device should be removed by the sorting device's element elimination system.

[0060] More specifically, the present disclosure provides a mechanism for identifying one or more unwanted elements within a set of elements. A first image data of the set of elements is generated in response to fluorescence from light in a first wavelength band. A second image data of the set is generated in response to reflected / transmitted light in a second wavelength band. The first and second image data are processed to identify one or more unwanted elements.

[0061] Figure 2 is a flowchart of a method 200 for identifying unwanted elements in a set of one or more elements. This method may be performed by a sorting device, such as the sorting device described above, to identify unwanted elements. More specifically, method 200 may be performed by the detection system described above.

[0062] Method 200 may include irradiating the aggregate with a first light in a first wavelength band, as shown in step 210 of Figure 2. In a non-limiting example, irradiating the aggregate with a first light in a first wavelength band may include irradiating the aggregate with at least one light source 124 of the sorting device 102. The first wavelength band of the first light is selected or determined so as to produce a different fluorescence level in at least one first unwanted element than in the desired element. Thus, the first wavelength band of the first light may be selected or determined so as to fluoresce when a subset of known unwanted elements (e.g., a single type of unwanted element) is irradiated with the first light.

[0063] In the first embodiment, the first light includes ultraviolet (UV) light, and may include, for example, only ultraviolet light. Ultraviolet light causes fluorescence at various levels (e.g., visible light) in contaminated / moldy / infected grains (i.e., grains containing contaminants such as aflatoxins) and healthy grains, grains with different gluten content, grains of different genera (e.g., oat grains of different genera), etc. This makes it possible to distinguish between elements with such different properties.

[0064] In a second embodiment, the first light may include red light, or for example, only red light. Red light causes chlorophyll to fluoresce (for example, it causes infrared light to fluoresce). Therefore, elements with different chlorophyll content will exhibit different fluorescence to such a first light, i.e., produce different fluorescence levels. This makes it possible to distinguish between organic and inorganic elements, grain and non-grain elements (e.g., leaves), etc.

[0065] Method 200 may also include illuminating the aggregate with a second light in a second wavelength band, as shown in step 220 of Figure 2. In a non-limiting example, illuminating the aggregate with a second light in a second wavelength band may include illuminating the aggregate with at least one light source 124 of the sorting device 102. The second light is configured to transmit or reflect the second light so that at least one second unwanted element is different from the desired element. The second light may include, for example, visible light. Therefore, the second light can be light that is reflected / transmitted differently by different elements. This allows for the distinction of elements such as different colors, shapes, or patterns.

[0066] Steps 210 and 220 may be performed using one or more light sources of the detection system. In particular, illumination with the first light may be performed using the first light source, and illumination with the second light may be performed using a second different light source.

[0067] Method 200 may also include generating a first image data that changes depending on the amount of fluorescence produced or emitted by each element in the collection in response to a first light, as shown in step 230 of Figure 2. Thus, the values ​​of the first image data will differ when the amount or level of fluorescence produced by the elements of the collection is different. Step 230 can be performed using at least one image sensor.

[0068] The first image data may be, for example, a first two-dimensional image. Techniques for generating two-dimensional images are well known.

[0069] The fluorescence wavelength may be known in advance due to the known fluorescence properties of the unwanted or desired element. The image sensor used to generate the first image data may be designed to be sensitive to the specific wavelength at which the unwanted or desired element fluoresces. This makes it easier to distinguish between the unwanted and desired elements.

[0070] Step 230 may be performed by generating a photosensitive signal using at least one image sensor, i.e., by capturing an image with at least one image sensor. The image sensor is configured to be sensitive to fluorescence generated or emitted as a result of each element in the set being irradiated with a first light.

[0071] In this way, the first image data is sensitive to fluorescence at various levels emitted by any element in the set (as a result of being irradiated with the first light). Therefore, the first image data is configured to distinguish fluorescence at various levels from any element in the set of imagesed elements. In other words, the first image data changes or becomes different when the fluorescence levels and / or amounts emitted by any element in the set of one or more elements irradiated with the first light are different.

[0072] Step 230 may include, for example, step 231, capturing light containing at least a portion of the fluorescence emitted by any element in the assembly as first capture light. Thus, step 231 includes capturing light containing at least a portion of the fluorescence emitted by any element in the assembly as a result of illuminating the assembly with first light as first capture light. More specifically, step 231 may include capturing such light using at least one image sensor 128.

[0073] Step 230 may also include, as Step 232, generating first image data representing first captured light. Step 232 is similarly performed by at least one image sensor and connected circuitry, configured to generate image data in response to captured light. Mechanisms for converting captured light into image data are well known and established in the art.

[0074] Method 200 may also include generating a second image data that changes depending on the amount of second light reflected or transmitted by each element in the set, as shown in step 240 of Figure 2. This can be done using at least one image sensor. Thus, the second image data changes depending on the reflected / transmitted light in the second wavelength band.

[0075] The second image data could be, for example, a second two-dimensional image. Techniques for generating two-dimensional images are well known.

[0076] The second image data may include, for example, only image data corresponding to the amount of second light reflected by each element in the set. This can be achieved by using an image sensor positioned to receive only the second light reflected by the set (and not transmitted light). For example, the image sensor(s) used may be positioned on the same side as the light source used to generate the second light, so that the second light is reflected by the set and incident on the image sensor.

[0077] In other examples, the second image data may contain only the image data corresponding to the amount of second light transmitted by each element in the set. This can be achieved by using an image sensor positioned to receive only the second light reflected by the set (and not transmitted light). For example, the image sensor(s) may be positioned on the opposite side of the light source that generates the second light so that the second light passes through the set and enters the image sensor.

[0078] These methods recognize that unwanted elements may have different reflection / transmission properties than the desired elements. For example, an unwanted element may absorb more light than the desired element. Another example is that an unwanted element may have a different color than the desired element. Yet another example is that a desired element may transmit more light (i.e., absorb less) than the desired element.

[0079] Step 240 may be performed by generating a light-sensitive signal using at least one image sensor, i.e., by capturing an image with at least one image sensor. The image sensor is configured to be sensitive to light reflected or transmitted by each element in the set as a result of being irradiated with a second light.

[0080] In this way, the second image data is sensitive to the various levels of the second light that any element in the set reflects / transmits (as a result of being illuminated by the second light). Therefore, the second image data is configured to distinguish between reflectances / transmitters of varying magnitudes by any element in the set of captured elements. In other words, the second image data changes or becomes different when any element in the set of one or more elements illuminated by the second light reflects / transmits different levels and / or amounts of the second light.

[0081] Step 240 may include, for example, step 241, capturing light as second captured light, which includes at least a portion of the second light reflected or transmitted by any element in the collection. Thus, the second captured light includes at least a portion of the second light reflected or transmitted by any element in the collection as a result of illuminating the collection with the second light. More specifically, step 241 may include capturing such light using at least one image sensor 128.

[0082] Step 240 may also include, as Step 242, generating a second image data representing the light captured in Step 241. Step 242 is similarly performed by at least one image sensor and connected circuitry, configured to generate image data in response to captured light. Mechanisms for converting captured light into image data are well known and established in the art. In some embodiments, the generation of the second image data may include generating a true color image (e.g., a matrix of RGB pixels where R is red, G is green, and B is blue). In additional embodiments, the generation of the second image data may include generating a pseudo-color image in which the R, G, and B channels reflect light of specific wavelengths (e.g., colors invisible to the naked eye). In non-limiting examples, the R channel could be 1200 nm light, the B channel 1000 nm light, and the G channel 1400 nm light. Figure 4 shows an example of a pseudo-color image 400a that can be generated by the method described herein. Furthermore, Figure 5 shows a graph 500 showing the wavelengths of light contained in the second image data in one or more embodiments.

[0083] From the above, it is clear that the image sensors used to perform steps 230 and 240 may be adapted to be sensitive to an appropriate wavelength band. For example, the image sensor includes a first image sensor for generating first image data, the first image sensor being sensitive to light in the wavelength band in which unwanted elements fluoresce. The image sensor further includes a second image sensor for generating second image data, the second image sensor being sensitive to light in a second wavelength band.

[0084] The first and second image data may be generated independently of each other, meaning they are not generated simultaneously by the same image sensor. Rather, the first and second image data may be generated at different times or by different image sensors. In some examples, steps 210 and 220 are not performed simultaneously. Rather, steps 210 and 220 may be performed at different times. In particular, steps 210 and 220 are temporally multiplexed with each other (for example, steps 210 and 220 are performed alternately).

[0085] Alternatively, the first and second image data may be generated at the same time, directly creating a combined image data. This can be achieved, for example, by simultaneously illuminating the area with the first and second light sources and capturing the first and second image data with the same or separate image sensors. For example, the image sensor 128 may be a multispectral image sensor that generates the first and second image data simultaneously.

[0086] Method 200 also includes processing at least the first and second image data to identify unwanted elements in the set, as shown in step 250 of Figure 2. More specifically, the processing in step 250 may include identifying the location of the unwanted elements in the set.

[0087] Image data processing techniques for identifying unwanted elements are known in the art. Generally, such techniques use image processing algorithms (e.g., one or more segmentation or classification algorithms) to determine whether representations of unwanted elements exist in the image data. An essential element for distinguishing unwanted and desired elements using image data is that the representations of unwanted elements are distinguishable from the representations of desired elements. Thus, by using first and second image data generated from first and second light sources, it becomes easier to distinguish between unwanted and desired elements. In particular, by generating two different types of image data using two different types of light, different unwanted elements can be distinguished from desired elements.

[0088] Regarding the first image data, some seeds reflect fluorescence, while others do not. Therefore, an algorithm can be used on the acquired image to identify seeds that do not reflect fluorescence (e.g., do not emit light), and the original multilayer image can be converted into a binary mask. Furthermore, as described herein, seeds may be sorted based on whether or not they reflect fluorescence. Each bit in the binary mask indicates whether or not the seed corresponding to that pixel should be removed by the elimination device. The position of that pixel in the image is converted into real-space coordinates, and thus becomes the actual position and time when the air jet should be opened to remove objects passing through. This method can also be applied to sorting machines that use, for example, a conveyor belt instead of a slide, or sorting machines that use a mechanical elimination device or lever instead of an air elimination device.

[0089] The algorithm can be used to distinguish between seeds that reflect fluorescence and those that do not. In some embodiments, the algorithm may distinguish seeds based at least partially on a color threshold. Algorithms that form shapes around colors and detect edges and contours can be used to store data in an n-dimensional space (e.g., including other spectra such as IR and UV in addition to RGB). Anomaly detection is based, for example, on eliminating anything that is too different from the desired target. This can be done in an automated way using cluster detection, and classifiers or neural networks can be trained to perform the analysis. This may allow for the distinction of elements even when the difference is not visually apparent to the operator or cannot be detected by standard tools such as histogram analysis of the data.

[0090] The precise criteria for distinguishing unwanted and desired elements depend on the specific application of the method. As mentioned above, known usable criteria include the color, intensity, and appearance of elements within image data. Therefore, image processing algorithms may identify unwanted elements by searching for specific colors or intensities within image data.

[0091] Therefore, it is understood that the exact procedure for carrying out step 250 depends on the type and identification of the unwanted and desired elements.

[0092] As an example, if we want to include oat grains (desired elements) in a set of elements and identify barley and wheat grains (undesired elements), we can identify barley grains by identifying the fluorescent grains from the first image data, and wheat grains by identifying the orange grains from the second image data. Fluorescent elements can be identified based on their color and / or intensity (e.g., intensity exceeding a specific value for a particular fluorescence color).

[0093] One image data processing technique for identifying unwanted elements is to identify regions that satisfy one or more predetermined criteria (e.g., a specific color, intensity, etc.). The exact criteria depend on the nature of the unwanted element. Each identified region (e.g., exceeding a predetermined size) may represent an unwanted element.

[0094] Another method for processing image data to identify unwanted elements is to use one or more machine learning techniques. The image data is received as input to a machine learning technique, and an index indicating the presence of unwanted elements is output. This index may take the form of a binary index, probability, location information, or segmentation information that identifies the location of unwanted elements within the image data.

[0095] Therefore, in some embodiments, one or more machine learning algorithms are used. Suitable machine learning algorithms can be used in different embodiments of this disclosure. Suitable machine learning algorithms include (artificial) neural networks, support vector machines (SVMs), naive Bayes models, decision tree algorithms, etc., but other suitable examples will be apparent to those skilled in the art.

[0096] Numerous established methods exist for training machine learning algorithms. Typically, such training methods utilize large databases of known input and output data. The machine learning algorithm is modified until the error between the predicted output data obtained by processing the input data and the actual (known) output data approaches zero, i.e., until the predicted output data and the known output data converge. This error value is often defined by a cost function. The exact mechanism for modifying the machine learning algorithm depends on the type of model. Examples used in neural networks include gradient descent and backpropagation algorithms.

[0097] For use in the approach described above, known input data includes image data (e.g., combined image data). Corresponding known output data includes an index indicating the presence of unwanted elements for each example of image data. The index may be prepared or generated by a person with the appropriate skills.

[0098] In some preferred examples, the first light irradiation step 210 is performed at a different time than the second light irradiation step 220. This avoids cross-contamination between the generation of the first and second image data.

[0099] In some examples, the processing in step 250 includes, as step 251, combining the first image data and the second image data to generate combined image data. Thus, the first image data and the second image data are combined to form combined image data. Combining can be performed, for example, by overlaying the second image data on top of the first image data (or overlaying the first image data on top of the second image data). Alternatively, the first image data and the second image data can be combined to create combined image data.

[0100] As described above, the first image data and the second image data are generated using different types of light (first light and second light). Therefore, step 251 substantially generates the combined image data using independent data sources.

[0101] In some examples, the first image data is the first 2D image, and the second image data is the second 2D image.

[0102] For example, step 251 can be simply performed by overlaying a second 2D image on top of a first 2D image. The overlay process can be easily performed, for example, by associating each pixel of the first 2D image with a pixel of the second 2D image, and then summing, averaging, or performing other algorithmic processing on the pixel values ​​for each corresponding pixel to generate a combined image. Alternatives to summing and averaging include multiplication, division, and weighted blending (e.g., weighted sum, weighted multiplication).

[0103] As another example, step 252 can be performed by creating a 3D image by stacking a first 2D image on top of a second 2D image.

[0104] In some cases, the first image data is generated by a first image sensor using first light generated by a first light source. Similarly, the second image data may be generated by a second image sensor, separate from the first image sensor, using second light generated by a second light source. In this way, the first and second image data may be generated using independent or different data sources. Step 251 combines the image data from the two independent data sources and processes them together.

[0105] As an example, the first image data might be a fluorescence image, and the second image data might be a (visible) color image or a pseudo-color SWIR image. The two images are combined, for example, by summing the corresponding pixels, to produce a single combined image.

[0106] Process 250 can then perform step 252, which involves processing the combined image data to identify unwanted elements within the set. Generating combined image data improves the computational efficiency of data processing and reduces the required memory capacity. Combining the first and second image data allows for more precise identification of unwanted elements, such as including criteria that the unwanted elements have a specific appearance, for example, having a specific color in the combined image data.

[0107] The image data processing technique for identifying unnecessary elements within a set is as described above.

[0108] One method for performing step 252 is to process the combined image data to identify the representation of any element (in the set) within the combined image data, which is represented by step 252A in Figure 2. Step 232A can be performed, for example, by processing the combined image data with a partitioning algorithm to partition or identify the elements represented in the combined image data.

[0109] Step 252 may then perform step 252B, which involves processing each identified expression to determine or predict whether it is a desired or unwanted element. This includes determining whether the expression of the identified element meets one or more predetermined criteria, the criteria depending on the exact nature of the unwanted element (e.g., a certain color exceeding a certain intensity).

[0110] Figure 2 also shows any further optional steps of Method 200.

[0111] Method 200 may further include irradiating the assembly with a third light in a third wavelength band different from the second wavelength band, as shown in step 260 of Figure 2. The third light is configured to transmit or reflect the second light such that at least one third unwanted element (different from the second unwanted element) is different from the desired element.

[0112] Therefore, the third light is configured to transmit or reflect the third light in such a way that at least one third unwanted element is different from the desired element. The third unwanted element may transmit / reflect the second light in a way that is similar to or indistinguishable from the desired element. Therefore, the third light may enable the distinction between the desired element and at least one unwanted element that could not be distinguished by the first / second light.

[0113] For example, some wheat varieties are yellow. Currently, it is not possible to accurately distinguish between yellow wheat grains and oat grains based solely on color (i.e., visible light reflectance) or fluorescence properties. However, wheat grains exhibit a different response to infrared light than oat grains (e.g., different reflection properties in the infrared spectrum).

[0114] Exemplary techniques for generating image data that responds to reflected or transmitted light were described in the context of the second image data. A similar understanding can be applied to the third image data, and the relevant procedures (e.g., the arrangement of the image sensor) can be adapted as appropriate. Such examples will not be repeated for the sake of emphasis.

[0115] The third wavelength band may be, for example, the infrared wavelength band, or the short-wavelength infrared wavelength band. The short-wavelength infrared band is generally defined as the wavelength band of 800 nm to 3000 nm, for example, 1000 nm to 3000 nm, or 1400 nm to 3000 nm.

[0116] Method 200 may include generating a third image data that varies depending on the amount of third light reflected or transmitted by each element in the (illuminated) set, as shown in step 270 of Figure 2. This can be done using at least one image sensor. The third image data may be, for example, a third two-dimensional image. Techniques for generating two-dimensional images are well known.

[0117] Therefore, the third image data changes depending on the reflected / transmitted light in the third wavelength band.

[0118] The processing of at least the first and second image data in step 250 may include processing at least the first, second, and third image data to identify unwanted elements in the set. The processing may include, for example, step 251, combining the first, second, and third image data to generate a (second) combined image data. The method may then include, as step 252, processing the combined image data. Processing the (second) combined image data to identify unintended elements in the set.

[0119] Step 270 may be performed by generating a light-sensitive signal using at least one image sensor, that is, by acquiring an image using at least one image sensor. The image sensor is configured to be sensitive to any light that is reflected or transmitted as a result of each element in the set being illuminated with the third light.

[0120] In this way, the third image data is sensitive to the reflection / transmission of third light at various energy levels by any element in the set (as a result of illuminating the set with second light). Therefore, the third image data is configured to distinguish between reflectances / transmitters of various magnitudes by any element in the imaged set of elements. In other words, the third image data changes or becomes different when the energy levels and / or amounts of third light transmitted / reflected by any element in a set containing one or more elements illuminating with third light are different.

[0121] Step 270 may include, for example, as step 271, acquiring light as third acquired light, which includes at least a portion of the third light transmitted or reflected by any element in the collection. Thus, the third acquired light includes at least a portion of the third light reflected or transmitted by any element in the collection as a result of illuminating the collection with the third light. More specifically, step 271 may include acquiring such light using at least one image sensor 128.

[0122] Step 270 may also include step 272 for generating or creating a third image data representing the light acquired in step 271. Step 272 may also be performed by at least one image sensor and / or connected circuitry configured to generate image data in response to acquired light. Mechanisms for converting acquired light into image data are well known and established in the art.

[0123] The proposed embodiment is particularly advantageous when the first light includes ultraviolet light, the second light includes visible light, and the third light includes infrared (IR) light, such as short-wavelength infrared (SWIR) light. This provides a wide range of various properties, making it easier to distinguish between different elements (especially different organic elements, such as different grains).

[0124] In any of the embodiments described above, the output of Method 200 may be information identifying the presence and / or location of an unintended element within a set of one or more illuminated elements. This may take the form of, for example, annotated image data for identifying the location of an element (e.g., for identifying the location of an element) and / or binary data (e.g., for identifying the presence and / or absence of an unintended element). The correspondence between the location in the image data (e.g., the location of an unintended element) and the location in real space may be known, for example, based on a known / predetermined location of the image sensor that acquires the image data.

[0125] Conventionally, it has been found that it is possible to remove unwanted elements from a set of one or more elements using one or more element elimination systems in a sorting device.

[0126] However, it will be understood that this is not a mandatory step. For example, method 200 may be used to monitor for the presence of unintended elements in order to determine whether a set of one or more elements is contaminated. This can then be used to determine whether the entire set of one or more elements needs to be discarded.

[0127] In the example above, only three instances of image data (corresponding to three types of light) are shown, but those skilled in the art will understand that embodiments may include more than three instances of image data for more than three types of light. Thus, N is an integer value of 2 or greater, and N instances of image data may be generated for N types of light illuminating one or more sets of elements. Each of the N types of light may be light in a different or distinguishable wavelength band. The N instances of image data may be processed to identify unintended elements. For example, the N instances of image data may be combined to form combined image data, which may then be processed to identify unintended elements.

[0128] Figure 3 is a flowchart illustrating a method 300 for removing unintended elements from a set of one or more elements.

[0129] Method 300 includes the step of performing Method 200 for identifying unintended elements in the set. Method 300 also includes the step 310 for removing the identified unintended elements. Techniques for removing elements from a set of elements are well known in the art and include, for example, the use of an air injection device, an air nozzle, etc.

[0130] Figure 6 is a schematic diagram of the computer device 402. In some embodiments, the detection system 108 may include a computer device such as the computer device 402 in Figure 6. The computer device 402 may include a communication interface 404, at least one processor 406, memory 408, storage device 410, input / output device 412, and bus 414.

[0131] In some embodiments, the processor 406 includes hardware for executing instructions that constitute a computer program. For example, but not limited to, the processor 406 may retrieve (or fetch) instructions from internal registers, internal caches, memory 408, or storage device 410, decode them, and execute them. In some embodiments, the processor 406 may include one or more internal caches for data, instructions, or addresses. For example, but not limited to, the processor 406 may include one or more instruction caches, one or more data caches, and one or more address translation buffers (TLBs). Instructions in the instruction cache may be copies of instructions in memory 408 or storage device 410.

[0132] Memory 408 may be connected to processor 406. Memory 408 may be used to store data, metadata, and programs for execution by the processor. Memory 408 may include one or more volatile and non-volatile memories, such as random access memory (RAM), read-only memory (ROM), solid-state disk (SSD), flash memory, phase-change memory (PCM), or other types of data storage devices. Memory 408 may be internal memory or distributed memory.

[0133] The storage device 410 may include storage for storing data or instructions. For example, but not limited to, the storage device 410 may include the non-temporary storage media described above. The storage device 410 may include a hard disk drive (HDD), flash memory, optical disk, magneto-optical disk, magnetic tape, Universal Serial Bus (USB) drive, or a combination of two or more of these. The storage device 410 may, as appropriate, include removable or non-removable (fixed) media. The storage device 410 may be internal or external to the storage device 410. In one or more embodiments, the storage device 410 is a non-volatile solid-state memory. In other embodiments, the storage device 410 includes read-only memory (ROM). This ROM may, as appropriate, be a mask ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), electrically variable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0134] The input / output device 412 may enable the operator of the sorting device 102 to provide inputs to the computer device 402, receive outputs, and send and receive other data. The input / output device 412 may include a mouse, keypad or keyboard, joystick, touchscreen, camera, optical scanner, network interface, modem, other known I / O devices, or a combination of these I / O interfaces. The input / output device 412 may include one or more devices for presenting outputs to the operator, such as a graphics engine, a display (e.g., a display screen), one or more output drivers (e.g., display drivers), one or more audio speakers, and one or more audio drivers. In certain embodiments, the input / output device 412 is configured to provide graphical data to the display for presentation to the operator. The graphical data may represent one or more graphical user interfaces and / or other graphical content useful in certain embodiments. The computer device 402 and the input / output device 412 may be used to display data (e.g., image and / or video data) relating to the sorting process and the adjustment of operating parameters of the light source 124 and / or the image sensor 128. Computer device 402 and input / output device 412 may be used.

[0135] The communication interface 404 may include hardware, software, or both. The communication interface 404 may provide one or more interfaces for communication (e.g., packet-based communication) between the computer device 402 and one or more other computing devices or networks (e.g., servers). For example, but not limited to, the communication interface 404 may include a network interface controller (NIC) or network adapter for communication with Ethernet or other wired networks, or a wireless NIC (WNIC) or wireless adapter (e.g., Wi-Fi) for communication with wireless networks.

[0136] In some embodiments, the bus 414 (e.g., a Controller Area Network (CAN) bus) may include hardware, software, or both that connect components of the computer device 402 to each other and to external components.

[0137] All references cited herein are incorporated herein in their entirety. In the event of any conflict between the definitions herein and those in the incorporated references, the definitions herein shall prevail.

[0138] The above example describes first image data related to fluorescence and second image data related to reflection or transmission, but the image data may be collected in either order. Therefore, "first" and "second" do not indicate a temporal order. Furthermore, the image data does not necessarily contain only single light information. For example, both the first and second lights may be combined into the first generated image. In this case, the fluorescence response may be superimposed on the RGB data. The same can happen with multiple IR wavelengths. Contributions from different lights may be combined by simple superposition or multiplexing.

[0139] For example, the first image data may include RGB and fluorescence information, while the second image data may include only fluorescence information.

[0140] The first and second image data may be different parts of a single image. In this case, the second and third images may not exist. In some applications, a single color image or pseudo-color image (i.e., wavelength-shifted RGB) may be used as a means of locating an object and guiding data, and this may be a single multi-layer image.

[0141] The embodiments of the Disclosure described herein and shown in the accompanying drawings do not limit the scope of the Disclosure, which is encompassed by the appended claims and their legal equivalents. Any equivalent embodiments are within the scope of the Disclosure. In fact, in addition to those shown and described herein, various modifications of the Disclosure, such as other useful combinations of the described elements, will be apparent to those skilled in the art from the description herein. Such modifications and embodiments are also within the scope of the appended claims and their equivalents.

Claims

1. A method for identifying an unnecessary element in a set of one or more elements, A step of irradiating the assembly with first light in a first wavelength band, wherein the first light causes fluorescence at a level in at least one first unwanted element of the assembly, and the level in the at least one first unwanted element is different from the fluorescence level in the desired element of the assembly. A step of irradiating the assembly with a second light in a second wavelength band, wherein the second light is transmitted to or reflected by at least one second unwanted element of the assembly in a manner different from that of a desired element of the assembly. A step of generating first image data that changes according to the amount of fluorescence generated by each element of the set in response to the first light, A step of generating a second image data that changes according to the amount of second light reflected or transmitted by each element in the set, A step of identifying unnecessary elements in the set by processing at least the first image data and the second image data. A method of having.

2. The method according to claim 1, The step of irradiating the assembly with the first light is performed at a different time than the step of irradiating the assembly with the second light. At least the step of processing the first image data and the second image data is, A step of generating a first combined image data by combining the first image data and the second image data, The step of identifying unnecessary elements within the set by processing the first combined image data. Methods that include...

3. A method according to claim 2, wherein the step of combining the first image data and the second image data includes the step of superimposing the second image data onto the first image data, or superimposing the first image data onto the second image data.

4. A method according to claim 1, wherein the first wavelength band is the ultraviolet wavelength band.

5. A method according to claim 1, wherein the second wavelength band is the visible light wavelength band.

6. The method according to claim 1, A step of irradiating the assembly with a third light in a third wavelength band different from the second wavelength band, wherein the third light is transmitted to or reflected by a third unwanted element of the assembly that is different from a second unwanted element of the assembly, in a manner different from the desired element of the assembly. The method further comprises the step of generating a third image data that changes according to the amount of third light reflected or transmitted by each element in the set, The step of processing at least the first image data and the second image data includes the step of identifying unnecessary elements in the set by processing at least the first image data, the second image data, and the third image data. method.

7. The method according to claim 6, The step of irradiating the assembly with the first light is performed at a different time than the step of irradiating the assembly with the second light. The step of irradiating the assembly with the third light is performed at a different time than the step of irradiating the assembly with the first light and the step of irradiating the assembly with the second light. At least the step of processing the first image data and the second image data is, A step of generating a second combined image data by combining the first image data, the second image data, and the third image data, Step of identifying unnecessary elements in the set by processing the second combined image data. Methods that include...

8. A method according to claim 7, wherein the step of combining the first image data, the second image data, and the third image data includes the step of generating the combined image data by overlapping the first image data, the second image data, and the third image data on top of each other.

9. A method according to claim 1, wherein the third wavelength band is the infrared wavelength band.

10. A method according to claim 9, wherein the third wavelength band is the short-wavelength infrared wavelength band.

11. A method according to claim 1, wherein the first image data includes a first two-dimensional image, and the second image data includes a second two-dimensional image.

12. The method according to claim 1, The step of identifying unnecessary elements in the set by processing at least the first image data and the second image data is, The steps include: identifying a representation of any element in a set of one or more elements in the first image data and the second image data by processing at least the first image data and the second image data; A step in which, by processing the first image data and the second image data, it is determined whether or not each of the identified representations of any element is an unnecessary element. Methods that include...

13. The method according to claim 1, wherein the step of generating the first image data is: The first acquisition image includes the step of acquiring light that includes at least a portion of the fluorescence emitted by any element in the collection as a result of irradiating the collection with first light, Step of generating first image data representing the first captured image. Methods that include...

14. The method according to claim 1, wherein the step of generating the second image data is: The second acquisition image includes the step of acquiring light that includes at least a portion of the second light reflected or transmitted by any element in the collection as a result of irradiating the collection with the second light, Step of generating second image data representing the second captured image. Methods that include...

15. A set support section configured to support a set of one or more elements, At least one light source, At least one image sensor, Computer device A detection system comprising, The at least one light source is, The assembly supported by the assembly support part is irradiated with first light in the first wavelength band, The assembly supported by the assembly support portion is configured to be irradiated with a second light in the second wavelength band, The first light causes fluorescence at a level in at least one first unwanted element of the set, and the level in the at least one first unwanted element is different from the fluorescence level in the desired element of the set. The second light is transmitted to or reflected by at least one second unwanted element of the set in a manner different from that of the desired element of the set. The at least one image sensor is A first image data is generated that changes according to the amount of fluorescence produced by each element of the set in response to the first light. The system is configured to generate a second image data that changes according to the amount of second light reflected or transmitted by each element in the set, The aforementioned computer device At least one processing unit, The system includes at least one non-temporary computer-readable storage medium that stores instructions causing the at least one processing unit to process at least the first image data and the second image data to identify unnecessary elements in the set, when executed by the at least one processing unit. Detection system.

16. The detection system according to claim 15, wherein the at least one light source is A first light source configured to generate the first light, A second light source configured to generate the second light, and separated from the first light source. A detection system equipped with the following features.

17. A detection system according to claim 15, wherein the at least one image sensor is A first image sensor configured to generate the first image data, A second image sensor configured to generate the second image data and separated from the first image sensor. A detection system equipped with the following features.

18. A detection system according to claim 12, wherein the at least one image sensor includes an image sensor positioned to receive reflection of the second light by each element in the set.

19. A detection system according to claim 1, wherein the at least one image sensor includes only at least one image sensor that operates in the visible light spectrum.

20. A detection system according to any one of claims 12 to 14, An element elimination device configured to control and eliminate multiple elements from a set of one or more elements, A system control device configured to control the operation of the element elimination device and eliminate unwanted elements from the set. A sorting device equipped with the following features.

Images