METHOD AND DEVICE FOR OPTICAL ANALYSIS OF FRUITS OR VEGETABLES AND AUTOMATIC SORTING DEVICE

Description

[0001] The invention relates to a method and device for the optical analysis of fruits or vegetables for the purpose of their automatic sorting. It extends to an automatic fruit or vegetable sorting device implementing an optical analysis method according to the invention and / or comprising an optical analysis device according to the invention.

[0002] We know that optical analysis of fruits or vegetables by imaging in a wavelength band typically between 250 nm and 1000 nm, i.e. from ultraviolet to infrared via the visible range, allows for the non-contact measurement of various parameters, including dimensions (size), sugar content, acidity, degree of ripeness, firmness, presence of external or internal defects, color, etc. To do this, however, it is necessary to multiply the lighting devices and / or imaging devices to activate, for each parameter to be measured, an optical analysis in a wavelength range appropriate for measuring that parameter.Furthermore, measuring the same parameter may require several lighting wavelength ranges and / or several imaging wavelength ranges, or even several different images for the same lighting and / or imaging wavelength range, for example a reflection image and a transmission image.

[0003] The publication by Sun Jason et al., "Multispectral scattering imaging and NIR interactance for apple firmness predictions," published in Potharvest Biology and Technology, vol. 119, 2016, pages 58-68, describes a spatially resolved multispectral imaging technique using reflectance spectroscopy to assess apple firmness. This technique involves applying laser beams at different wavelengths, focused through a 50 µm aperture, to the apple and capturing the image backscattered by the apple. Besides being limited to predicting firmness, this technique does not allow for the analysis of all possible defects across the entire surface of the apple.

[0004] It should be noted that optical analysis techniques by imaging are distinguished in particular from spectrometric or spectroscopic techniques which only allow the analysis of light coming from a focused source such as an optical fiber or a laser, and therefore from a very localized point portion of each object.

[0005] Furthermore, optical analysis of objects must be performed on the different portions of their external surfaces, which are generally rotated while being transported past the lighting devices and cameras (see, for example, WO 01 / 01071, FR 2874424). It is not uncommon for an optical analysis within a single wavelength range of lighting and / or imaging to require multiple lighting devices and / or cameras.

[0006] To perform optical imaging analysis across multiple wavelength ranges, it is necessary to use multiple lighting devices and / or cameras and / or filters placed in front of the lighting devices or cameras to select the different wavelength ranges. For imaging in the ultraviolet and infrared ranges, at least one monochrome camera is used. For imaging in the visible range, at least one color camera is generally used, namely a trichromatic camera with at least one CMOS, CCD, or other sensor.

[0007] WO2016 / 05448 describes a device for the identification of microorganisms.

[0008] Thus, in practice, until now each optical analysis station of an automatic fruit or vegetable sorting device has a plurality of cameras, at least one camera per wavelength range of image capture, that is classically between four and eight cameras per line for imaging each portion of the external surface of each object.

[0009] These various devices are particularly expensive, fragile, bulky, and cumbersome, requiring regular and costly maintenance. It should be noted in this regard that they are subjected to relatively harsh environmental conditions (humid and dirty), typical of fruit and vegetable processing and sorting.

[0010] In addition, the use of filters to select wavelengths is particularly expensive, as such filters are costly to purchase and degrade rapidly over time.

[0011] Furthermore, the transmission to an automated system, consisting of a computer, of the various images delivered by the different cameras, and their processing by this computer system, particularly for the purpose of automatic sorting of objects, requires a time that, although small in absolute value, is no longer negligible in the context of the most modern automatic sorting devices, which can operate at very high speeds, typically more than ten—in particular up to approximately fifty, or even more—fruits or vegetables per second per conveyor line. Thus, this transmission and processing time in known optical analysis processes and devices is likely to limit the increase in speed of certain automatic fruit or vegetable sorting systems. WO 2016 / 018157 discloses a method and device for the optical analysis of fruit by imaging in several wavelength ranges.

[0012] The invention therefore aims to overcome these drawbacks.

[0013] It aims in particular to propose a process and device for optical analysis of fruits or vegetables by imaging in several wavelength ranges which are considerably simplified, and less expensive to install, use and maintain.

[0014] In particular, it aims to propose such a process and such an optical analysis device that can be free of filters for the selection of wavelength domains for image capture.

[0015] Its aim is, in particular, to propose such a process and optical analysis device that are better compatible with the environmental conditions of automated fruit and vegetable sorting, and that allow for the measurement of various parameters at very high speeds and at lower cost. Specifically, it aims to offer an optical imaging analysis of fruits and vegetables that is compatible with very high-speed automated sorting, meaning that it does not, in itself, limit the increase in the speed of automated sorting.

[0016] It also aims to enable optical analysis of all sorting criteria, other than weight, of fruits or vegetables exclusively by imaging in different wavelength ranges.

[0017] It also aims to offer an automatic sorting system for objects such as fruits or vegetables with the same advantages.

[0018] Throughout the text, the following terminology is adopted: “Light radiation” means any electromagnetic radiation with a wavelength between 200 nm and 1000 nm; “wavelength range” means a wavelength or a band of wavelengths; “visible range” means any range of wavelengths included in the band of wavelengths between 380 nm and 700 nm; “infrared range” means any range of wavelengths included in the band of wavelengths between 700 nm and 1000 nm; “ultraviolet range” means any range of wavelengths included in the band of wavelengths between 200 nm and 380 nm.

[0019] The invention therefore relates to a method for the optical analysis of objects belonging to the fruit and vegetable group, in which representative images of the objects are produced in different wavelength ranges for image acquisition, characterized in that: a plurality of light sources formed of light-emitting diodes are arranged so that each can apply light radiation to at least a portion of the outer surface of at least one object, called the illuminated object, the different light sources being adapted to be able to selectively apply light radiation in different wavelength ranges to each illuminated object, the light radiation from at least one light source is applied to the entirety of a visible face (by this light radiation, that is to say, according to the direction of propagation of this light radiation) of the outer surface of each object illuminated by this light source, The light sources are controlled according to a predetermined lighting sequence for each illuminated object, successively according to the said different wavelength ranges of lighting. Images are taken by at least one—in particular one and only one—color camera sensitive to light radiation in the visible range and to light radiation in the infrared range, called a multispectral camera, oriented towards a portion of the outer surface of at least one illuminated object corresponding to the entire visible face of the outer surface of the object along the optical axis of the multispectral camera and whose exposure is controlled in synchronicity with the said lighting sequence so as to take, with this same multispectral camera, a plurality of images in different wavelength ranges of the said portion of the outer surface of at least one illuminated object.including at least one image in a visible range and at least one image in an infrared range, each multispectral camera is chosen from the group of cameras comprising a CMOS sensor with a color filter array, free of infrared-cutting filters; and cameras comprising three CMOS sensors, one CMOS sensor for each primary color, free of infrared-cutting filters.

[0020] The invention also extends to an optical analysis device for objects belonging to the fruit and vegetable group, comprising means for producing images of the objects in different wavelength ranges, characterized in that it comprises: a lighting device comprising a plurality of light sources formed of light-emitting diodes arranged to be able to apply light radiation each to at least a portion of the external surface of at least one object, referred to as the illuminated object, the different light sources being adapted to be able to selectively apply light radiation in different wavelength ranges to each illuminated object, the lighting device being adapted to be able to apply light radiation from at least one light source to the entirety of a visible face of the external surface of each object illuminated by that light source, a control device adapted to be able to control these light sources according to a predetermined lighting sequence of each illuminated object, successively according to said different wavelength ranges,at least one - in particular one and only one - color camera sensitive to light radiation in the visible range and to light radiation in the infrared range, called a multispectral camera, oriented towards a portion of the outer surface of at least one illuminated object corresponding to the entire visible face of the outer surface of the object along the optical axis of the multispectral camera, each multispectral camera being chosen from the group of cameras comprising a CMOS sensor with a color filter array, free of infrared blocking filters; and cameras comprising three CMOS sensors, one CMOS sensor for each primary color, free of infrared blocking filters, , and in that said control device is adapted to control the exposure of each multispectral camera in synchronism with said lighting sequence so as to produce with this same multispectral camera a plurality of images in different wavelength ranges of said portion of external surface of at least one illuminated object, including at least one image in a visible range and at least one image in an infrared range.

[0021] The invention also extends to an automatic sorting device for objects belonging to the fruit and vegetable group according to predetermined sorting criteria including: at least one conveyor line capable of transporting objects to object analysis stations according to said sorting criteria, including at least one optical analysis station, an automated system connected to the analysis stations to receive analysis signals, object unloading stations in a plurality of unloading zones, the automated system being programmed to control the selective unloading of each object in a selected unloading zone according to the analysis signals received by this automated system for that object, characterized in that it comprises at least one optical analysis station formed of an optical analysis device according to the invention.

[0022] The inventor was surprised to discover that it is possible to use the same color camera, sensitive to both visible and infrared light, to capture multiple images in different wavelength ranges, including at least one image in the visible range and at least one image in the infrared range. This is achieved by selecting a color camera that is sensitive to infrared light (and, in particular, free of infrared-blocking filters) and, if necessary, adjusting the camera's settings for each infrared image, according to the sensitivity of each camera's color in that specific infrared range.

[0023] Thanks to this invention, all the images needed for the optical analysis of objects in the visible and infrared ranges can be acquired with a single multispectral camera controlled in synchronization with a suitable lighting sequence. This results in considerable cost savings and a significant simplification of each optical analysis station.

[0024] In certain advantageous embodiments of a method according to the invention, at least one multispectral camera is used, equipped with a buffer for recording the different images that can be taken by this camera. Thus, the plurality of images of the external surface portion taken by the same multispectral camera is recorded in a buffer of this multispectral camera. Similarly, advantageously, in an optical analysis device according to the invention, each multispectral camera is equipped with an image storage buffer.Thus, the various images captured during each lighting sequence can be acquired at very high speed (at the maximum image acquisition speed of each multispectral camera) and stored in a buffer of each multispectral camera before being transmitted to an automated image processing system, which can perform image processing at a much slower speed. Consequently, in particular, the bandwidth of the link between each multispectral camera (which is typically a USB 3.0 connection) and the automated system no longer limits the speed of automated fruit or vegetable sorting.

[0025] Each multispectral camera is a color camera comprising a CMOS sensor and a color filter array (such as a Bayer array), and is free of infrared blocking filters. It turns out that such a camera is indeed sensitive to infrared and can be adapted to capture images in both the visible and infrared spectrums.

[0026] A color camera with a CMOS sensor comprises three groups of photosensitive elements, each group detecting one of the primary colors. In certain preferred embodiments of a method according to the invention, each image in an infrared range is acquired with white balance adjusted according to the sensitivity of the multispectral camera to each color (i.e., to one of the primary colors) in said infrared range. Furthermore, in these embodiments, a device according to the invention for acquiring each image in an infrared range is adapted to adjust the white balance according to the sensitivity of the multispectral camera to each color in said infrared range. This results in a higher-quality infrared image.

[0027] Indeed, it has been observed that the sensitivity of each group of photosensitive elements in a color camera to an infrared wavelength varies according to the detection color of that group of photosensitive elements. If the infrared range comprises a single infrared wavelength, the white balance is adjusted according to the sensitivity of each color in the multispectral camera to that infrared wavelength. If the infrared range comprises a band of infrared wavelengths, the white balance can be adjusted according to the sensitivity of each color in the multispectral camera to a characteristic wavelength, particularly the central wavelength, of that band of infrared wavelengths.

[0028] It should be noted, however, that nothing prevents us from planning, as an alternative, the production of infrared images without adjusting the white balance.

[0029] Various imaging techniques can be used in an optical analysis method and device according to the invention, and different imaging techniques can be combined with the same multispectral camera and / or with different multispectral cameras. In particular, in an optical analysis method and device according to the invention, the images acquired by the same multispectral camera can be reflection images, transmission images, and / or scatter images.

[0030] Similarly, each illumination wavelength range may or may not correspond to each imaging wavelength range. Thus, for example, it is possible to illuminate the object in a visible range and acquire images with a multispectral camera in that same visible range; to illuminate the object in an infrared range and acquire images with a multispectral camera in that same infrared range; to illuminate the object in an ultraviolet range and acquire images with a multispectral camera in a visible range (via fluorescence); and so on.

[0031] For reflection imaging, the illuminated area of ​​the object's exterior surface by at least one light source is the entirety of a visible face of the object's exterior surface, illuminated by the light emanating from that light source. At least one multispectral camera is positioned relative to this light source to capture reflection images of this entire visible face. Thus, to create a reflection image, at least one light source applies light emanating from the illuminated portion of the object's exterior surface toward which the multispectral camera is directed.

[0032] To acquire a transmitted image, at least one light source applies light radiation to an area of ​​the object's exterior surface diametrically opposite the illuminated portion of the object's exterior surface toward which the multispectral camera is oriented. To acquire a diffuse image, at least one light source applies light radiation to an area, called the illuminated area, of the object's exterior surface that is distinct from the illuminated portion of the object's exterior surface toward which the multispectral camera is oriented, the multispectral camera being oriented relative to this illuminated area at an angle greater than 90° and less than 180°.

[0033] However, in certain advantageous embodiments according to the invention, at least one multispectral camera is arranged relative to said light sources to acquire images in reflection only. Indeed, in many applications, the optical analysis of fruits or vegetables can be performed exclusively with such reflected images. This is particularly the case for the optical analysis of fruits belonging to the apple, pear, stone fruit (peaches, nectarines, apricots, etc.), and tomato groups.

[0034] In other embodiments according to the invention, at least one multispectral camera can be arranged relative to at least some of the light sources to acquire scatter / transmit images, particularly in the visible range. This is especially relevant for the optical analysis of citrus fruits. Advantageously, at least some of the light sources can be arranged relative to the multispectral camera to enable the acquisition of reflected images, in particular, reflected images in the infrared range under infrared illumination and / or reflected images in the visible range under visible illumination and / or reflected images in the visible range under ultraviolet illumination.

[0035] Depending on the imaging technique used, the illuminated area of ​​the object's outer surface by a light source can be: the entirety of a visible face of the outer surface of the object along the direction of propagation of said light radiation emitted by this light source and applied to the outer surface of the object - in particular a diametrical cap of the object when the latter is globally spherical-; this is particularly advantageously the case for the production of images in reflection; a focal zone of said light radiation on the outer surface of the object, this focal zone being smaller than the visible face of the outer surface of the object along the direction of propagation of said light radiation - in particular smaller than a diametrical cap of the object when the latter is globally spherical-; this is particularly advantageously the case for the production of images in diffusion / transmission.

[0036] That being said, advantageously and according to the invention at least one light source is arranged to be able to apply light radiation over the entirety of one visible face of the outer surface of each object illuminated by this light source.

[0037] Furthermore, nothing prevents certain images produced by a multispectral camera according to the invention from not representing the entire visible face of the object's outer surface along the camera's optical axis. Thus, the portion of the illuminated object's outer surface may, in certain embodiments and for at least one multispectral camera, correspond to a portion of the object's outer surface smaller than a visible face of that outer surface along the camera's optical axis.

[0038] That being said, at least one multispectral camera is positioned relative to the illuminated object such that the portion of the object's outer surface imaged by the multispectral camera corresponds to the entire visible face of the object's outer surface along the optical axis of the multispectral camera. Thus, when the object is generally spherical, the images captured by the multispectral camera are images of a diametrical cap of the object.

[0039] Furthermore, the said lighting sequence and the synchronization of the exposure of each multispectral camera with respect to this lighting sequence can be the subject of numerous implementation variations.

[0040] In particular, in certain advantageous embodiments of the invention, the lighting sequence consists of a succession of lighting durations, a portion of the light sources being activated during each lighting duration, this portion of the light sources being selected to illuminate each object within one of the wavelength ranges. Furthermore, preferably, the wavelength ranges of two successive lighting durations are distinct from one another.

[0041] Each illumination period can be particularly short, and the exposure time of the multispectral camera during each illumination period can also be particularly short, with the image captured during this exposure time being recorded in a buffer of the multispectral camera. Thus, in certain advantageous embodiments according to the invention, each illumination period of said illumination sequence is between 0.1 ms and 5 ms—in particular between 0.1 ms and 1.5 ms.

[0042] Similarly, there is nothing preventing the successive illumination durations of a lighting sequence from being separated by periods of complete extinction of all said light sources. These extinction periods are advantageously between 0.05 ms and 0.5 ms—particularly on the order of 0.1 ms. However, preferably, the different illumination durations of the same lighting sequence follow one another without interruption, that is, without any extinction period between two illumination durations.

[0043] Preferably, the exposure of a multispectral camera is triggered with a delay relative to the activation of a lighting period. This delay is chosen so that each light source within that portion of the lighting period is fully illuminated and active before the multispectral camera is triggered. Indeed, light sources, including ultrafast LEDs, require a certain delay between receiving their activation signal and fully illuminating. For example, this delay is between 0.01 ms and 0.5 ms—specifically, on the order of 0.05 ms.

[0044] Similarly, the exposure shutdown of the multispectral camera is preferably triggered with a slight advance relative to the extinction of at least one of the light sources. This ensures that for each illumination duration, each corresponding light source is and remains fully active and illuminated throughout the entire exposure time of the multispectral camera. For example, this advance is between 0.01 ms and 0.5 ms—specifically, on the order of 0.05 ms.

[0045] Thus, for example, it is possible to acquire, with a single multispectral camera, image series comprising between two and ten—specifically, three to five—successive images of the same illuminated object, in different wavelength ranges, including at least one image in the visible range and at least one image in the infrared range, during an illumination sequence with a total duration of less than 10 ms, specifically between 1 ms and 5 ms. For instance, a series of four successive images can be acquired in a total duration of approximately 3.5 ms. The number and nature of each image in the same series of images in different wavelength ranges are chosen to allow for the optical analysis of the fruits or vegetables to be sorted based on these different images.They can be subject to many variations depending on the important sorting criteria for these fruits or vegetables, which can vary from one variety of fruit or vegetable to another and / or according to the needs of the end customer with regard to the sorting of fruits or vegetables.

[0046] Furthermore, the various light sources are chosen to be compatible with the lighting sequence, the different wavelength ranges of the lighting, and the characteristics of each multispectral camera used. In particular, in certain advantageous embodiments, these light sources include at least one ultrafast-drive LED.In particular, said light sources comprise at least one LED for emitting visible white light, at least one LED for emitting ultraviolet light—in particular, an ultraviolet LED with a wavelength between 250 nm and 380 nm, for example 365 nm—and at least one infrared LED—in particular, an infrared LED with a wavelength between 720 and 780 nm, for example 740 nm; an infrared LED with a wavelength between 800 nm and 850 nm, for example 810 nm; and an infrared LED with a wavelength between 900 nm and 1000 nm, for example 940 nm. Other examples are possible.

[0047] In certain advantageous embodiments according to the invention, said light sources are mounted in an optical chamber having a reflective internal surface, the shape of which is chosen according to the position of the light sources so as to allow uniform illumination of the objects located in the optical field of said at least one multispectral camera. Such uniform illumination is a homogeneous illumination of the entire face of each object exposed to the light radiation from the optical chamber. Furthermore, at least one multispectral camera is oriented so as to acquire reflected images of the entire face of each object thus illuminated.

[0048] A method and optical analysis device according to the invention allow, in particular, the optical analysis of fruits or vegetables even while they are being moved by a conveyor—especially a high-speed conveyor, for example, one capable of transporting more than ten objects per second, particularly up to fifty objects per second or even more, in front of each multispectral camera. Thus, in certain embodiments of a method according to the invention, each illuminated object is moved by a conveyor during said illumination sequence.

[0049] Similarly, a method and device for optical analysis according to the invention allow for the optical analysis of fruits or vegetables while they are rotating. Thus, in certain embodiments of a method according to the invention, each illuminated object is rotated during the said lighting sequence.

[0050] The invention significantly reduces the complexity, number, and size of each optical analysis station in an automated fruit or vegetable sorting system. In particular, in certain embodiments of the automated fruit or vegetable sorting system according to the invention, each optical analysis station comprises fewer than four multispectral cameras—specifically, one or two multispectral cameras—per conveyor line. Indeed, it is possible to acquire, with a single multispectral camera, all the images of a single portion of the external surface of the objects necessary for the optical analysis of fruits or vegetables for automated sorting: size, color, internal defects, external defects, and firmness.For example, when objects are globally symmetrical of revolution - in particular globally spherical - which is the case for most fruits or vegetables, each optical analysis station can include, for each conveyor line, only one multispectral camera (or two multispectral cameras offset laterally and inclined with respect to the vertical, one on each side of the line) to take pictures of the entire external surface of the objects during their passage through the optical analysis station, each object being rotated at least 180° on itself between its entry into the field of the multispectral camera and its exit from the field of the multispectral camera.

[0051] In certain embodiments of the invention, it is even possible to provide for the automatic sorting device to include a single optical analysis station. This is particularly the case for the automatic sorting of fruits or vegetables selected from the group consisting of apples, pears, kiwis, citrus fruits, tomatoes, peaches, apricots, nectarines, plums, persimmons, avocados, mangoes, pomegranates, cantaloupe melons, blueberries, and cherries. Other examples are possible.

[0052] The invention also relates to a method and device for optical analysis of fruits or vegetables, as well as a device for automatic sorting of fruits or vegetables characterized in combination by all or part of the characteristics mentioned above or below.

[0053] Other objects, features and advantages of the invention will become apparent from the following description of some of its embodiments, given by way of non-limiting example, and which refers to the accompanying figures in which: there figure 1 is an example of the sensitivity spectrum of a monochrome camera used in a state-of-the-art optical analysis device, the figure 2 is an example of the sensitivity spectrum of a color camera used in a state-of-the-art optical analysis device, the figure 3 is an example of the sensitivity spectrum of an infrared-sensitive color camera that can be used as a multispectral camera in an optical analysis process and device according to the invention, the figure 4 is a schematic elevation view of an optical analysis station of an automatic sorting device according to an embodiment of the invention, the figure 5 is a schematic cross-sectional view along the VV line of the figure 4 , there figure 6 is a chronogram of an example sequence of lighting and shooting of an optical analysis method according to the invention, the figure 7 is a synoptic diagram of steps during a lighting sequence of an embodiment of an optical analysis method according to the invention, the figure 8 is a synoptic diagram of the cameras, light sources and a camera and light source control device of an optical analysis device according to the invention for an object conveyor line.

[0054] There figure 1 is a sensitivity spectrum of a monochrome camera comprising a CMOS sensor, conventionally used in the prior art for optical analysis for the automatic sorting of fruits or vegetables by infrared and / or ultraviolet imaging. As can be seen, the camera's sensitivity in the infrared range is not zero, but is relatively low. The same is true in the ultraviolet range. figure 2 This is a sensitivity spectrum of a color camera comprising a CMOS sensor and a Bayer filter array, as well as an infrared-cutting filter, conventionally used in the prior art for optical analysis for the automatic sorting of fruits or vegetables by imaging in the visible range. As can be seen, such a color camera is completely insensitive in the infrared range.

[0055] The inventor, however, was surprised to find that a color camera without an infrared-blocking filter is actually particularly sensitive in the infrared range, as can be seen on the figure 3 and can therefore be used in both the visible and infrared ranges, significantly simplifying optical analysis stations in automated fruit and vegetable sorting systems. The invention thus consists of using a single color camera to capture images in at least one visible range, images in at least one infrared range, and optionally images in at least one ultraviolet range.

[0056] An example of an embodiment of an optical analysis device 3 according to the invention is shown in the figures 4 et 5 This optical analysis device constitutes an optical analysis station of an automatic sorting device whose general characteristics are well known in themselves (see for example US 5626238) and which can be the subject of very numerous embodiment variants, the invention being applicable to all these embodiment variants, without limitation, provided that the automatic sorting device allows a plurality of objects 6 made up of fruits or vegetables to be driven horizontally one after the other in relation to the optical analysis device, on supports 9 arranged according to at least one conveyor line 8, in general according to several parallel conveyor lines 8 of fruits or vegetables as in the example shown.

[0057] Preferably, the supports 9 are rotating supports such as rollers driven not only in longitudinal translation but also in rotation about transverse axes of rotation, thus causing the objects 6 to rotate as they pass through the optical analysis device. Indeed, fruits and vegetables generally exhibit at least one rotational symmetry and can therefore be rotated to allow imaging of their entire external surface through successive acquisitions of the same object 6 moving along the conveyor line. The rollers are, for example, formed of a plurality of parallel discs, with two successive rollers along the longitudinal direction of translational drive defining a receiving cavity for an object 6.

[0058] The optical analysis device 3 according to the invention comprises, in the example shown, two lighting devices 7a, 7b, one 7a located upstream and the other 7b located downstream. The two lighting devices 7a, 7b are identical and each comprises a plurality of light sources formed of LEDs (light-emitting diodes) emitting light radiation in different wavelength ranges.

[0059] For example, each lighting device 7a, 7b includes at least one LED, referred to as LED1, emitting white light in the visible range; at least one LED, referred to as LED2, emitting light in the infrared range at a wavelength between 720 and 780 nm, for example centered on 740 nm; at least one LED, referred to as LED3, emitting light in the infrared range at a wavelength between 800 nm and 850 nm, for example centered on 810 nm; at least one LED, referred to as LED4, emitting light in the infrared range at a wavelength between 900 nm and 1000 nm, for example centered on 940 nm; at least one LED, referred to as LED5, emitting light in the ultraviolet range at a wavelength between 250 nm and 380 nm, for example centered on 365 nm. Preferably, each lighting device 7a, 7b includes several LEDs lit simultaneously for each lighting wavelength range.

[0060] The LED1, LED2, LED3, LED4, and LED5 light sources are positioned above the conveyor line 8 and directed upwards, so that they do not provide any direct illumination to the objects 6. Instead, the LED1, LED2, LED3, LED4, and LED5 light sources are mounted in an optical chamber 2 with a reflective inner surface. The optical chamber 2 also has an open base so that the radiation emitted by the light sources is reflected by the inner surface of the optical chamber and directed towards the fruits or vegetables passing below the optical chamber 2.

[0061] The optical analysis device 3 according to the invention comprises at least one infrared-sensitive color camera 4. It should be noted that in a device according to the invention, it is possible to provide one and only one camera 4.

[0062] In the example shown of an optical station for two parallel conveyor lines 8, the optical analysis device 3 preferably includes four cameras 4a, 4b arranged above the two fruit or vegetable conveyor lines 8, in the upper part of the optical chamber 2, i.e. two cameras 4a, 4b for each conveyor line 8.

[0063] The cameras 4a, 4b are arranged with their optical axis slightly inclined with respect to the vertical above the conveyor lines 8. They are positioned respectively on one side and the other of the conveyor line 8, which they image, so that they acquire a substantially different image of each object 6, a first camera 4a allowing to obtain an image of a top portion and a first lateral face of the object carried by the conveyor line 8, and a second camera 4b allowing to obtain an image of a top portion and a second lateral face (opposite to the first lateral face) of the object carried by the conveyor line 8.

[0064] Each camera 4a, 4b has an optical field that covers a sufficiently large length of the corresponding conveyor line 8 so that it can acquire a plurality of images of each object 6 carried along this conveyor line 8, including at least two images of two diametrically opposed portions of each object 6. Each image represents several successive objects 6 longitudinally along the conveyor line 8, the image processing enabling the identification of each object in each image in a manner well known per se. In addition, each camera acquires a plurality of successive image series of the same object as it passes the optical analysis device 3.In practice, it is possible to produce, for example, between 5 and 50 series of images of each object, typically around 10 series of images of each object during its transport in front of each camera, each image corresponding to a different portion of the outer surface of the object, the latter being driven in rotation.

[0065] There is nothing to prevent the alternative provision of other camera arrangements, for example with the same camera focused longitudinally on a single object 6, the optical analysis device 3 then comprising a number of successive longitudinally sufficient cameras to allow the production of series of images of the entire external surface of each object and / or the same multispectral camera allowing the production of images of objects carried by several parallel conveyor lines 8, i.e. images in which objects are juxtaposed laterally.

[0066] As depicted in the figure 4 , the shape of the internal surface of the optical chamber 2 is advantageously chosen according to the position of the light sources of the lighting devices 7a, 7b so as to allow uniform illumination of the objects 6 located in the optical field of the cameras 4a, 4b of the optical analysis device 3.

[0067] Each camera 4a, 4b is an infrared-sensitive color camera, for example chosen from the group of cameras comprising one CMOS sensor (with a color filter array such as a Bayer array but free of an infrared-cutting filter); cameras comprising three CMOS sensors (one CMOS sensor for each primary color) and free of an infrared-cutting filter.

[0068] As shown, cameras 4a, 4b are positioned above conveyor lines 8, with their optical axis oriented downwards inclined towards one of the conveyor lines 8.

[0069] Each camera 4a, 4b is equipped with internal memory 42a, respectively 42b, allowing it to store a plurality of images taken successively by the camera. Thus, each camera can be controlled in burst mode according to a high-speed sequence to successively take several images of each object present in its optical field; the different images in the same series of images can be taken in different wavelength ranges and stored in real time in the camera's memory.Preferably, each camera 4a, 4b is chosen so that its internal memory is sufficient to store several images taken successively by the camera during a lighting sequence. These different images correspond to a series of images in different wavelength ranges for image acquisition. This series of images allows for the optical analysis of the various sorting criteria necessary for the automatic sorting of objects. Furthermore, preferably, each camera 4a, 4b is a high-definition camera, i.e., with more than 1 million pixels. For example, very good results have been obtained with cameras incorporating a 1920 × 1200 pixel CMOS sensor.

[0070] For example, each series of images may include: an image taken by reflection when the object is illuminated by white light (by each LED1), an image taken by reflection when the object is illuminated in an infrared range (by each LED2) at a wavelength between 720 and 780 nm, for example centered on 740 nm, an image taken by reflection when the object is illuminated in an infrared range (by each LED3) at a wavelength between 800 nm and 850 nm, for example centered on 810 nm, an image taken by reflection when the object is illuminated in an infrared range (by each LED4) at a wavelength between 900 nm and 1000 nm, for example centered on 940 nm, an image taken by reflection when the object is illuminated in an ultraviolet range (by each LED5) at a wavelength between 250 nm and 380 nm, by example centered on 365 nm.

[0071] These different images make it possible, in particular, to detect and discriminate between external or internal characteristics, diseases or defects, chosen from the group consisting of scab, bitter spot disease, various forms of rot, superficial scalding, parasites such as the Gloesporium, sunburn, hail damage, bites, stings, punctures and superficial skin roughness such as russeting.

[0072] Other examples are possible, and in particular, nothing prevents us from also producing images using diffusion / transmission, for example by focusing light in the visible spectrum onto an area of ​​an object's outer surface and capturing an image with a camera of a portion of the object's outer surface that is not part of the focal zone, notably with the camera's optical axis forming an angle between 90° and 180° with respect to the axis of the light beam. To do this, it is not necessary to increase the number of cameras in the optical analysis system; it is sufficient, for example, to provide a light source to the side and / or below each conveyor line.

[0073] It should be noted that since each camera 4a, 4b is an infrared-sensitive color camera free of filters other than those necessary for color detection (notably the Bayer filter array), each image captured covers the entire sensitivity spectrum of the camera. Thus, when objects are illuminated with white light, the image formed by the camera is a color image in the visible range. Similarly, when objects are illuminated with infrared light, the image formed by the camera is an image in the same infrared wavelength range. And when objects are illuminated with ultraviolet light, the image formed by the camera is a color representation of the objects' fluorescence in the visible range.

[0074] Each LED1, LED2, LED3, LED4, LED5 light source and each camera 4a, 4b is controlled by a computer system 10 that analyzes the images acquired by the cameras 4a, 4b. This computer system 10 can be modified in any way provided it is adapted and programmed to: control the switching on and off of each light source according to a predetermined lighting sequence to allow the creation of each series of images, receive the different images acquired by the different cameras, analyze these images and deduce criteria for automatic sorting of objects, control the automatic sorting device according to the sorting criteria resulting from the optical analysis of the objects thus carried out.

[0075] Such a computer system 10 may comprise a single computing device, such as a computer as shown in the figures, or a plurality of computing devices and / or computing resources and / or terminals and / or peripherals that are remote from one another and networked. The computer system 10 may also consist of several separate and unconnected computing devices, each dedicated to a specific function: for example, one computing device to control the optical analysis device, and thus perform the imaging and optical analysis of objects; and another computing device for the automatic sorting of objects. The computer system 10 is also adapted to be able to execute a computer program or a plurality of computer programs, in particular for implementing a method according to the invention.

[0076] The computer system 10 includes an electronic control board 20 ( figure 8 ) LED1, LED2, LED3, LED4, LED5 light sources and cameras 4a, 4b. On the figure 8 Only two cameras, 4a and 4b, oriented towards the same conveyor line 8, are shown. Each light source, LED1, LED2, LED3, LED4, and LED5, is connected to the electronic control board 20 by a power cable 21, 22, 23, 24, and 25, each dedicated to a specific group of LEDs that must be activated simultaneously, i.e., corresponding to the same wavelength range of illumination. The switching on and off of the different light sources is controlled by this electronic control board 20. The individual LEDs in each group are preferably connected to the same high-speed control and power board 31, 32, 33, 34, and 35, each dedicated to that group of LEDs and receiving the power cable 21, 22, 23, 24, and 25.

[0077] The electronic control board 20 includes an integrated circuit 26, which is, for example, a field-programmable gate array (FPGA), to which the various LED power cables 21, 22, 23, 24, 25 are directly connected. This integrated circuit 26 also has an input port 27 connected to a network connector 28 on the control board 20, to which a cable 29, such as an Ethernet cable from a motherboard of the computer system 10, can be connected.

[0078] The electronic control board 20 also includes a power supply circuit 36 ​​which can be connected to an electrical power source 37, this power supply circuit 36 ​​supplying the various components of the electronic control board 20.

[0079] Each camera 4a, 4b is also connected to the electronic control board 20 by a dedicated electrical cable 40a, 40b, respectively, so that it can be triggered by the camera to capture an image or a series of images. Each electrical cable 40a, 40b transmits a TRIG CAM trigger signal to the camera, generated by a formatting circuit 41 under the control of the integrated circuit 26. Thus, the integrated circuit 26 can trigger each camera 4a, 4b at a precisely defined time by sending a signal to the formatting circuit 41, which then formats the TRIG CAM trigger signal and delivers it via the electrical cables 40a, 40b.

[0080] Preferably, each camera 4a, 4b includes a programmable sequencer electronic circuit 43a, 43b respectively, enabling the programming of a predetermined sequence of shots corresponding to a series of images as defined above, i.e., corresponding itself to a predetermined lighting sequence. The TRIG CAM trigger signal thus triggers the camera's sequencer and initiates a pre-programmed shooting sequence for each camera 4a, 4b. The sequencer 43a, 43b itself triggers each shot successively, maintaining the camera's exposure for a duration corresponding to the capture of one image. Furthermore, the sequencer 43a, 43b is programmed to control the camera according to different photographic parameters adapted to the creation of each image, particularly depending on the wavelength range considered.

[0081] In particular, for imaging in the infrared range, the sequencer 43a, 43b adjusts the camera's white balance to optimize image quality, based on the infrared sensitivity of each group of photosensitive elements to one of the camera's primary colors. Indeed, as we can see figure 3 The different groups of photosensitive elements in the camera's various primary colors do not all have the same infrared sensitivity. It is therefore advantageous to rebalance these sensitivity differences simply by adjusting the camera's white balance before capturing the corresponding infrared image. This adjustment can be made by experimentally measuring these sensitivity differences for each infrared wavelength of illumination beforehand, that is, by knowing the spectrum as represented figure 3 .

[0082] The computer system 10, its electronic control card 20 and each sequencer 43a, 43b of each multispectral camera together constitute a control device for each lighting sequence and each shooting sequence, enabling each series of images to be produced by each multispectral camera.

[0083] Each camera 4a, 4b is also connected to the electronic control board 20 by a high-speed USB3 cable 44a, respectively 44b, for its power supply and for the transmission of the individual images from a series of images to the computer system 10. These USB3 cables are connected to a multi-port USB3 connection circuit 45 on the electronic control board 20.

[0084] The electronic control board 20 also advantageously includes a button 46 for turning it on or off, and a light 47 for signaling any possible malfunction.

[0085] There figure 6 represents an example of a timing diagram of a lighting sequence according to the invention allowing to produce a series of images as mentioned above.

[0086] At time t1, the electronic control board 20 emits the TRIG CAM trigger signal for cameras 4a and 4b. Upon receiving this trigger signal, the sequencer of each camera executes the sequence for which it has been programmed, corresponding to the CAM signal shown. figure 6 The first shot is triggered at time t3 after a short waiting period after time t1 of receiving the trigger signal.

[0087] Also, the electronic control board 20 triggers the lighting sequence of the different light sources LED1, LED2, LED3, LED4, LED5 according to the corresponding signals represented figure 6 As can be seen, the LED1 light sources are switched on at a time t2 immediately preceding time t3 for a sufficient duration to allow each corresponding LED to light up fully, following the activation of the corresponding control and power supply board.

[0088] Each camera 4a, 4b captures the image for a duration that depends on the image size (number of pixels in rows and number of pixels in columns) and the camera's integration time. At the end of this duration, at time t4, the first image is captured by the camera and stored in its memory 42a, 42b. It should be noted that throughout the entire image capture process, the LED1 light sources remain fully illuminated. At a time t5 following t4, the LED1 light sources are switched off, and the LED2 light sources, corresponding to a different wavelength range, are switched on. Time t5 is slightly offset from the time t4, the end of image capture by the camera, by as small a duration as possible, but long enough to ensure that the light sources remain illuminated throughout the entire capture process.

[0089] The cycle is repeated to capture four more images under successive illumination by light sources LED2, then LED3, then LED4, and finally LED5. The capture of the second image begins at time t6 and ends at time t7. Light sources LED2 are off and light sources LED3 are on at the subsequent time t8. The capture of the third image begins at time t9 and ends at time t10. Light sources LED3 are off and light sources LED4 are on at the subsequent time t11. The capture of the fourth image begins at time t12 and ends at time t13. Light sources LED4 are off and light sources LED5 are on at the subsequent time t14. The capture of the fifth image begins at time t15 and ends at time t16. Light sources LED5 are off at the subsequent time t17.

[0090] After capturing all the images of the same series, all these images can then be transmitted, from the subsequent instant t18 defined by the sequencer of each camera, to the computer system 10 via the USB3 cables 44a, 44b.

[0091] There figure 7 represents a variant in which the electronic control board 20 triggers not only the switching on of the different light sources, but also each camera if the latter are without a sequencer, for a sequence of lighting and capture of a series of images conforming to the timing diagram of the figure 6 .

[0092] In the first step 51, an index i representing each group LEDi of LEDs is initialized to 1. In the example mentioned above, i ranges from 1 to N = 5. In the subsequent step 52, the LEDi light sources are switched on. After a wait Δt1 (equal to t3-t2 for LED1) during step 53, the IMi images from the different cameras 4a, 4b are captured (between t3 and t4 for LED1) in the subsequent step 54 and then stored in the memory of each camera in step 55. After a wait Δt2 (equal to t5-t4 for LED1) during step 56, the LEDi light sources are switched off in the subsequent step 57, and then a test 58 is performed on the index i to determine whether the maximum value N of this index has been reached. If not, the index i is incremented by 1 at step 59 and the process is repeated from step 52 for the lighting of the light sources of the next group.If so, the images are transmitted to the computer system 10 during step 60.

[0093] The invention thus enables optical analysis of objects using multispectral infrared-sensitive color cameras in burst mode, allowing for the acquisition of successive image series at very high speed. Each image series acquired by a multispectral camera according to the invention corresponds to an image acquired by a camera in the prior art. However, instead of requiring multiple cameras to acquire these different images, the invention allows the use of only one or two multispectral cameras for each conveyor line.

[0094] It goes without saying that the invention can be the subject of many variants other than the embodiments described above and represented in the figures.

Claims

1. - Method of optically analysing objects belonging to the fruit and vegetable group in which images representative of the objects are produced in different imaging wavelength ranges, wherein: - a plurality of light sources formed of light-emitting diodes are arranged to be able each to apply light radiation to at least an external surface portion of at least one object, named illuminated object (6), the different light sources being adapted to be able to apply light radiation in different illumination wavelength ranges selectively to each illuminated object, - the light radiation from at least one light source is applied to the whole of a visible face of the external surface of each object illuminated by this light source, - the light sources are controlled according to a predetermined illumination sequence for each illuminated object in succession according to said different illumination wavelength ranges, - images are produced by at least one colour camera (4, 4a, 4b) sensitive to light radiation in the visible range in the wavelength band between 380 nm and 700 nm and to light radiation in the infrared range in the wavelength band between 700 nm and 1000 nm, named multispectral camera (4, 4a, 4b), said at least one multispectral camera being orientated towards an external surface portion of at least one illuminated object corresponding to the whole visible face of the external surface of the object on the optical axis of the multispectral camera, and the exposure of said at least one multispectral camera being controlled in synchronism with said illumination sequence so as to produce, with this same multispectral camera, a plurality of images in different imaging wavelength ranges of said external surface portion of at least one illuminated object (6), said plurality of images including at least one image in a visible range and at least one image in an infrared range, - each multispectral camera (4, 4a, 4b) is chosen from the group of cameras comprising a CMOS sensor with a matrix of colour filters without an infrared cut-off filter; and cameras comprising three CMOS sensors, one CMOS sensor for each primary colour, without an infrared cut-off filter.

2. - Method according to claim 1, characterised in that said plurality of images of said external surface portion produced by a single multispectral camera (4, 4a, 4b) is recorded in a buffer memory (42a, 42b) of this multispectral camera.

3. - Method according to any one of claims 1 or 2, characterised in that each image in an infrared range is produced with an adjustment of the white balance according to the sensitivity for each colour of said at least one multispectral camera (4, 4a, 4b) in said infrared range.

4. - Method according to any one of claims 1 to 3, characterised in that said illumination sequence is formed of a succession of illumination periods, some of said light sources being activated during each illumination period, this group of said light sources being chosen to illuminate each object in one of said illumination wavelength ranges, in that each illumination period is between 0.1 ms and 5 ms, and in that the exposure of said at least one multispectral camera (4, 4a, 4b) is trigged with a delay with respect to the activation of an illumination period, this delay being chosen so that each light source of said group of said light sources of this illumination period is effectively completely switched on and active before said at least one multispectral camera is triggered.

5. - Method according to claim 4, characterised in that for the production of reflection images the external surface region of the object illuminated by at least one light source is the whole of a face of the external surface of the object visible by the light radiation of this light source, and in that at least one multispectral camera is arranged with respect to this light source to produce reflection images of the whole of this visible face.

6. - Method according to any one of claims 1 to 5, characterised in that said plurality of images produced by a single multispectral camera comprises: - an image produced by reflection while the object is illuminated by white light, - an image produced by reflection while the object is illuminated in an infrared illumination range at a wavelength between 720 and 780 nm, - an image produced by reflection while the object is illuminated in an infrared illumination range at a wavelength between 800 nm and 850 nm, - an image produced by reflection while the object is illuminated in an infrared illumination range at a wavelength between 900 nm and 1000 nm, - an image produced by reflection while the object is illuminated in an ultraviolet illumination range at a wavelength between 250 nm and 380 nm.

7. - Method according to any one of claims 1 to 6, characterised in that each illuminated object (6) is rotated on itself and is moved by a conveyor during said illumination sequence.

8. - Device for optically analysing objects belonging to the fruit and vegetable group, comprising - means for producing images of the objects in different wavelength ranges, comprising: - an illumination device comprising a plurality of light sources formed of light-emitting diodes arranged to be able each to apply light radiation to at least an external surface portion of at least one object, named illuminated object, the different light sources being adapted to be able to apply light radiation in different illumination wavelength ranges selectively to each illuminated object, - the illumination device being adapted to be able to apply light radiation from at least one light source to the whole of a visible face of the external surface of each object illuminated by this light source, - a control device (10, 20, 43a, 43b) adapted to be able to control these light sources according to a predetermined sequence of illumination of each illuminated object in succession according to the different illumination wavelength ranges, - at least one colour camera sensitive to light radiation in the visible range and to light radiation in the infrared range, named multispectral camera (4, 4a, 4b), orientated towards an external surface portion of at least one illuminated object corresponding to the whole visible face of the external surface of the object on the optical axis of the multispectral camera, - each multispectral camera (4, 4a, 4b) being chosen from the group of cameras comprising a CMOS sensor with a matrix of colour filters without an infrared cut-off filter; and cameras comprising three CMOS sensors, one CMOS sensor for each primary colour, without an infrared cut-off filter, and in that said control device (10, 20, 43a, 43b) is adapted to control the exposure of each multispectral camera (4, 4a, 4b) in synchronism with said illumination sequence so as to produce, with this same multispectral camera, a plurality of images in different imaging wavelength ranges of said external surface portion of at least one illuminated object, said plurality of images including at least one image in a visible range and at least one image in an infrared range.

9. - Device according to claim 8, characterised in that said light sources are mounted in an optical chamber (2) having a reflective internal surface of a shape which is chosen depending on the position of the light sources so as to permit uniform illumination of the objects in the optical field of said at least one multispectral camera.

10. - Device according to any one of claims 8 or 9, characterised in that, for the production of each image in an infrared range, said control device (10, 20) is adapted to adjust the white balance according to the sensitivity of each colour of said at least one multispectral camera in said infrared range.

11. - Device according to any one of claims 8 to 10, characterised in that each multispectral camera (4, 4a, 4b) is a colour camera comprising a CMOS sensor and a colour filter matrix, without an infrared cut-off filter, and fitted with a buffer memory (42a, 42b) for storing the images.

12. - Device according to any one of claims 8 to 11, characterised in that said light sources comprise a least one visible white light illumination LED, at least one ultraviolet light radiation illumination LED and at least one infrared illumination LED.

13. - Device for automatically sorting objects belonging to the fruit and vegetable group according to predetermined sorting criteria comprising: - at least one conveying line (8) able to transport the objects in front of stations for analysing the objects in accordance with said sorting criteria, including at least one optical analysis station, - an automated machine (10) connected to the analysis stations to receive analysis signals therefrom, - stations for discharging the objects in a plurality of discharging regions, the automated machine being programmed to control the selective discharging of each object in a discharging region selected in accordance with the analysis signals received by this automated machine for this object, characterised in that it comprises at least one optical analysis station formed by an optical analysis device (3) according to any one of claims 8 to 12.

14. - Device according to claim 13, characterised in that each optical analysis station comprises less than four multispectral cameras (4a, 4b) per conveying line.

15. - Device according to any one of claims 13 or 14, characterised in that it comprises a single optical analysis station (3).