Method and device for inspecting containers from at least two different directions of observation for the purpose of classifying the containers

Abstract

Method and device for inspecting containers from at least two different directions of observation for the purpose of classifying containers. Method for inspecting containers made of transparent or translucent material (2) for the purpose of classifying a container, the method comprising: a use phase including: - the acquisition for each container of at least a first and a second image (Ic) of at least the same portion of a container from two different directions of observation and from at least one modality; - the provision as input to a deep learning model (NN), for each container, of a recording of at least the first and second images of at least one portion of the container from at least one modality and from two different directions of observation;- and the analysis by the deep learning model, for each container, of this record to determine the membership of this portion of the container, in a result class from a list of classes. Figure for the abbreviation: Fig. 1.;

Description

Title of the invention: Method and device for inspecting containers from at least two different observation directions with a view to classifying the containers Technical field

[0001] The present invention relates to the technical field of the inspection of transparent or translucent containers such as, for example, glass bottles, jars or flasks or preforms or bottles (including returnable containers) made of plastic material for the purpose of their quality control in order to detect and identify any defects likely to affect these containers.

[0002] The subject of the invention finds particularly advantageous applications for analyzing physical characteristics of containers in order to determine the presence or absence of defects and to identify conforming optical singularities such as decorations, functional reliefs, mold joints or non-conforming optical singularities corresponding to defects, such as for example surface defects, such as folds or crevices, internal defects in the material, such as cracks, inclusions, or bubbles, dimensional defects such as deformations. Prior art

[0003] In the field of manufacturing glass containers, it is known that the manufacturing process comprising the melting of the glass and then its transport to forming units is implemented by means of a manufacturing installation comprising a melting furnace, a feeder for supplying molten glass to a forming machine generally of the type designated by IS machine. The containers which have just been formed by the forming machine are placed successively on an output conveyor to form a row of containers. The containers are transported in a row by a conveyor in order to transport them successively to different processing stations. In particular, the formed containers are brought into an annealing furnace, which raises their temperature and then cools them in a controlled manner so that the thermal stresses created by the forming process disappear.Other glass container forming processes are known for tableware, insulators, syringes, and ampoules. For example, there are forming machines such as rotary and sequential presses, and not in parallel aligned sections like IS machines. There are also machines that transform preforms into tubes, particularly borosilicate glass, to make syringes and ampoules dedicated to pharmaceutical products.

[0004] It is known to systematically inspect all containers leaving the machine of forming, using different inspection equipment, either at an intermediate stage of their forming, or immediately after their forming, when they pass on the output conveyor, still hot (what is called hot inspection), or after their annealing in the annealing arch, in inspection equipment (what is called cold inspection).

[0005] For example, to detect glaze-type defects in containers moving in a translation direction, patent application WO2021 / 209704 describes an inspection station comprising at least six imagers (typically 6, 12, 18 or 24) forming images and having an optical axis directed towards the inside of the inspection zone, being mounted in such a way that their optical axes are distributed around the central axis of the containers by choosing their azimuth angles between 0 and 360° relative to the translation direction, so that all the points of the circumference of the edge of the containers are represented in at least one image acquired during the crossing of the inspection zone by the container edge. The containers are also illuminated by at least twelve projectors, each having a beam direction, tangent to a cylinder centered on the central axis of the container, and the illumination beam directions are distributed in azimuth.Such a device allows for multiple beam directions and multiple observation directions to ensure the detection of glazes that reflect incident light towards the imagers.

[0006] It is also known to systematically inspect all containers leaving the annealing furnace using different inspection equipment, in particular transmission wall inspection systems for which a light source is arranged on one side of the conveyor and at least one camera (typically 2 to 6, 12 or 24) is arranged on the other side to acquire at least one image formed by the light transmitted through the walls of the container. Patent application WO2023 / 052732 describes an inspection device provided on each side of the conveyor with a series of three cameras opposite which a light panel is arranged.

[0007] Also known from patent EP 1 109 008 is a method for analyzing images of containers for cold inspection. A segmentation step detects features in the images and regions around the features. Discriminant parameters of each region corresponding to a feature are calculated and combined with a fuzzy logic method to determine the most probable type of features from a list of features corresponding to possible defects. The conformity of the region is then decided, by applying different criteria depending on the type of feature selected. For example, a fold-type defect will be ejected for a certain surface area while an inclusion-type defect will be rejected even if it has a small surface area. This patent teaches in particular that not all defects have the same criticality, which justifies seeking to determine its nature before deciding to reject a container. In this method, the illumination is a homogeneous extended source, which correctly reveals absorbent defects, but in which refracting defects are visible but with low contrast and therefore with insufficient sensitivity.

[0008] However, the inventors of the present invention have found that the determination of the type of features with the method proposed in EP 1 109 008 is not reliable enough, and moreover there appears to be a difficulty in reliably identifying certain defects and in particular "appearance defects", namely visual defects of any type: inclusions (of foreign bodies such as ceramics, metal), bubbles, folds, rivers (surface grooves), glazes (cracks), fins, trapezoids, grease stains, very thin areas, unmelted. These appearance defects are manifested in the images by local optical variations, or pixels having deviations from the background. These appearance defects can be critical if they lead to a risk for the consumer, a risk of breakage or a loss of functionality of the container.Since the recognition of an appearance defect from an image can be ambiguous, safety margins are taken during detection. It follows that containers are considered defective even though these containers are acceptable or compliant.

[0009] Furthermore, it should be noted that acceptable optical singularities such as engravings or decorations, weakly marked mold joints can be distinguished in the images. Also, there appears to be a need to identify exactly the nature of the appearance defects in order to identify critical defects by distinguishing them from other defects. The prevention of critical defects requires improving the reliability of classification of appearance defects. In addition to the fact that improving the identification of defects improves production efficiency, this identification of defects makes it possible to determine their potential causes so that the manufacturing installation can be controlled according to the category of detected defects. Indeed, without reliable identification of the detected defects, no safe decision to correct the manufacturing process can be taken, whether manually or automatically.

[0010] Patent application WO2021 / 213864 proposes a method for more reliably inspecting containers transported by a conveyor on a line, in particular a bottling line. The containers are transported to at least a first inspection unit and a second inspection unit, each comprising a transmitter and a receiver. The inspection units can inspect the containers with, as transmitter, white light, laser light, high-frequency electromagnetic waves, gamma rays and / or X-rays. The containers are transported between the transmitter and a receiver such as a camera making it possible to acquire, with the first inspection unit, first measurement data and with the second inspection unit, second measurement data. According to this method, the first measurement data and the second measurement data are combined to form common input data for an evaluation unit based on artificial intelligence and providing as output, an inspection result, such as a filling level.

[0011] As a single example of embodiment, this document describes the detection of the filling level of containers by combining X-ray imaging and inspection with an infrared light source. This document specifies that such a method can be used to also check the side wall, the bottom, the mouth, the contents of the container such as for example contamination by foreign bodies or product residues. The result of the inspection can also be defects, such as damage to the containers, in particular cracks and / or glass shards.

[0012] However, patent application WO2021 / 213864 does not provide any teaching for detecting or identifying with certainty, in particular, appearance defects. Independently of the obligation to detect these defects, there appears to be a need to identify exactly the nature of the appearance defects in order to identify critical appearance defects compared to other defects that may be considered non-critical.An incorrect identification of a type of defect on a container can lead to the rejection of this container when a correct identification of this defect would have allowed the container to be identified as good, or to the non-rejection of a critical defect generating a serious non-quality, or to an inappropriate correction of the manufacturing process which can go as far as destabilizing it, or even to the failure to take into account a process error because if a system generates false alarms too frequently, the operator may end up no longer taking into account correctly the indications of the inspection system.

[0013] Also known from patent application EP 3 679 356 is a container inspection device for increasing detection reliability, in particular to reliably distinguish decorative elements from contamination or dirt. The device comprises a light source emitting radiation with different wavelength and intensity ranges in spatially separated areas. This light source illuminates the container to be examined and a color camera is configured to detect the radiation emitted by the source and having passed through the container.

[0014] The device also comprises an evaluation device which is designed to analyze the intensity image to determine pixels or regions which have an intensity different from that of their neighborhood to deduce the presence of a light-absorbing defect such as soiling. The evaluation device makes it possible to analyze the color images to determine pixels or regions which have a different color than their surroundings to deduce the presence of light-refracting elements such as decorative elements. Thus, if a brightness contrast is observed locally and at the same time there is no color contrast in that area, the presence of contamination in that area is detected by the evaluation unit. If a local brightness contrast coincides with a local color contrast, the evaluation unit detects the presence of a decorative element.

[0015] The evaluation unit can also identify structures that cause local color contrast but virtually no local brightness contrast or only low local brightness contrast. For example, chips in glass or water droplets can cause such local color contrast, while light shining through can radiate through these areas substantially without loss of brightness.

[0016] Such an inspection device makes it possible to distinguish decorative elements from dirt or contamination. In other words, light-refracting defects are distinguished from light-absorbing defects. However, this inspection device does not make it possible to distinguish between light-refracting defects, as indicated by this application, in particular with regard to glass inclusions and water droplets.

[0017] From the state of the prior art, we also know the document JP 4886830 which describes the inspection of a rotating bottle to reveal a crack. This document proposes to process the images, but remains silent on the image processing methods to be used. The document WO 2018 / 061196 has a teaching close to this document, and it has the same shortcomings.

[0018] Document EP 3180135 is also known, which describes an acquisition of images in stereoscopy to remove the parts of images visible in the two stereoscopic images.

[0019] Finally, the prior art document EP 4078156 is known, which describes the acquisition of several images of a container (from a high angle and from a low angle). Processing of the individual images is disclosed.

[0020] It is apparent from the technical inspection solutions of the prior art that there is a need to make the classification of optical singularities appearing in the images more reliable so as to detect whether or not the container has a defect and to recognize these optical singularities in order to be able to apply a specific acceptance criterion for each type of optical singularity detected. The recognition of these optical singularities improves production efficiency by optimizing the detection ratio over false rejections. This identification makes it possible to determine their potential causes so that the manufacturing installation can be controlled according to the category of defects detected. Statement of the invention

[0021] The present invention aims to remedy the drawbacks of the prior art by proposing a method for controlling the quality of containers, designed to achieve more effective detection of defects by ensuring their identification in a more reliable and certain manner in order to optimize the sorting or classification of containers.

[0022] An object of the invention is to propose a control method making it possible to classify each container according to at least one class taken from a list of classes comprising in particular a class of absence of defect, a class of presence of defect, a class relating to a normal optical singularity and / or a class relating to an abnormal optical singularity.

[0023] To achieve such objectives, the subject of the invention relates to a method for inspecting containers made of transparent or translucent material (typically glass or plastic, the person skilled in the art being able to identify the containers considered to be transparent or translucent) with a view to classifying a container, the method comprising; a usage phase including: - the acquisition for each container, of at least a first and a second image of at least one same portion of a container according to two different observation directions and according to at least one modality; - providing as input to a deep learning model, for each container, a recording of at least the first and second images of at least one portion of the container according to at least one modality and according to two different observation directions; - and the analysis by the deep learning model, for each container, of this recording to determine the belonging of this portion of container, to a result class among a list of classes.

[0024] Thus, a single deep learning model is here configured to receive at least two images as input and deliver a class membership for the container, based on the two images. The deep learning model may for example comprise a number of inputs chosen to receive at least two images as input (or processing results of these two images if processing or pre-processing is carried out on the images).

[0025] In the method, the images are images of at least one portion of a container. Thus, it may be an image acquired by a sensor, a portion of an image acquired by a sensor, an image of an entire container, an image of a part of a container. The at least two images target the same portion.

[0026] The deep learning model can be an artificial neural network (typically a neural network with convolution layers) or a transformer-type model.

[0027] For example, for an artificial neural network, the number of input neurons can correspond to the cumulative number of pixels in the two images.

[0028] In other words, multi-view data fusion is implemented before classification.

[0029] According to the invention, several images of a portion of a container are produced with different observation directions in order to improve the recognition of the content of the images with a view to recognizing optical singularities. Indeed, the inventors have noticed that certain defects seen from a particular angle have a signature which does not allow the type to be determined with certainty. For example, the shape of the image of the defect under a single observation direction is not always sufficiently characteristic, due to different optical phenomena. Typically, the defect is not entirely visible in any image and / or the defect is deformed by refraction of light through the wall of the container and / or its appearance is different in photometry and geometry according to different observation directions.

[0030] When a defect is observed from different directions, its image is transformed from one view to another, on the one hand because these two views are two different perspective projections of the same object, but also because the light passes through it along different paths for the different views, undergoing different refractions or reflections, which produces contrasts and therefore a photometry of different appearance. It has been observed that the deformation of a defect between two different views is a characteristic of each defect, of its geometry and its apparent photometry. In other words, taking into account the way in which the image of a defect varies when its direction of observation varies makes it possible to identify the type of defect more precisely.

[0031] It should also be noted that when a defect is observed from different observation directions, it should be considered that it is the same defect so that it is wrong to consider that the container has two defects. However, generally according to the prior art, each view is treated separately. This is sufficient for a simple counting of defects without discrimination, but this technique is not suitable if one wishes to count the types of defect separately. Indeed, the classification systems can wrongly attribute a different type to the two observations of the same defect and there is no suitable solution to favor the diagnosis according to one or other of the observations. This is the reason why performing a fusion of multi-view data makes it possible to obtain a more reliable verdict, directly taking into account several viewing angles for the same defect to be categorized.

[0032] It may be noted that the deep learning model may be preceded by one or more processing modules individually processing each image, the results of these processings then being provided as input to the deep learning model.

[0033] For information purposes, these treatments can be implemented themselves by learning models, or by conventional models which are not learning models. That being said, classification, that is to say the determination of membership in a class by analysis, takes into account elements coming from each image, jointly.

[0034] In other words, there is a joint processing of the images which have possibly undergone processing (which can be called pre-processing).

[0035] According to a particular mode of implementation, the method comprises: - the provision of an inspection system for acquiring images according to at least a first modality and a second modality; - the acquisition for each container, of a recording of at least one same portion of a container comprising at least two images according to at least two different observation directions according to the first modality and at least one image according to the second modality.

[0036] According to a particular embodiment, the method comprises the provision of an inspection system configured to acquire images of the same portion of a container according to at least two different observation directions and according to at least one modality taken from the list of the following modalities: absorption, birefringence, refraction, reflection, infrared radiation.

[0037] According to a particular embodiment, each portion of container is classified according to at least one class taken from a list of classes comprising at least one class of absence of defect in the portion and one class of presence of defect in the portion.

[0038] According to a particular embodiment, each portion of container is classified according to at least one class taken from a list of classes comprising at least one class including the presence in the portion of at least defects such as, in particular, trapezium, inclusion, bubble.

[0039] According to a particular embodiment, each portion of container is classified according to at least one class taken from a list of classes comprising at least one class including the presence in the portion of at least one singularity such as a marking, a mold seal, a notch, a thread, an impression, a stitch, a handle, a counter ring.

[0040] According to a particular embodiment, the method comprises the acquisition for each container having a central axis, of at least four images of at least one same portion of a container according to four different observation directions and according to at least the first modality, the observation directions being distributed around the central axis two by two according to an azimuth angle of at least 45°.

[0041] According to a particular mode of implementation, at least one sorting characteristic is compared to a rejection criterion, the sorting characteristic and the rejection criterion being dependent on the membership class to decide whether or not the container is compliant, the sorting characteristic being calculated on at least one image of the container according to a modality.

[0042] According to a particular mode of implementation, a step is implemented for taking into account at least one identified defect in order to deduce adjustment information for at least one control parameter of a container manufacturing installation.

[0043] According to a particular mode of implementation: -the deep learning model associates a confidence score with the ranking of containers that are part of an inspected production; - the classification of containers is taken into account only when the confidence score exceeds a confidence threshold for; - count defects by defect class; -and / or decide to reject the container; -and / or trigger an alarm for the presence of at least one critical defect in the inspected production.

[0044] Accounting may involve the implementation of time statistics (defect frequencies). For example, time statistics may be provided for a sliding window in order to determine trends (emergence / appearance of defects, drift in the process, etc.).

[0045] The invention also proposes a method for training a deep learning model for inspecting containers made of transparent or translucent material with a view to classifying a container, the method comprising a construction phase comprising: - provision of a learning set comprising recordings each composed of at least a first and a second image of the same portion of container according to two different observation directions and according to at least one modality; - provision of at least one deep neural learning model having been trained on a training set comprising recordings each composed of at least a first and a second image of the same portion of container according to two different observation directions and according to at least one modality, the deep learning model determining at least one membership class for said portion from a list of classes.

[0046] This learning method can be configured to obtain models according to all the methods of implementing the inspection process defined above.

[0047] According to a particular mode of implementation, the method comprises providing at least one deep learning model having been trained on a training set comprising recordings each composed of at least two images of the same portion of container according to observation directions different by at most 5° during the acquisition of the images.

[0048] According to a particular embodiment, the method comprises providing at least one deep learning model having been trained on a training set comprising recordings each composed of images of the same portion of container according to different directions and modalities.

[0049] The invention also proposes a method comprising a phase of construction of the learning method defined above to obtain a deep learning model, and a phase of inspection of the inspection method defined above using the deep learning model of said construction phase.

[0050] According to a particular embodiment, during the construction phase, the image records are ordered according to a determined sequence while during the use phase, the image records are ordered according to a sequence identical to the sequence of the construction phase.

[0051] In fact, the images are ordered according to, for example, an observation direction and a modality specific to each image, in an identical manner within each recording. A recording therefore comprises a set of images for a container, captured by a device similar to the inspection device.

[0052] Also, by sequence, we mean an ordered series of elements, the order being able to be fixed by the directions and the modalities.

[0053] The invention also proposes a device for inspecting containers made of transparent or translucent material leaving a manufacturing or recovery installation with a view to classifying the containers in relation to defects, the device comprising: - an inspection system comprising at least one camera arranged to recover light or radiation (for example infrared) coming from at least one portion of a container and configured to acquire images of the same portion of a container according to at least two different observation directions and according to at least one first modality; - an information processing unit connected to the inspection system and comprising a deep learning model, the deep learning model determining at least one membership class for said portion from a list of classes, the deep learning model receiving as input, for each container, a recording of at least two images of at least one portion of the container according to the first modality and according to two different observation directions, the model deep learning, for each container, analyzing this recording to determine the membership of this portion of container, to a result class among the list of classes.

[0054] This device can be configured to implement the inspection method such as as defined above.

[0055] The invention also proposes a device configured for implementing the learning method defined above.

[0056] The invention also proposes a deep learning model obtained by the learning method defined above.

[0057] Various other characteristics emerge from the description given below with reference to the appended drawings which show, by way of non-limiting examples, embodiments of the subject of the invention. Brief description of the drawings

[0058] [Fig-1] [Fig.l] represents an example of the realization of an installation of fa manufacturing of containers inspected by an inspection device according to the invention.

[0059] [Fig.2] [Fig.2] is a schematic view illustrating an exemplary embodiment of a inspection device according to the invention using two cameras adapted, for example, to simultaneously take two images of a container from two different observation directions.

[0060] [Fig.3A] [Fig.3A] is a schematic top view showing the taking of successive images, by a camera, under a first direction of observation of a container moving in translation.

[0061] [Fig.3B] [Fig.3B] is a schematic top view showing the taking of images, by a camera, from a second observation direction of a container moved in translation relative to its position illustrated in [Fig.3A].

[0062] [Fig.3C] [Fig.3C] is a schematic top view showing the taking of images, by a camera, from a third direction of observation of a container moved in translation relative to its position illustrated in [Fig.3B].

[0063] [Fig.4] [Fig.4] is a schematic view illustrating an exemplary embodiment of a inspection device according to the invention using two cameras adapted to obtain from each three analysis images according to three different methods, in order to constitute a recording of analysis images comprising at least 6 container images.

[0064] [Fig.5] [Fig.5] is a schematic view illustrating yet another example of a rea lisation of an inspection device according to the invention adapted to take simultaneously, three images of a container from three different observation directions and analyzed by four neural networks which collaborate to classify the container.

[0065] [Fig.6] [Fig.6] is a schematic view illustrating yet another example of a rea lisation of an inspection device in accordance with the invention implementing two cameras adapted to simultaneously take two images of a container from two different observation directions and analyzed by algorithms for detecting image singularities and then by a neural network which classifies the container.

[0066] [Fig.7A] [Fig.7A] shows an arrangement of a plurality of cameras arranged at different elevations.

[0067] [Fig.7B] [Fig.7B] corresponds to the arrangement of [Fig.7A] but where the different azimuths are visible.

[0068] [Fig.8A] [Fig.8A] is a photograph of a container on which a defect is visible.

[0069] [Fig.8B] [Fig.8B] is a photograph of the container of [Fig.8A] in which the defect is no longer visible.

[0070] [Fig.9A] [Fig.9A] shows a photograph of another container on which a defect is visible.

[0071] [Fig.9B] [Fig.9B] is a photograph of the container of [Fig.9A] in which the defect is still visible, but with a different appearance.

[0072] [Fig.9C] [Fig.9C] shows how the cameras used to obtain the images in Figures 9A and 9C are arranged.

[0073] [Fig. 10A] [Fig. 10A] is a photograph of yet another container on which a two-part defect is visible.

[0074] [Fig.lOB] [Fig.lOB] is a photograph of the container of [Fig.lOA] on which the defect remains visible with a different appearance. Description of the embodiments

[0075] In the present description, artificial neural networks are used as a deep learning model. The invention is nevertheless in no way limited to these neural networks and can be adapted to the use of transformers, or even to other learning models.

[0076] Figures 1 and 2 illustrate a device 1 according to the invention for inspecting containers 2 leaving an installation 3 of any type known per se. The installation 3 provides transparent or translucent containers 2 such as, for example, bottles, pots, flasks, syringes, ampoules or preforms. Generally, a container 2 has a central axis R, considered as an axis of symmetry, or even an axis of symmetry of revolution. These containers 2 can be made of different materials such as glass, plastic or renewable raw material such as wheat, sugar cane or corn, for example. example. These containers 2 can be filled or empty. The installation 3 thus ensures the manufacture of the containers 2 or even their decoration or covering, filling and closing. It should be noted that the installation 3 is capable of manufacturing the new containers 2 from raw or recycled materials, or of reconditioning recovered containers. According to a preferred example of implementation, the installation 3 is a forming installation from which empty glass containers emerge.

[0077] Conventionally, the installation 3 comprises a production computer 4 for supervising the various functionalities of the installation 3 during the manufacture of the containers. The production computer 4 is typically for a glass container forming machine, a sequencer which controls pneumatic or motorized actuators as well as valves controlling the circulation of the cooling air or the blowing pressure for the manufacturing molds.

[0078] At the outlet of the installation 3, the containers 2 are taken over by an outlet conveyor 5 to form a line of containers, being in the example illustrated, placed successively on the outlet conveyor. The containers 2 are transported in line by the conveyor 5 in a direction of movement F in order to convey them successively to different treatment and / or control stations and in particular to an annealing arch 6 (i.e. a treatment station) and to the inspection device 1 according to the invention (i.e. a control station). The direction of movement F of the containers 1 is established along a rectilinear trajectory with a horizontal axis X of a direct orthonormal reference frame X, Y, Z comprising a vertical axis Z perpendicular to the horizontal axis X and a transverse axis Y perpendicular to the vertical axis Z and to the horizontal axis X, and the axes X and Y being in a plane parallel to a conveying plane Pc of the containers which is considered to be horizontal.The inspection device 1 according to the invention can also, in a variant, be installed downstream of the installation 3 and upstream of the annealing arch 6, when the hot containers 2' are transported in line by the conveyor 5' towards the arch. In other words, the inspection device 1 according to the invention can be installed upstream or downstream of any treatment of formed containers and transported according to the movement F. .

[0079] The inspection device 1 according to the invention aims to implement a method for detecting, for each container 2 moving in translation, whether the container has a defect and to identify, for a container having a defect, a type of defect from a family of possible defects.

[0080] The inspection device 1 according to the invention comprises an inspection system 7 visible in [Fig. 2] and comprising at least one camera Ci (Cl, C2,.. .Ci,.. .Cn, with i ranging from 1 to n) arranged to recover light or radiation (for example infrared) coming from at least one portion of a container 2 and configured to acquire images of the same portion of a container according to at least two different observation directions D1, D2 and according to at least one first inspection method.

[0081] Each camera Ci comprises, in a conventional manner (Figures 3A to 3C), an optical objective B having an optical center O and an optical axis A, allowing the formation of an optical image on a photoelectric sensor E, linear or matrix, generally flat, positioned in the focal plane of the objective. The images acquired by the cameras Ci are transmitted to an electronic information processing unit 9 forming part of the inspection device 1, but which may possibly be remote. This electronic information processing unit 9 is a computer system of all types comprising computers, external peripherals (display unit, storage unit, keyboards, connection to different factory networks, connection to cameras, etc.), equipped with programs implementing in particular image processing algorithms, databases, etc.

[0082] This information processing unit 9 is connected to the production computer 5 in order to receive, if necessary, from the production computer (or even from other peripherals), manufacturing information for association with the containers 2, their images and their detected defects with this manufacturing data. The manufacturing information may be time information, mold or molding cavity numbers, etc. In the case where the container is a glass container, received time information makes it possible to associate the containers 2, their images and their detected defects, with the mold number or the forming cavity or with a time stamp or an individual identifier. Typically, the operation of the inspection system 7 is synchronized with the operation of the forming cavities of the containers, in particular in the case where it is installed between the forming installation 3 and a processing station such as the annealing arch 6..

[0083] Advantageously, this information processing unit 9 transmits to the production computer 4 the identified defects and the measurements carried out, so that the production computer can automatically deduce adjustment information for at least one control parameter of the installation 3 or processing 6. Such an adjustment of the control parameters is carried out manually or automatically. Finally, the information processing unit 9 is connected to an ejector to control the ejection of containers identified as defective, and / or to a display unit to present to an operator the identified defects and the images of the containers.

[0084] According to the invention, the inspection system 7 is configured to recover, by at least one camera, the light coming from the container 2 in order to acquire images of the same portion of a container according to at least two different observation directions D1, D2, D3,...Dj and according to at least one inspection method. Such that As shown in [Fig.3A] to [Fig.3C], each direction of observation D1, D2, D3 corresponds to the straight line passing through the optical center O of the camera and the center T of the portion P of the container 2 placed in the field of observation of the camera.

[0085] According to a first embodiment illustrated in Figures 3A to 3C, the different observation directions D1, D2, D3 are produced for different positions of the container in the field of observation of the camera. According to this embodiment, the containers 2 are moved along a rectilinear trajectory represented by the arrow F so as to pass in front of a fixed camera C1 positioned to inspect the body of a container. According to this example, the center T of the portion P of the container 2 observed by the camera corresponds to the central axis R of the container. As illustrated in [Fig.3A], the camera C1 takes an image ICI 1 of the container 2 when the observation direction D1 forms an angle alpha with the optical axis A. As is apparent from [Fig.3B], the movement of the container leads to its positioning in a position for which the observation direction D2 corresponds to the optical axis A.The camera Cl takes an image IC12 of the container 2 when the direction of observation D2 coincides with the optical axis A. The continued movement of the container leads to its positioning in a position for which the direction of observation D3 has exceeded the optical axis A and forms an angle alpha with the optical axis A ([Fig.3C]). The camera Cl takes an image IC 13 of the container 2 when the direction of observation D3 forms an angle alpha with the optical axis A.

[0086] In the embodiment illustrated in Figures 3A to 3C, the direction of observation is modified by the relative movement between the container 3 and the camera. According to this example, the camera is fixed while the container is mobile. Of course, the subject of the invention also applies to a fixed container and a mobile camera.

[0087] According to a second embodiment illustrated in [Fig. 2], the different observation directions D1, D2 are produced by cameras having observation directions of the container which are different at the time of image acquisition. According to this example, two cameras C1, C2 are positioned so that in a vertical plane, parallel to the central axis R, the optical axes A of the cameras form between them a determined elevation angle alpha so that if the observation directions correspond to the optical axes, the observation directions are offset by an angle alpha.Of course, it may be envisaged to distribute the cameras in the azimuth plane (plane parallel to the conveying plane, as will be visible with reference to figures 7A and 7B described below) in such a way that the optical axes A of the cameras form between them a determined azimuth angle alpha so that if the observation directions correspond to the optical axes, the observation directions are offset by an angle alpha.

[0088] Advantageously, two observation directions D1, D2, D3...Di have di different observation directions if these two observation directions are offset by at least 5°.

[0089] Whatever the inspection system 7 implemented, this inspection system provides the information processing unit 9 with images of the same portion of a container according to at least two different observation directions and according to at least one first inspection method. By the same portion, it is meant that an overlap of the views is possible between the images.

[0090] An inspection modality corresponds to a type of interaction of the light with the wall of the containers and with the defects to be identified. Typically, the inspection system 7 is configured to acquire images of the same portion of a container according to at least two different observation directions and according to at least one modality taken from the list of the following modalities: absorption, birefringence, refraction, reflection, infrared radiation.

[0091] For example, one inspection method is called absorption. This method mainly highlights the absorption of the light passing through the wall of the container through which it passes, but also refraction effects such as shadows on the edges of the container or refracting defects or thickness variations. Some defects have an absorbent character, totally or partially absorbing. These defects thus appear opaque or dark when seen in transmission, that is to say that the light passing through a glass wall without defects undergoes a so-called normal absorption corresponding to the tint and thickness of the material constituting the container, assumed to be homogeneous for the glass wall. But the absorbent defects present a local anomaly with an absorption sometimes lower (bubble or thin) but generally higher than the normal absorption.In the following, absorption will only refer to the abnormal absorption of absorbent defects. Such defects include inclusions in the glass, especially ceramic or metal, and / or dirt (grease, etc.) on the glass. But such defects also include certain glazes (cracks) that would be oriented in the glass in such a way as to block the inspection light, mainly because the inspection light is then reflected in a direction that is not seen by the camera. Due to the limitation in size of the light source, shadows appear in the image due to refraction on the outer edges of the container silhouette. Shadows do not generally reveal defects. Some shadow shapes reveal glass distribution defects. Some refracting defects have a particular signature at the edge of the container silhouette and another signature in the center.They will therefore be distinguished by observation from different angles of view.

[0092] Another inspection modality is called birefringence. This modality mainly implements a modification of the polarization state of the light passing through the wall of the container by a so-called stress defect, which gives the glass a birefringence property. Some defects have a birefringent character. Thus, certain defects result in the presence of residual mechanical stresses in the material (sometimes called internal mechanical stresses or, in particular in English, "stress"). In the wall, a birefringent or stress defect, such as an inclusion of foreign bodies (ceramic, metal, "devitrified glass") causes a modification of the polarization state, that is to say a polarization phase shift between two components of the electric field or a modification of the direction of linearly repolarized light.

[0093] Another inspection method is called refraction. This method mainly implements a modification of the direction of propagation of the light passing through the wall of the container by a refracting defect, due to an angle between the dioptric surfaces crossed and / or a difference in refractive index. Each surface is an air / glass or glass / air interface, therefore a dioptre which refracts the light passing through it. In the absence of a defect, the surfaces of the walls are substantially parallel and the refraction does not cause any visible deviation of the light rays passing through the container. A so-called refracting defect is a defect which locally causes abnormal refraction, mainly when the defect manifests itself by differences in slope between surfaces or dioptres crossed by the wall(s).We will therefore speak of refractive defects only to designate deviations of light by the particular refraction at the level of so-called refractive defects. Refractive defects are defects that are mainly detectable by the refractive anomalies that they generate, in particular in a through-light inspection. Typically, surface defects (folds, rivers,) or glass distribution defects (bubbles, thin, compression ring), trapezoids and fins are generally classified among refractive defects. It should be noted that trapezoids and fins generally cause such strong refractions that these defects are generally clearly visible also in absorption images.

[0094] Another inspection method is called transmission-reflection. This involves in particular the detection of glazes in glass containers. Glazes are very narrow cracks, of different shapes and lengths and different orientations in the material. Glazes behave like diopters reflecting light. Their detection consists of illuminating a part of the container according to one or more directional beams having angles of incidence specific to the illuminated portion of the container. The direction of the light reflected by a glaze is specific to the shape and orientation of the glaze, and the image by a camera receiving or not the reflected light allows the detection of glazes. In other words, the direction of observation is a characteristic of the glaze. According to this method and in In the example of patent WO 2021 / 209704, a large number of cameras are arranged around the container with observation directions distributed in azimuth and elevation.

[0095] Finally, the invention is not limited to these modalities. It can be applied, for example, in the case of a pure reflection modality, according to which a light source is designed to illuminate the surface of a portion of the container, the surface reflecting the light in the direction of the camera. The image is then mainly constituted by the reflected light, and we observe as potential defects, either deformations of the geometry of the surface, or of its reflectivity, or the appearance of a reflection outside the normal contour of the surface.

[0096] Another inspection method is called infrared radiation. This inspection method, without using a light source to illuminate the containers, aims to inspect the still-hot containers, typically made of glass, at the end of their manufacture and emitting, taking into account their temperature, infrared radiation depending on the volume (in other words, the thickness) and / or the temperature of the material constituting the containers. So-called infrared cameras are used, equipped with image sensors sensitive to the infrared radiation emitted by the still-hot containers, the temperature of which is greater than 300°C.The sensors used in this type of modality have a length sensitivity spectrum adapted according to the cases of use, for example in near infrared (SWIR or NIR) and / or mid infrared (MWIR). A defect modifying the emissivity, therefore the image of the infrared radiation of a container, can be linked to an accumulation of material at a specific location corresponding for example to an excess thickness of the wall or to a trapezoid or swing defect (defect of a glass wire inside the container and connected by its ends to the inner wall). According to the invention, the observation of the infrared radiation of the hot containers according to at least two different observation directions allows for example that the neural network (or the model used) takes into account the directional emissivity and / or the depth of the defect to determine the class to which it belongs.

[0097] In a known manner, the inspection of the containers 2 according to one and / or other of these inspection methods can be carried out using various configurations of the inspection system 7. It is considered that the inspection system is configured to acquire images in order to obtain images according to at least one modality and advantageously images relating to several modalities. It should be noted that depending on the inspection system 7 used, the images according to these modalities can be obtained directly from the acquired images or from calculations or processing.

[0098] The remainder of the description describes, by way of non-limiting examples, various methods for obtaining absorption images, birefringence images, refraction images and infrared radiation images.

[0099] To obtain absorption images, a first simple method is to produce on the light source a uniform intensity and unpolarized illumination in an active portion. A second solution for obtaining the absorption image is to use a light source uniform in intensity linearly or circularly polarized and to produce the image by means of a camera without any polarizing filter between the container and the camera. The intensity uniformity of the light source can be perfect, that is to say that the intensity is constant over the entire active zone of a flat and extended light source emitting diffuse light. The uniformity can also be local, in particular when the glass of the containers is tinted, it can be provided that a region of the source illuminating the neck where the glass wall is thicker, emits a stronger uniform intensity, while a region illuminating the body of the container where the wall is thinner emits a weaker uniform intensity.The intensity of a light source which generates, as described in document FR 2794241, a continuous spatial variation of intensity slower than that observed in the vicinity of a defect is also considered to be uniform or relatively uniform, so that an image analysis algorithm which compares the pixels to their neighbors to detect rapid local variations in intensity as defects does not detect the slow variations in intensity emitted by the source.

[0100] Another solution for obtaining the absorption image is described in patent application EP 3 679 356, which proposes producing an illumination with a source whose color varies spatially and acquiring a composite image (RGB) which is transformed into the HSV (Hue, Saturation, Value) color representation space. The absorption image is obtained by the V image or a transformation of this image, for example by means of a gradient calculation. It should be noted that a refraction image is obtained by the H image, or a transformation of this image, for example by means of a hue gradient calculation.

[0101] Another solution for obtaining the absorption image is to use a source of uniform light in monochrome intensity linearly or circularly polarized and to produce a composite polarimetric image by means of a polarimetric camera, and to calculate the absorption image from at least two partial polarimetric images corresponding to observations through two linear filters of analysis directions at 90° to each other.

[0102] Another solution for obtaining an absorption image is to produce the illumination using a source having variations in a polarization characteristic with the intensity remaining relatively uniform and to acquire a composite image containing at least 2 to 4 partial images using a polarimetric camera and to calculate an absorption image from 2 to 4 partial images taken through polarization analyzers at 90° to each other.

[0103] To obtain birefringence images, a first simple method is to produce on the light source, a uniform monochrome illumination linearly polarized in a determined direction, in an active portion. The image is acquired either by a black and white camera in front of which is placed a linear polarization analyzer whose direction is orthogonal to the determined direction, or by a polarimetric camera in which the pixels of the partial image corresponding to the analysis are taken into account through a linear filter orthogonal to the determined direction. The value of the pixels of the birefringence image is then almost zero except in the presence of a stress defect. When the light passes through a stress defect, the measured / received light intensity obtained depends on the direction and intensity of the stresses.

[0104] A second method for obtaining a birefringence image is to produce on the light source, a uniform monochrome illumination circularly polarized in a given direction in an active portion. The image is acquired with a black and white camera in front of which is placed a % wave delay plate then a linear polarization analyzer. The value of the pixels of the birefringence image is then almost zero except in the presence of a stress defect. When the light passes through a stress defect, the outgoing light intensity depends on the intensity of the stresses but not on the direction of the stresses.

[0105] A third method of obtaining a birefringence image is to produce on the light source, a uniform monochrome illumination polarized linearly in a single direction or circularly in a single sense in an active portion. The image is acquired by a polarimetric camera in front of which is possibly placed a % wave delay plate delivering a composite image. From two or four partial images, a birefringence quantity is calculated which depends on the polarization phase shift between the Ex and Ey components of the electric field and which is a measure of the stress. A person skilled in the art will be able to find the calculation formulas from the Mallus equation and the Stockes formalism.The polarization phase shift can be calculated to obtain as values ​​of the pixels in the birefringence image, a value that depends on the intensity of the stresses but preferably not on the direction of the stresses, the detection is therefore isotropic and proportional to the stresses. According to one of these variants, the polarization phase shift can be measured between 0 and 90° or even between 0 and 180°. This method also makes it possible to calculate an absorption image with the composite image delivered by the same polarimetric camera, by calculating each pixel as explained previously.

[0106] To obtain refraction images, a first method of obtaining a refraction image is illustrated in patent US4606634, which describes a type of refracting defect detection which consists of modifying the "angular spectrum" of an extended light source. A light source of variable dimension, for example a luminous disc of variable diameter, is at the focus of a converging projection lens. In the image obtained using a camera which receives the light having passed through the container, a reinforced contrast is obtained on the refracting objects, this contrast being able to be increased by reducing the angular spectrum, which is produced by reducing the diameter of the luminous disc.

[0107] A second set of methods for obtaining a refraction image consists of varying spatially along the emitting surface of an extended light source, such as a light panel, a property of the light emitted by the source that the camera can distinguish. As a property of light that varies spatially, its intensity was initially used. At least one image is acquired with an intensity-sensitive camera, therefore a priori monochrome, delivering monochrome images. The intensity of light emitted by each unit of light-emitting surface varies spatially according to a law of spatial variation in one or two dimensions.In other words, these inspection methods adapted for the detection of refracting defects use illumination devices which provide light which is sometimes called "structured", that is to say having an emitting surface, generally two-dimensional, which presents variations or patterns of intensity.

[0108] A third set of methods for obtaining a refraction image consists of varying spatially along the emitting surface, as a property of the light emitted by the source, a polarization property, a color property and / or a phase property. By way of example, documents WO2020 / 244815 and EP3679356 illustrate variants of this third set of methods for obtaining a refraction image.

[0109] To obtain images of infrared radiation, one solution is to use a camera sensitive to the infrared radiation emitted by the containers 2. Typically, the sensor of the camera sensitive to infrared radiation captures infrared radiation in a wavelength range greater than 0.8 pm.

[0110] It is clear from the examples given above that the inspection method using the inspection system 7 associated with the electronic information processing unit 9 makes it possible to obtain images according to different observation directions and at least one inspection method or several inspection methods.

[0111] According to an alternative embodiment, the inspection system 7 is configured to obtain images according to two different inspection modalities. According to a preferred alternative embodiment, the inspection system 7 is configured to obtain images according to three different inspection modalities. Thus, the method according to the invention aims to inspect the containers 2 using the inspection system 7 configured to acquire absorption, refraction and birefringence images, making it possible to recognize defects characterized by all 3 types of interaction with light.

[0112] The invention is also advantageous for an inspection system for detecting glazes using a device as described in EP4136434, a certain number of cameras observing the same portion of containers under different observation directions. In this case, the variations in the shape and the position of the reflections corresponding to the glazes between the different images mainly make it possible to distinguish them from parasitic reflections, and a learning classification model allows effective detection / classification while it is difficult to define a priori by professional knowledge, the geometric characteristics of these very random shape defects.

[0113] The objective of the combination of these various lighting and image acquisition techniques is to obtain from each inspected region of each container, at least two and preferably at least three images each according to a different modality and different directions, with for each modality, the value of each pixel which depends on a different modality of interaction of the light with the wall crossed and with the defects to be detected. Of course, the configuration of the lighting and the cameras depends on the inspected region of the container which can correspond to the body, the bottom, the neck, the shoulder, the rim, the ring or to an area where engravings are present for example.

[0114] The images acquired from the same portion of a container according to at least two different observation directions and according to at least one first modality are made available to the information processing unit 9. Of course, it is possible to prepare the images by pre-processing, such as: - geometric transformations (i.e. perspective correction); - transformations on the image format (number of channels, image size) - photometric treatments such as low-pass or high-pass filters, or background elimination, etc.

[0115] According to the invention, this information processing unit 9 comprises a neural network having been trained during a construction phase, on a learning set comprising recordings each composed of at least two images of the same portion of the same container according to at least two different observation directions and according to at least one modality. During this construction or learning phase, the neural network determines at least one membership class Kj for said portion from a list of classes Kl, K2, ...Kj, ...Kp.

[0116] During a use phase, the trained neural network receives as input, for each container, a recording of at least two images of at least one portion of the container according to a modality and according to two different observation directions, these images being acquired using the inspection system 7 during the scrolling of the containers. For each container to be inspected, the neural network analyzes this recording to determine the belonging of this portion of container to a result class from the list of classes Kl, K2, .. .Kj, .. .Kp.. During this phase of use or inspection, each container 2 passing in front of the device 1 is inspected in order to detect in the images taken, the presence of defects and to classify the detected defects. Preferably, the classes Ki should also include non-defect objects such as decorations, markings, shadows, etc.

[0117] The information processing unit 9 is thus adapted to implement an inspection method for detecting defects on containers and classifying the containers, according to previously defined classes. The inspection method thus makes it possible to classify each container according to at least one class taken from a list of classes comprising in particular a class of absence of defect and a class of presence of defect. It must be understood that the subject of the invention makes it possible at least to detect the presence of a defect or the absence of a defect.

[0118] Advantageously, the subject of the invention makes it possible to identify or recognize defects, thus allowing them to be classified. The list of classes K1, K2, ...Kj, ...Kp comprises at least one class comprising the presence in the image portion of at least one abnormal optical singularity corresponding to a defect. In the field of inspection of glass containers, some of the classes correspond to glass defects, i.e. to defects linked to the manufacturing process of the containers. The glass defects concerned are glass defects having optical properties of interaction with the light passing through the container, such as at least one part of absorption, and / or one part of birefringence and / or one part of refraction, so that they are detectable by means of the aforementioned devices. For example, these classes may correspond to a bubble, an inclusion, a fold, a grain, a stone, a fin, a large bubble, a trapezoid or a swing.Furthermore, several classes can correspond to the same type of glass defect, such as a trapezoid-type glass defect. In this example, one class can correspond to large trapezoids with thick glass threads and another class to small trapezoids with small, unconnected points. These classes are given for illustrative purposes only.

[0119] Advantageously, the list of classes Kl, K2, ...Kj,...Kp comprises at least one class comprising the presence in the portion of at least one normal optical singularity not corresponding to defects. Thus, classes may correspond to reliefs with a technical function such as positioning notches or the grooves of the laying plane, with a decorative function such as coats of arms, or with a function of technical or commercial indications such as brand, capacity, mold number. Other classes may correspond to elements which can be distinguished on the container such as mold seams, which can be circular at the bottom or linear on the vertical wall. The recognition of a mold seam in the image, thus the classification of an image element as a mold seam, then allows a specific analysis to determine whether the mold seam is weakly marked, which is not a defect, or strongly marked, which requires the ejection of the container bearing such a mold seam.

[0120] Advantageously, it is possible to associate with each class in the list of classes a criticality, that is to say a high value for a class of critical defects such as a trapezoid, lower for a class of less critical defects such as a fold, even lower for a class of non-defect objects such as a mold joint.

[0121] It should be noted that the number p of image classes is independent of the number of modalities. Generally, the list of classes includes a number of classes greater than the number of modalities implemented. Furthermore, the list of classes may include classes not corresponding to glass defects. For example, the list may include, as a class not corresponding to glass defects, a class corresponding to a container 2 without defects, a class corresponding to a container 2 with a mold seal, a class corresponding to a container 2 with a coat of arms. According to an advantageous embodiment, the list of classes contains a non-glass defect class, at least one trapezoid class, at least one inclusion class, and at least one bubble class. These classes are recorded and accessible to the information processing unit 9.

[0122] The number of classes is therefore determined firstly by the need for production and quality control, therefore by the need to identify production defects in order to make the right decisions during sorting and to enable possible correction of the process. Conversely, the number of modalities is only determined by the technical and economic limits of the known means of highlighting the properties of absorption, refraction and birefringence.

[0123] The number of classes also depends on the quality of the sorting obtained by means of the neural network. Indeed, during the training of the neural network, it is known to verify on test sets, the rate of good classifications obtained. It has been observed that the classification is better when the list of classes contains several classes for the same defect such as trapezoids. In other words, the number of classes can be increased to improve the quality of the automatic classification.

[0124] As indicated previously, the classification operation is carried out by a neural network receiving as input, for each container, a recording of at least two images of at least one portion of the container according to a modality and according to two different observation directions. According to the invention, the at least two images are analyzed at the same time by a deep learning neural network which can determine, based on all significant photometric and / or geometric characteristics, the membership of an image singularity to a class.

[0125] [Fig.2] illustrates an exemplary embodiment of a convolutional neural network CNN 1 receiving as input the images of portions of the containers according to at least two different observation directions and one modality. The outputs of the convolutional neural network CNN1 are the input data of a neural network NN making it possible to classify the containers.

[0126] [Fig.4] is a schematic view illustrating an exemplary embodiment of an inspection device according to the invention implementing two cameras adapted to obtain from each three analysis images according to three different modalities, in order to constitute a recording of analysis images comprising at least 6 container images.

[0127] In this figure, a container 2 and two cameras C1 and C2 are shown, configured to respectively acquire images of the same portion of a container according to two different observation directions D1, D2. The camera C1 is configured to obtain three images according to three different modalities, and the camera C2 is configured to obtain three images according to these same different modalities. In the figure, K1, K2 are designated as possible classes for the recordings of these six images. During the learning phase, these six images can be recorded with the appropriate class, and during the detection phase, the six images can be acquired for classification.X is defined as context data or metadata about the images, which can be taken as input data to the classifier, for example X contains viewing directions, viewing direction differences, camera numbers, modality identifiers for each image.

[0128] [Fig.5] illustrates another exemplary embodiment of an inspection system 7 allowing the acquisition of images of portions of containers according to three different observation directions and one modality. One of the two cameras shown delivers images according to two different observation directions. Three convolutional neural sub-networks CNN1, CNN2, CNN3 each receive as input the images of portions of the containers according to a determined observation direction. The outputs of the three convolutional neural sub-networks CNN1, CNN2, CNN3 are the input data respectively of neural networks NN1, NN2, NN3 collaborating together with a neural sub-network NN4 allowing the classification of the containers. The convolutional neural subnetworks CNN1, CNN2, CNN3, the neural subnetworks NN1, NN2, NN3, and the neural subnetwork NN4 form a neural network, i.e., a single deep learning model within the meaning of the invention.

[0129] [Fig.6] illustrates another example of an embodiment of an inspection system 7 allowing the acquisition of images of portions of containers according to two different observation directions and one modality. A single NN network is used and is represented in the figure. On each image obtained by the camera C1 or by the camera C2, a singularity detection operation is carried out in order to detect the presence of one or more singularities (by the DSI module for the image obtained by the camera C1, by the DS2 module for the image obtained by the camera C2). This DSI or DS2 singularity detection detects and locates in the image an image singularity in the sense of a connected or unconnected set of pixels having particular local characteristics relative to neighboring pixels or the background.DS1 or DS2 singularity detection can implement any image processing method suitable for detecting singularities with here one or more preliminary processing steps (AND1 and AND2 in the figure) such as filtering, thresholding, mathematical morphology transformations, labeling, analysis of variance, edge detection etc. A typical result of DSI or DS2 singularity detection may include only the position of a pixel or a set of pixels (typically the connected pixels that form the singularity). In other words, singularity detection implements image segmentation. For each image, a rectangle framing the detected singularity can easily be sent to the neural network. It is possible to associate additional information with the detected singularities, such as image primitives (perimeter, contrast) or information relating to the container such as its orientation on the conveyor.Also, both singularity detection modules can output the position and dimensions of a bounding box that contains a detected singularity.

[0130] Having carried out this segmentation / detection of singularity on the at least two images, a matching is implemented by means of the MC module between the two singularities detected by the DSI and DS2 modules, on the basis of their positions in the images or on the container. For example, if relative to the image of the container or to its external contours, the two framing rectangles (if the DS1 and DS2 modules deliver such rectangles) overlap, or are at the same height for images of the vertical wall, or on the same circle during a background check, etc., they are considered as corresponding to the same part or to the same singularity of the container. Alternatively, a precise matching can be carried out according to the position on the container of the singularity(ies) detected by the DSI and DS2 modules, knowing the geometry of the container and of the acquisition device.

[0131] According to the example of [Fig.6], if a singularity is detected in a single image and not in the other by means of the DSI and DS2 modules, it is nevertheless possible to send to the NN network two rectangles framing (or any other output of the DSI and DS2 modules) corresponding to each other, the image without detection of singularity can nevertheless contain information useful for classification according to 2 directions of observation. Finally, when the NN network of the figure is of the CNN type, the quality and precision of the upstream detection of singularity does not matter, provided that this detection detects all singularities, including false detections, the NN network of the CNN type comprising so-called convolution stages capable of detecting artifacts, singularities, and of determining primitive forms to carry out a classification whose result can be the absence of singularity, the presence or absence of defect, or the type of normal singularity or defect.

[0132] In the example of [Fig.6], only the NN network is modified during learning, the MC, DSI, and DS2 modules not being modules that can be modified by learning. In other words, the back-propagation of the result of a cost function only changes parameters of the NN network and not the MC, DSI, and DS2 modules (it can be noted that learning as it can be implemented for example for the example of [Fig.2] is advantageous in that it modifies parameters that have an impact that is less limited than those of the example of [Fig.6]).

[0133] Figures 7A and 7B show an arrangement of a plurality of cameras arranged at different azimuths and elevations and usable for implementing the invention, in an inspection system 7.

[0134] In [Fig.7A], for a container 2 circulating on a conveyor track Pc, the position of cameras Ci and projectors Ei is shown, arranged with an elevation angle El visible in the figure and measured relative to a reference inspection plane Prib. Here, the cameras Ci and the projectors Ei cooperate to enable the detection of glazes, as explained in document WO2021 / 209704.

[0135] In [Fig.7B], the position of cameras Ci and projectors Ei is shown, arranged with an azimuth angle Az visible in the figure.

[0136] Figures 8A and 8B are two images of the same portion of the same container, here two images of the base of the same glass bottle. The figure also shows the orientations of the cameras C1 and C2 used respectively to obtain two images I1 and I2 in opposite directions and facing each other (angle of 180°).

[0137] The defect DF1 is only visible here on image II. A model according to the invention may be able to deduce from the visibility of the defect on image II alone that the defect is of the “open blister” type. Thus, the invention makes it possible to better classify defects in glass containers.

[0138] Figures 9A and 9B are two images of the same portion of the same other container, acquired by two cameras whose arrangement is visible in [Fig.9C]. Here two images of the same glass bottle are obtained, the bottom of this bottle resting on a conveyor, the axis of symmetry of the container being vertical. In [Fig.9C], more precisely, we also represented the orientation of the cameras C1 and C2 used respectively to obtain two images II and 12 according to observation directions forming an angle in a vertical plane noted [3 (which is a difference in elevation) and an angle in the horizontal plane noted a (which is a difference in azimuth). The appearance of the same defect DF2 differs between the two images.

[0139] In fact, in image 12 a defect appears with a low contrast appearance so the model can deliver a defect class with a low confidence index, or even a class error.

[0140] In image II, the DF2 defect is more contrasted, a model trained to process single images could give the broth class with high certainty, but nevertheless, taking into account image 12 can lower the confidence rate. Without the invention, there would be two defects and one of the two would be misclassified. But by combining the two images as input to a single model, the classifier makes the right decision, and assigns a single class to the two image portions with high confidence.

[0141] The invention therefore reinforces confidence in the classification of defects.

[0142] Figures 10A and 10B are two images of the same portion of yet another same container, here two images of the same glass bottle. The figure also shows the orientations of the cameras C1 and C2 used respectively to obtain two images I1 and I2 in directions forming an angle in a horizontal plane denoted a.

[0143] Two DFA and DFB defects are apparently visible in both images, these defects appearing to be fin-type. In none of the views taken alone is it possible to detect and classify the artifact as a fin with certainty. In fact, these two fin-type defects result from a single cause, namely poor closing of a mold used for forming the bottle. We therefore have a single defect in two parts.

[0144] DFA1 is the front view, by the camera Cl, of a first DFA part of the defect (larger)

[0145] DFB1 is the rear face view (camera Cl) of a second DFB part of the defect (small)

[0146] DFA2 is the rear face view (camera C2) of the first DFA part of the defect (larger)

[0147] DFB2 is the front face view (camera C2) of the second part DFB of the defect (small)

[0148] With the invention, the recognition of the front face defect (DFA1 or DFB2) despite a background (rear face) comprising shadows caused by the counter ring, is possible by the combined analysis of the images according to different directions.

[0149] The angle a between the 2 observation directions is here at least 60°.

[0150] The invention makes it possible to automatically detect and recognize / classify defects that are only partially visible in a single observation direction.

[0151] Prior to the inspection of containers (use phase), the object of the invention aims to train a neural network (construction or learning phase) on a learning set comprising recordings each composed of at least a first and a second image of the same portion of container according to two different observation directions and according to at least one modality.

[0152] According to an advantageous embodiment variant, the recordings of this learning set are carried out by an inspection system 7 which is the same system used for the use phase. In other words, the inspection systems 7 used to acquire the images during the construction phase and the use phase are the same. Advantageously, these are substantially the same observation directions as those during the inspection, with the same modality.

[0153] For information purposes, the construction phase may precede the use / inspection phase. Also, a construction phase may be implemented after an inspection use phase. For example, records usable for training may be obtained after a first implementation of the use / inspection phase, and the implementation of a construction phase subsequent to the use / inspection phase makes it possible to further improve the operation of the model according to the invention. A construction phase subsequent to a use phase is analogous to a construction phase that precedes a use phase, and it may itself be followed by a use phase.

[0154] According to an advantageous characteristic, during the construction phase, the image recordings are ordered according to a determined sequence. Thus, the images coming from the different cameras are classified according to a determined order. During the use phase, the image recordings are ordered according to a sequence identical to the sequence of the construction phase. In other words, the way in which the images are organized or arranged among themselves within the recordings to be presented as inputs to the classifier network, whether in an inspection phase, therefore classification, or in a construction phase, therefore training, is the same. The images obtained according to a given observation direction and a given modality will preferably be presented to the same input connectors of the model in both phases (typically to the same input neurons).However, although the identical order for both phases is a preferred method, it is possible alternatively, for example when the images are obtained by means of a device in which the observation directions are distributed in azimuth around the axis of the containers, and taking into account that the containers during inspection . can arrive in the device with an indeterminate orientation, only the relative arrangement modulo 360° of the observation directions for each image of a recording counts.

[0155] To construct the learning base, several containers presenting defects to be recognized are selected, but also several containers without defects, several containers bearing normal optical singularities to be recognized: decorations, codes, notches, screw moldings, positioning notches, etc., and / or several containers presenting commercially acceptable optical singularities.

[0156] It should be understood that the images of these recordings show at least a portion of a container to be inspected corresponding to regions of interest of the container such as the ring, the neck, the shoulder, the body, the rim or a right or left half-side or an area where engravings are present. Some of these images include normal optical singularities to be recognized or abnormal optical singularities corresponding to defects. These images may be limited to these optical singularities or take into account a rectangle framing these optical singularities. These images may also take into account an enlarged rectangle framing these optical singularities, taking into account the context or the positioning of these singularities in the image.

[0157] The method aims to bring together these recordings in a learning base, in a large number, typically at least a thousand recordings. For each class, several recordings are made comprising images of defects to be recognized, images with normal optical singularities to be recognized or images of containers without defects. It should be noted that the containers are preferably of the same material (glass or plastic material) as those to be inspected, and are preferably manufactured according to the same process. These containers can be of the same model, the same color, etc. but preferably the method consists of including in the learning base images of several containers of different models for each type of defect and several types of defects for each container model.

[0158] According to the invention, the construction phase is supervised learning. In other words, the neural network is trained by supervised learning, that is to say by operations imposing on it the classification to be carried out with an error to be minimized for a given set of records. Each record of the learning base is associated, at least as a label, with a membership class such as a defect type or a non-defect type.

[0159] According to this supervised learning method, the system is provided with a set of sorted records, labeled according to one of the previously defined classes. Each record is associated, via sorting and labeling, with one of said classes, which will allow the algorithm to calculate a more general model subsequently allowing to associate any unknown, unlabeled data with one of the previously defined classes. This supervised learning method differs from the unsupervised learning method (being without a priori on the classes). According to this unsupervised method, data is provided in bulk to the system, without any form of sorting or labeling. The system itself defines the number of classes that seems most relevant and associates each data item with one of the said classes (example: X-Means clustering algorithm). The supervised learning method also differs from unsupervised learning (with an a priori on the number of classes): data is provided in bulk to the system, without any form of sorting or labeling. On the other hand, the system is told the number p of expected classes. The system then automatically associates each data item with one of the p expected classes (example: K-Means clustering algorithm).Unsupervised learning is not suitable for the objective of classifying defects according to a defect nomenclature, particularly for glass, which would be determined a priori. On the other hand, in the case of detecting glazes using the invention, unsupervised learning may be more effective, the automatically determined classes allowing better classification due to the fact that the learned model would be able to take into account characteristics of the glazes which correspond to their visual “signature” without any link to their cause. As explained previously, it is difficult for a person skilled in the art to determine particular shapes of glazes, apart from their orientation (horizontal, vertical, inclined), and this orientation means that only certain transmitter-receiver groups are dedicated to a certain glaze orientation.

[0160] The method according to the invention aims to train the neural network so that it recognizes the types of defect and the containers not containing defects. This learning phase is carried out by a person skilled in the art of artificial intelligence who chooses in particular, a neural network model, a learning base and a learning algorithm. This neural network is trained and tested iteratively until the desired result is obtained (confidence matrix, metrics (precision, recall, fl-score, mAP50 and mAP75 (mAP being an English acronym meaning "mean Average Precision"). Optionally, the databases of these image recordings are reorganized with additions or deletions and / or the neural network model is changed as well as the learning algorithm.

[0161] It is clear from the above description that the subject of the invention is based on the observation that the shape (morphology) and sometimes the photometry (contrast in the broad sense, therefore intensity or color as well as separation of intensity and color) of a defect or non-defect type singularity varies according to the direction of observation and in a manner specific to the type of singularity. The change in appearance (morphology and / or pho- tometry) in the image of a defect depending on the observation directions and also the lighting is a characteristic which makes it possible to differentiate the defects.

[0162] To detect a glaze-type defect as described in patent application WO 2021 / 209704, a region of the container is illuminated under precise incidences, by directed light beams reaching the surface of the container at a precise incidence so that the majority of the beam penetrates the glass wall and propagates in the glass. If a glaze is present on the light path in the wall, then the glaze reflects the beam which leaves in a modified direction to exit the wall at a precise exit angle, which is a function of the incident angle and the position and shape of the glaze. A glaze-type defect in an image of observed shape differs depending on the observation angle, or is invisible in another direction even close (+ / -10°) to the first direction.However, in a glaze observation system, singularities corresponding to parasitic lights (called "parasites") appear in the images, often linked to voluntary reliefs which vary little or not at all or differently with the direction of observation. The modification of the image by the change of direction of observation therefore makes it possible to differentiate a glaze type from a parasite.

[0163] For transparent containers illuminated in transmission by light panels as described in patent application WO 2023 / 052732, the shadows on the edges of the containers have a shape and position depending on the distribution of the glass but also on the observation angles and the lighting angles. In transmission, the modification of the morphology and / or the photometry of a defect varies according to the relative position of the container with respect to the observation directions and in particular: - for an absorbing defect, mainly the shape varies but the photometry varies little; - for a refractive defect, the morphology and photometry vary.

[0164] Furthermore, the way in which the image of a defect varies when the point of view or lighting direction changes depends on the position of the defect in depth (surface, interior wall, interior container). Trapezoid-type defects can thus be recognized. The shape of trapezoid-type defects in the images taken changes according to the direction of observation in a manner specific to this defect, in particular because: - it is made of material extending inside the glass container. It therefore has characteristics linked to its three-dimensional shape which are not taken into account in an image according to a single direction of observation; - in the case of a machine with six cameras described in patent application WO 2023 / 052732, the trapezoid can be seen as: * a small object, attached to a wall; * two small objects each attached to a wall; * a large object attached to two walls * a small circular object apparently on the wall.

[0165] Also, for transparent containers illuminated in transmission by light panels, as described in patent application WO 2023 / 052732, it appears advantageous to acquire for each container, at least four images of at least one same portion of a container according to four different observation directions and according to at least the first modality, the observation directions being distributed around the central axis R of the container two by two according to an azimuth angle of at least 45°.

[0166] Furthermore, a bubble in a transparent container has a different observed shape depending on the observation angle, and this difference is not directly deducible by a geometric transformation. Indeed, the observed shape of the bubble and even the contrasts of the bubble depend on the relative position of i) the light source, ii) the bubble in the container (position / orientation of the container) and iii) the observation direction. Conversely, an inclusion or a grease stain on the surface will have an image shape that varies with the observation direction but in a quasi-deterministic manner by the geometry of the container and the observation direction, while the photometry varies little. It follows that the variation of the geometry and the photometry between different points of view is a discriminating characteristic of bubbles and stains or inclusions.

[0167] The inventors therefore deduced from these observations that the way in which the appearance of a singularity of the container or of the image changes, between two observations under a different direction is a characteristic of the bubble defect compared to the surface spot defect.

[0168] However, taking these behaviors into account is complex and difficult to model since the defects are of a very changeable complex form. It is extremely illusory to try to generically model the transformations accompanying the modification of the point of view, which cannot be reduced to a modification of simple primitives. Furthermore, the shape of the containers has an impact on the modeling of the defects, which makes these models difficult.

[0169] This is why it has been found that neural networks are capable of analyzing (taking into account) - the shapes and contrasts (Morphology / geometry) of the images of the defects - the variations of said shapes and contrasts between images taken from different observation directions, and this despite the fact that these shapes and / or contrasts as well as their variations according to the direction of observation are complex and very difficult to predict or model by a priori knowledge of the geometry of the acquisition system.

[0170] It is clear from the above description that the method according to the invention not only makes it possible to identify glass defects in containers, but also to classify these defects to enable the transition from inspection to optimization of the manufacturing process. One of the characteristics of the invention is to define a list of classes comprising classes of defects, which makes it possible to relate the glass defects to characteristics of the manufacturing process to be regulated. Improving the classification of glass defects makes it possible to better trace the causes of the glass defects.

[0171] The object of the invention is advantageously exploited in the context of manufacturing installations to allow better detection and categorization of defects present within the containers. Certain defects can be seen, detected and categorized more easily thanks to the different observation directions possibly combined with different modalities.

[0172] Preferably, the inspection method according to the invention is designed so that the neural network associates a confidence score with the classification of each inspected container of a production. The confidence score is typically the probability of the container belonging to the membership class. The score can be expressed as a % or a value between 0 and 1.

[0173] It should be noted that a container may have several defects. There are several ways to classify such containers. In the variants of the method which include a segmentation step, as illustrated in [Fig.6], it is possible to extract analysis images of the same container, several image regions and for example several SR segments are recognized as belonging to defect classes. In this case, according to a first variant, the belonging class of the container will be that of the segment classified in the defect class having the highest criticality. It is also possible to take into account the confidence score of the classification, that is to say that the class assigned to the container will be that of the segment classified with a confidence score higher than the confidence threshold. According to a second variant, it is possible to count all the defects carried by a single container, in particular when a container carries several critical defects.Thus the statistical analysis of production which will be presented later, can account for the distribution of defects independently of the number of containers rejected.

[0174] The neural network and in particular the CNN detects the defects and also provides information on their position in the container. This position is useful for correcting the process, for example it is useful to know if a defect is in the body or the neck. In addition, the CNN can take into account the position of the singularities in the container to carry out the classification. According to the invention, the neural network also takes into account the relative position of the defect in the images according to directions different observation.

[0175] The subject of the invention is used for sorting the production of containers in the following manner. After classifying a container, at least one container sorting characteristic is compared to a rejection criterion, and when the sorting characteristic exceeds the rejection criterion for a container, the container is considered non-compliant and rejected. Indeed, the installation comprises an ejector for removing defective containers from production. The sorting characteristic and the rejection criterion are dependent on the membership class to decide whether or not the container is compliant, the sorting characteristic being calculated on at least one image of the container according to one of two or three modalities. The sorting characteristic and the rejection criterion are for example a defect dimension such as its surface area or length measured in at least one analysis image.The confidence score can possibly be taken into account for sorting, by rejecting containers belonging to a low-critical defect class only if the confidence score is high and conversely by rejecting containers belonging to a critical defect class even if the confidence score is low. Since the neural network is able to determine the position of a defect, with high confidence thanks to the different observation directions, according to a variant of the invention the position is an additional rejection criterion, because the position of a defect can influence its criticality.

[0176] According to the invention, it is possible to define different rejection criteria depending on the position of the defects in the container.

[0177] The object of the invention is used to carry out a statistical analysis of a production of containers, that is to say an analysis of the frequency or distribution of the different types of defects and their criticality, the types of defects included in the list of classes and their criticality being determined in advance for the purpose of controlling the process.

[0178] Certain defects are caused during the stages of forming the containers in the molds, and are therefore to be related to the forming parameters which are different from one section to another or from one cavity to another. It is therefore preferable to count the classes of defect according to the section or cavity of origin of the containers. When the installation is installed at the exit of forming machines, therefore upstream of the annealing arch 6, the inspection is immediate after the manufacturing so the time stamp of the manufacture of the container is known, and the cavities or sections of origin of the containers are also known by synchronization since the order of exit of the containers from the forming machine is known.When the inspection device 1 according to the invention is installed downstream of the annealing arch 5, it is preferably equipped with a device for reading information carried on the containers and indicating the mold or the original section of the containers and / or a time stamp of their fa. brication and / or a unique identifier of each container such as a serial number, or linked to such a reading device. It is therefore possible and preferable to carry out the statistical analysis of the production, from the classification of the containers by the inspection process, according to the distribution of the defects in direct relation to the production parameters at the time of manufacture of each container and / or according to the different cavities and sections of the manufacturing machine.

[0179] The statistical analysis of the production of containers thus makes it possible to relate the defects to their causes, to obtain two results: - on the one hand, correlations can be determined between manufacturing parameters and the resulting defects, thus making it possible to define more efficient manual process control methods; - on the other hand, knowing the cause and effect relationships, deliver in real time to the production calculator 7, the possibility of creating an automatic feedback loop to regulate the process by correcting the defects and therefore the gaps between the desired quality and the estimated quality of the containers.

[0180] To summarize the above, the object of the invention is advantageously exploited for different production operations described below: - sorting of production; - statistical analysis of production; - determination of cause and effect relationships between production parameters and defects; - process control and command by reducing the defects observed; - warning operators of the occurrence of critical faults by an alarm.

[0181] According to a preferred variant of the invention, the classification of the containers is only taken into account when the confidence score of the assigned class exceeds a confidence threshold for: - count defects by defect class in a defect frequency statistic, - and / or decide to reject the container, - and / or trigger an alarm for the presence of one or more critical defects in the inspected production.

[0182] It should be noted that the confidence threshold corresponds to a predetermined or adjustable minimum value of the confidence score as an operating parameter of the inspection device, the adjustment being made on its HMI or remotely.

Claims

Claims

1. Method for inspecting containers made of transparent or translucent material (2) with a view to classifying a container, the method comprising; a use phase comprising: - the acquisition for each container, of at least a first and a second image (le) of at least one same portion of a container according to two different observation directions and according to at least one modality; - the supply as input to a deep learning model (NN), for each container, of a recording of at least the first and the second images of at least one portion of the container according to at least one modality and according to two different observation directions; - and the analysis by the deep learning model, for each container, of this recording to determine the membership of this portion of container, to a result class among a list of classes.

2. Method according to claim 1 according to which the method comprises: - the provision of an inspection system for acquiring images according to at least a first modality and a second modality; - the acquisition for each container, of a recording of at least one same portion of a container comprising at least two images according to at least two different observation directions according to the first modality and at least one image according to the second modality.

3. Method according to one of the preceding claims, according to which the method comprises the provision of an inspection system configured to acquire images of the same portion of a container according to at least two different observation directions and according to at least one modality taken from the list of the following modalities: absorption, birefringence, refraction, reflection, infrared radiation.

4. Method according to one of the preceding claims, according to which each portion of container is classified according to at least one class taken from a list of classes comprising at least one class of absence of defect in the portion and one class of presence of defect in the portion.

5. Method according to one of the preceding claims, according to which each portion of container is classified according to at least one class taken from a list of classes comprising at least one class comprising the presence in the portion of at least defects such as, in particular, trapezium, inclusion, bubble.

6. Method according to one of the preceding claims, according to which each portion of container is classified according to at least one class taken from a list of classes comprising at least one class including the presence in the portion of at least one singularity such as a marking, a mold seal, a notch, a thread, an impression, a stitch, a handle, a counter ring.

7. Method according to one of the preceding claims, according to which the method comprises the acquisition, for each container having a central axis, of at least four images of at least one same portion of a container according to four different observation directions and according to at least the first modality, the observation directions being distributed around the central axis two by two according to an azimuth angle of at least A

8. Method according to one of the preceding claims, in which at least one sorting characteristic is compared to a rejection criterion, the sorting characteristic and the rejection criterion being dependent on the membership class to decide whether or not the container is compliant, the sorting characteristic being calculated on at least one image of the container according to a modality.

9. Method according to one of the preceding claims, in which a step is implemented of taking into account at least one identified defect in order to deduce adjustment information for at least one control parameter of a container manufacturing installation.

10. Method according to one of the preceding claims, wherein: - the deep learning model associates a confidence score with the classification of the containers forming part of an inspected production; - the classification of the containers is taken into account only when the confidence score exceeds a confidence threshold for; - counting the defects by class of defects; - and / or deciding on the rejection of the container; - and / or triggering an alarm for the presence of at least one critical defect in the inspected production.

11. A method of training a deep learning model for inspecting containers made of transparent or translucent material in order to classify a container, the method comprising a construction phase comprising: - a provision of a training set comprising recordings each composed of at least a first and a second image of the same portion of container according to two different observation directions and according to at least one modality; - a provision of at least one deep learning model of neurons having been trained on a training set comprising recordings each composed of at least a first and a second image of the same portion of container according to two different observation directions and according to at least one modality, the deep learning model determining at least one membership class Kj for said portion from a list of classes.

12. Method according to claim 11, according to which the method comprises providing at least one deep learning model having been trained on a training set comprising recordings each composed of at least two images of the same portion of container according to observation directions different by at most 5° during the acquisition of the images.

13. Method according to one of claims 11 or 12, according to which the method comprises providing at least one deep learning model having been trained on a training set comprising recordings each composed of images of the same portion of container according to different directions and modalities.

14. A method comprising a phase of constructing the method according to any one of claims 11 to 13 to obtain a deep learning model, and a phase of inspecting the method according to any one of claims 1 to 10 using the deep learning model of said construction phase.

15. A method according to claim 14, wherein during the construction phase, the image records are ordered according to a determined sequence while during the use phase, the image records are ordered according to a sequence identical to the sequence of the construction phase.

16. Device for inspecting containers made of transparent or translucent material leaving a manufacturing or recovery facility with a view to classifying the containers in relation to defects, the device comprising: - an inspection system comprising at least one camera arranged to recover light or radiation coming from at least a portion of a container and configured to acquire images of the same portion of a container according to at least two different observation directions and according to at least one first modality; - an information processing unit connected to the inspection system and comprising a deep learning model, the deep learning model determining at least one membership class for said portion from a list of classes, the deep learning model receiving as input, for each container, a recording of at least two images of at least one portion of the container according to the first modality and according to two different observation directions, the deep learning model, for each container, analyzing this recording to determine the membership of this portion of container, to a result class from the list of classes.

17. A deep learning model obtained by the learning method according to any one of claims 11 to 15.

Images