Contactless method for checking sealing, closing or positioning by means of an ultrasound probe and system for said method
The integration of convolutional neural networks and preprocessing techniques in the ultrasonic sealing verification system addresses the limitations of existing methods by enhancing precision and efficiency in contactless sealing verification, utilizing ultrasonic waves for real-time classification and reducing computational burdens.
Patent Information
- Application Number
- PCT/ES2023/070336
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-25
- Publication Date
- 2024-11-28
AI Technical Summary
Current ultrasonic sealing verification methods rely solely on wave analysis without utilizing convolutional neural networks, limiting their ability to solve complex, non-linear problems and requiring extensive manual sample collection and labeling, which slows down the training process and increases computing requirements.
A method and system that employ convolutional neural networks for real-time classification using ultrasonic waves, incorporating preprocessing steps like band-pass filtering, FFT transformation, and normalization, along with a GPU/CPU server for processing, to enhance precision and efficiency in contactless sealing verification of containers and surfaces.
The system achieves high precision in sealing verification with reduced training time and computing requirements, enabling rapid deployment and minimal manual sample collection through self-regulation and transfer learning techniques.
Smart Images

Figure ES2023070336_28112024_PF_FP_ABST
Abstract
Description
[0001] CONTACTLESS METHOD OF CHECKING SEALING, CLOSING OR POSITIONING BY AN ULTRASONIC PROBE AND SYSTEM FOR SUCH
[0002] METHOD
[0003] DESCRIPTION
[0004] OBJECT OF THE INVENTION
[0005] The object of the present invention, as the title of the invention establishes, is a method of checking sealing, closure or positioning without contact using an ultrasonic probe, as well as the system for carrying out said method.
[0006] The present invention is characterized by the special design and configuration of each and every one of the stages that form part of the sealing verification method, such that an effective, robust method is achieved that allows the sealing of a container to be verified with a very high degree of accuracy.
[0007] Therefore, the present invention falls within the scope of methods for checking the closure of containers based on ultrasonic waves.
[0008] BACKGROUND OF THE INVENTION
[0009] Sound propagation is medium-dependent. Building on this, the difference in longitudinal wave propagation in a sealed medium compared to one that isn't sealed exists, so, taking advantage of the different propagation characteristics, a device capable of detecting this difference and classifying it in real time must be manufactured.
[0010] The methods and systems for ultrasonic sealing verification used until now are limited to the analysis of received waves, without employing convolutional neural networks that allow non-linear problems to be solved, generating an on-demand improvement and capable of modifying the operating point at runtime. Therefore, the object of the present invention is to develop both a method and a procedure that employs neural networks to solve complex problems and that are capable of self-regulating based on the results obtained in order to increase their precision after the training phases, developing for these purposes a method and a system such as the one described below and set forth in its essence in the first claim.
[0011] DESCRIPTION OF THE INVENTION
[0012] The object of the present invention is set forth in its essence in the independent claim and the different embodiments are set forth in the dependent claims.
[0013] The present invention relates to a sealing verification method, the sealing concept being extendable to that of closing in the case of bottles, as well as to that of positioning in the case of car windows, where the verification is carried out without contact and by means of ultrasonic waves, the system by which the application of said method is carried out.
[0014] Contactless sealing verification comprises the following steps:
[0015] - Sampling, where a series of samples are taken from the containers, packaging, or items (such as car windows, for example) to be inspected.
[0016] - The samples are then preprocessed, either digitally or analogously, using a band-pass filter. Very high-frequency components are eliminated to protect the models from being trained with noise artifacts. Low-frequency components that are not the result of our excitation of the medium are also eliminated because they do not contain deterministic information about the sample.
[0017] Not performing this cleaning does not necessarily mean that the model will perform poorly in classifying, but it will result in less reliable and necessarily more expensive training, which will slow down the prediction process and increase the system's computing requirements. Other techniques can be applied to speed up the process, such as trimming the samples over time, keeping only the portion of the sample where the distinction between classes is greatest.
[0018] A constant could also be subtracted, for example, by filtering out all frequency components that coincide with the excitation pulse. Normalizing samples to narrow their values is also a recommended step, but doing so must take into account the sample cutoff and the number of microphones in the sensor.
[0019] - The procedure continues with a sample processing stage that seeks to facilitate the subsequent training of the models where the samples are transformed into the frequency domain using mathematical operations that discretize the signals, such as the Fast Fourier Transform (FFT), thus obtaining their spectrum.
[0020] At this point, the sample values must be normalized again if they are not already normalized. This frequency decomposition, in addition to being visually useful when analyzing the behavior of a sample, will directly provide information to the subsequent Artificial Intelligence model, which it would have learned to extract. Therefore, this step is relevant and, in the long run, accelerates the process. The margin that the variables used in preprocessing and processing allow before a loss of relevant information occurs is, in any case, wide. Whether optimal parameterization is significant for subsequent classification is under study. Frequency resolution is the parameter that must be correctly set to avoid loss of information, so the only limit placed on this parameter should be one that prevents real-time operation.
[0021] - The next step is to generate models. The samples the model will work with are, therefore, two-dimensional matrices composed of m samples in time and n samples in frequency, where both m and n are equal to or greater than one. Each of these will be labeled as a valid or invalid product. Furthermore, they will be labeled based on the reason for their invalidity. We could have, for example, three categories:
[0022] 1. Valid product
[0023] 2. Invalid product a) Due to cause (0) b) Due to cause (1 ) c) ...
[0024] In order to create reliable prediction models, they must be pre-trained with samples similar to those they are expected to be able to classify later. Methods such as zero-shot learning are not considered easily applicable in this scenario. However, methodologies along the lines of transfer learning are considered viable. In the former, we would expect a model to be able to classify samples that it has not seen in its training set. In the latter, the model would be trained with samples that may share characteristics with the samples it will later predict, but after this training, the model's weights would be fine-tuned with a relatively small set of real samples. This implies rapid deployment with minimal need for manual sample collection and labeling, as this step could be performed in a controlled and automated environment.
[0025] Convolutional neural network models are used to classify samples. These are neural network architectures widely used in image processing. Convolution is a mathematical function in which two images, which can be represented as matrices, are multiplied.
[0026] In this way, the convolutional layers of a network act as feature extraction filters. After this training, a sample will pass through these filters layer by layer until it is reduced to a sample space in which it can be classified and, ideally, its characteristics are measurable.
[0027] The system used to apply the method described above comprises a loudspeaker capable of emitting in the frequency ranges corresponding to ultrasound, and at least one microphone. All elements will be positioned perpendicular to the surface to be examined. If there is only one microphone or more than one loudspeaker, this angle could be varied to maximize the chances of correctly classifying the samples. The distance between the microphones and their arrangement will be established to maximize the distance between microphones while maintaining their -3 dB pickup limit always above the surface under examination.
[0028] The sampling frequency of the microphones used must be high enough to analyze the relevant wave frequencies. Wave frequencies will be considered relevant in a range whose upper limit is twice the frequency at which the sample was excited. To obtain the minimum sampling frequency of the microphone, we must first select the upper cutoff frequency of the filters that will be used as part of the data preprocessing. Once this frequency is selected, the minimum sampling frequency that we can allow in the microphone will be twice this frequency, according to the Nyquist Theorem. This ensures data acquisition in which information that may be relevant for detection has been captured and can be processed.The data acquisition card containing the analog-to-digital signal converter, whether an external element or part of the microphones, must also respect the sampling frequency margins stipulated above and, in addition, have a bit resolution and dynamic range that does not increase the SQNR (Signal to Quantization Noise Ratio) of the system to a point where the samples lose their potential information.
[0029] In series with this device, a GPU / CPU Server will be responsible for processing the different samples, assigning the collected audio frames to the containers of interest in order to qualify their correct sealing and, potentially, classify any defects that may be present.
[0030] Unless otherwise indicated, all technical and scientific elements used herein have the meaning commonly understood by one of ordinary skill in the art to which this invention pertains. Methods and materials similar or equivalent to those described herein can be used in the practice of the present invention.
[0031] Throughout the description and claims, the word "comprises" and its variants are not intended to exclude other technical features, additives, components, or steps. For those skilled in the art, other objects, advantages, and features of the invention will be apparent in part from the description and in part from the practice of the invention.
[0032] EXPLANATION OF THE FIGURES
[0033] To complement the description being made and in order to help better understand the characteristics of the invention, in accordance with a preferred example of practical implementation thereof, a set of drawings is attached as an integral part of said description, in which the following has been represented for illustrative and non-limiting purposes.
[0034] In Figure 1, we can see a flow diagram of the sampling process.
[0035] In Figure 2, we can see the flow diagram of the sampling processing and preprocessing process.
[0036] Figure 3 shows the flow diagram of the model creation phase.
[0037] Figure 4 shows the flow diagram for the model classification phase. Figure 5 shows a schematic representation of the device used for ultrasound transmission / reception.
[0038] Figure 6 shows the entire system used to apply the method.
[0039] PREFERRED EMBODIMENT OF THE INVENTION
[0040] A preferred embodiment of the proposed invention is described below in view of the figures.
[0041] The non-contact sealing testing method using an ultrasonic probe comprises the following steps:
[0042] - Sampling, which detects the element, is excited by a pulse and the reflected information is acquired and which, as can be seen in figure 1, in a possible non-limiting embodiment, in turn comprises the sub-stages of: o Detection of the product (1) to be analyzed o Beginning of the start of the recording (2) o Emission of a pulse (3) o Active wait (4) o End of the recording (5) o Obtaining the recording (6) and the identified sample
[0043] - Pre-processing stage of the recordings obtained (6), which includes the sub-stages of: o Passing the recordings obtained (6) through a filter (7).
[0044] - Processing stage of the recordings after having passed through a filter (7) includes the sub-stages of: o Mathematical discretization o Obtaining a training set (11) and some real-time samples (10).
[0045] - Stage of creation or generation of models, as shown in figure 3, which takes place based on samples collected in laboratory tests and collected in production lines starting from the training set (11) obtained in the previous stage and which in turn comprises the sub-stages of: o Training (13) where the layers of this neural network are convolutional neural networks that in a possible non-limiting embodiment have four layers: the convolutional layer, a max-pooling layer, an activation layer and a last dropout layer o Classification of the trained samples using two models:
[0046] ■ Use of an encoding model to classify the sample to be trained using as an error metric the accuracy of the classifier that will separate the valid samples from the invalid ones, which allows obtaining a classification model (16)
[0047] ■ Use of a decoder model to classify the trained sample based on the reconstruction error metric obtained after adding a decoder to an encoder of the first model, which will allow obtaining an error characterization model (18).
[0048] - Stage of classification of the samples obtained by means of the models obtained in the previous stage and as can be seen in figure 3, where from the real-time samples (10) and to which the classification model (16) or the error characterization model (18) is applied, a labeled sample is obtained (19), to then proceed to carry out a corresponding action (20).
[0049] In this way, the resulting sample (or "latent") space, represented in a sufficient number of dimensions, will have on its axes the information that causes the samples to be classified as belonging to one class or another and would provide a metric for the different errors. This second model is what is known as a variational autoencoder. It is also a generative model that allows the creation of synthetic samples with which to improve similar models and modify the working point to change, at runtime, for example, the fill level at which a sample is classified as invalid.
[0050] Figure 5 shows a surface (21) to be analyzed to see if it is perfectly sealed on a container, a transmitter / receiver device (22) comprising a transmitter speaker (23) and at least one microphone (24) being arranged by automated means and in a position facing the surface (21) to be analyzed, and which may be more than one microphone, and which, in a possible embodiment without being limiting, the microphones (24) are three and are arranged in an equilateral triangle.
[0051] At the microphone input there is a power amplifier to regulate the intensity of the emitted wave according to the pilot's needs.
[0052] Additionally, a photocell is provided to detect when to begin the measurement cycles. The active waiting time corresponds to a minimum amount of time for all echoes resulting from the pulse emission to dissipate. The preferred, but not limiting, excitation method for the tests is a delta pulse, where the pulse parameters were analytically selected through numerical simulations and would be the most informative in this use case.
[0053] In figure 6, in addition to the elements previously mentioned, it can be observed that the ultrasound transmitter / receiver device (22) is positioned at the distance necessary to carry out the process by means of a telescopic positioner (25), the transmitter / receiver device being in communication with a data acquisition card (26) arranged in an integrated manner in the transmitter / receiver device (22) itself or in a separate assembly, the data acquisition card (26) being in communication with a GPU / CPU Server (27) that will be responsible for processing the different samples, assigning the collected audio frames to the containers of interest in order to qualify their correct sealing and, potentially, classify any faults that may be present.
[0054] Having sufficiently described the nature of the present invention, as well as the manner of putting it into practice, it is noted that, within its essence, it may be put into practice in other embodiments that differ in detail from the one indicated as an example, and to which the protection sought will also be achieved, provided that it does not alter, change or modify its fundamental principle.
Claims
MODIFIED CLAIMS received by the International Bureau on July 23, 2024 (23.07.2023) 1.- Method of checking the sealing, closing or positioning without contact by means of an ultrasonic wave characterized in that it comprises the stages of: - Sampling, which detects the element, is excited by a pulse and the reflected information is acquired. - Pre-processing stage of the recordings obtained (6), which includes the sub-stages of: o Passing the recordings obtained (6) through a filter (7). - Processing stage of the recordings after having passed through a filter (7) that includes the sub-stages of: o Mathematical discretization. o Obtaining a training set (11) and some real-time samples (10). - Stage of creation or generation of models, which takes place based on samples collected in laboratory tests and collected in production lines starting from the training set (11) obtained in the previous stage and which in turn includes the substages of: o Training (13) of the neural networks, where the layers of this neural network are convolutional neural networks. o Classification of the trained samples using two models: ■ Use of an encoding model to classify the sample to be trained using as an error metric the accuracy of the classifier that will separate the valid samples from the invalid ones, which allows obtaining a classification model (16). ■ Use of a decoder model to classify the trained sample based on the reconstruction error metric obtained after adding to an encoder of the first model a MODIFIED SHEET (ARTICLE 19) decoder, which will allow obtaining an error characterization model (18). - Stage of classification of the samples obtained by means of the models obtained in the previous stage and as can be seen in figure 3, where from the real-time samples (10) and to which the classification model (16) or the error characterization model (18) is applied, a labeled sample is obtained (19), to then proceed to carry out a corresponding action (20). Where sampling includes the sub-stages of: o Detection of the product (1) to be analyzed o Beginning of the start of the recording (2) o Emission of a pulse (3) o Active waiting (4) o End of the recording (5) o Obtaining the recording (6) and the identified sample. 2.- Method for checking the contactless seal using an ultrasonic wave according to claim 1, characterized in that the convolutional networks have four layers: the convolutional layer, a max-pooling layer, an activation layer and a final dropout layer. 3.- System for checking sealing, closing or positioning without contact by means of an ultrasonic wave that uses the previous method, characterized in that it comprises: - a transmitter / receiver device (22) comprising a transmitter speaker (23) and at least one microphone (24), where a power amplifier is located at the microphone input to regulate the intensity of the emitted wave. - a photocell to detect when to start the measurement cycles. - a data acquisition card (26) arranged either in an integrated manner in the transmitter / receiver device (22) itself or in a separate assembly, MODIFIED SHEET (ARTICLE 19) the data acquisition card (26) remaining in communication with a GPU / CPU server (27) that will be responsible for processing the different samples, assigning the collected audio frames to the containers of interest in order to qualify their correct sealing and, potentially, classify any faults that may be present. 4.- System for checking sealing, bonding or positioning without contact by means of an ultrasonic wave according to claim 2, characterized in that the number of microphones (24) is three and is placed in an equilateral triangle arrangement around the transmitting speaker (23). 5.- System for checking the sealing glued or positioned without contact by means of an ultrasonic wave according to claim 3 or 4, characterized in that the ultrasound emitter / receiver device (22) is positioned at the distance necessary to carry out the process by means of a telescopic positioner (25). MODIFIED SHEET (ARTICLE 19)
Citation Information
Patent Citations
Three-microphone array far-field pickup method used for conference call
CN108650593A
Procedure and device for acoustically detecting microparticles
US20010007210A1
System and apparatus for dual transducer ultrasonic testing of package seals
US20150293061A1
Method and apparatus for determining an intermediate layer characteristic
US20210396719A1
