SELF-DIAGNOSTIC OF AN ELECTRICAL DEVICE USING SOUND PRINT

Description

[0001] The invention relates to the field of electrical equipment integrating or capable of being connected to at least one microphone. BACKGROUND OF THE INVENTION

[0002] The implementation of efficient and precise predictive maintenance is particularly interesting for the user of “multimedia” or “consumer” type electrical equipment, but also for the manufacturer of said electrical equipment - and this, whatever the equipment in question: “classic” set-top box (or STB, for Set-Top Box ), improved decoder box (integrating for example one or more speakers), connected speaker, computer, smartphone, game console, etc.

[0003] Predictive maintenance involves monitoring electrical equipment in operation and comparing the measured data with data previously collected on other equipment of the same type, in order to anticipate the occurrence of a breakdown. Predictive maintenance therefore makes it possible to act before the user is deprived of the use of their equipment.

[0004] Anticipating a breakdown early is of course positive for the manufacturer's image among users. Predictive maintenance also allows the manufacturer to acquire a certain amount of particularly relevant and useful information, as it comes from equipment in service, and in particular information on equipment failure modes and on specific features possibly associated with certain batches of equipment.

[0005] It is of course advantageous for the manufacturer that predictive maintenance is carried out at a lower cost and without requiring a large number of components dedicated to this monitoring, and that predictive maintenance makes it possible to monitor the greatest possible number of components of the electrical equipment.

[0006] Document DE 10 2018 124210 A1 describes a circuit breaker in which microphones are integrated. This document teaches how to analyze the sound signal captured by the microphones to detect a circuit breaker failure. SUBJECT OF THE INVENTION

[0007] The object of the invention is to implement more comprehensive and therefore more effective predictive maintenance on electrical equipment which incorporates or is connected to one or more microphones. SUMMARY OF THE INVENTION

[0008] In order to achieve this goal, a method for diagnosing electrical equipment is proposed which comprises: a processing unit; at least one internal microphone, and / or means for connection to at least one external microphone; electrical components other than microphones or loudspeakers; the diagnostic method being implemented at least partially in the processing unit and comprising the steps of: acquiring a received audio signal produced, from a capture of an ambient sound signal, by the at least one internal microphone or by the at least one external microphone; producing from the received audio signal monitoring parameters which are representative of a parasitic sound signal included in the ambient sound signal and emitted by at least one of the electrical components; detecting from the monitoring parameters a sound anomaly resulting from a current or future failure of at least one electrical component of the electrical equipment, the method further comprising the steps of: detecting whether the electrical equipment, at the time of the acquisition step, is emitting a useful sound signal, which is formed of sounds emitted voluntarily;if so, applying the received audio signal to the input of an echo cancellation module, to produce a resulting signal, the monitoring parameters being obtained from the resulting signal. ;

[0009] The diagnostic method according to the invention therefore consists of capturing, using the microphone(s), the parasitic sound signal emitted by the electrical components, and detecting from the parasitic sound signal an instantaneous or imminent failure of one or more components of the electrical equipment.

[0010] The diagnostic process therefore makes it possible, through sound fingerprint analysis, to monitor electrical components of electrical equipment, which are not components normally designed to emit or capture sound signals.

[0011] This particularly clever monitoring method allows monitoring to be extended to components that are usually not monitored or difficult to monitor, and this in a low-cost manner, since no hardware components ( hardware ) additional is required for this monitoring. In particular, the diagnostic method does not require the electrical equipment to incorporate or be connected to a loudspeaker, and does not require an external diagnostic instrument.

[0012] We further propose a diagnostic method as previously described, in which the detection step comprises the steps of: performing an inference of a pre-trained machine learning model, using the monitoring parameters as input data, the machine learning model being a classification model; detecting the sound anomaly based on at least one output value obtained by performing said inference.

[0013] Further provided is a diagnostic method as previously described, wherein the monitoring parameters form current spectrograms from the received audio signal, and wherein the machine learning model has been trained using a database comprising images representing training spectrograms.

[0014] We further propose a diagnostic method as previously described, in which the machine learning model is an artificial neural network model of the convolutional neural network or multi-layer perceptron type.

[0015] A diagnostic method is further proposed as previously described, further comprising the step, if the received audio signal is applied as input to the echo cancellation module, of increasing a sampling frequency of the resulting signal, so as to obtain a resampled signal having a sampling frequency equal to that of the received audio signal, the monitoring parameters being obtained from the resampled signal.

[0016] We further propose a diagnostic method as previously described, comprising the step, if the received audio signal has been applied to the input of the echo cancellation module, of using a first inference model, and of using a second inference model otherwise.

[0017] We further propose a diagnostic method as previously described, further comprising the steps of: performing a plurality of acquisitions of the received audio signal; performing an inference of the learning model for each acquisition; performing a first filtering of the output values to obtain at least one filtered value; comparing the filtered value with a first predefined threshold and / or a variation of the filtered value over a first predefined duration with a second predefined threshold to detect the sound anomaly.

[0018] We further propose a diagnostic method as previously described, the processing unit comprising an NPU in which at least the execution of the inference of the machine learning model is carried out.

[0019] We further propose a diagnostic method as previously described, in which the detection step consists of comparing the monitoring parameters with third predefined thresholds and / or comparing variations of the monitoring parameters, over a second predefined duration, with fourth predefined thresholds to detect the sound anomaly.

[0020] We further propose a diagnostic method as previously described, further comprising the steps of: decompose the received audio signal into elementary audio signals on frequency sub-bands; calculate an elementary energy value of each elementary audio signal, the monitoring parameters being obtained from the elementary energy values.

[0021] We further propose a diagnostic method as previously described, further comprising the steps of: performing a plurality of acquisitions of the received audio signal; for each frequency sub-band, performing a second filtering on the elementary energy values associated with said frequency sub-band to produce a filtered elementary energy value associated with said frequency sub-band, the monitoring parameters being the filtered elementary energy values.

[0022] We further propose a diagnostic method as previously described, further comprising the steps of: produce a spectral representation of the received audio signal; detect peaks in the spectral representation; the monitoring parameters being obtained from pairs each comprising a frequency or a sub-band of frequencies, and an amplitude of a peak at said frequency or in said sub-band of frequencies.

[0023] We further propose a diagnostic method as previously described, further comprising the steps of: performing a plurality of acquisitions of the received audio signal; for each frequency or sub-band of frequencies, performing a third filtering on the amplitudes of the peaks associated with said frequency or sub-band of frequencies to produce a filtered peak amplitude associated with said frequency or sub-band of frequencies, the monitoring parameters being the filtered peak amplitudes.

[0024] We further propose a diagnostic method as previously described, comprising the step of storing in a long history, for each frequency or sub-band of frequencies, an average of a predetermined number of the lowest filtered peak amplitudes.

[0025] We also offer electrical equipment including: a processing unit comprising an NPU; at least one internal microphone, and / or means for connection to at least one external microphone; electrical components other than microphones or loudspeakers; the processing unit being arranged to implement the diagnostic method as previously described, and the NPU being arranged to carry out at least the execution of the inference of the machine learning model.

[0026] We further propose electrical equipment as previously described, the electrical equipment being a decoder box.

[0027] Further provided is a computer program comprising instructions which cause the processing unit of the electrical equipment as previously described to execute the steps of the diagnostic method as previously described.

[0028] Further provided is a computer-readable recording medium on which the computer program as previously described is recorded.

[0029] The invention will be better understood in light of the following description of particular non-limiting embodiments of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Reference will be made to the attached drawings, including: [ Fig. 1 ] there figure 1 schematically represents a decoder box in which the invention is implemented; [ Fig. 2 ] there figure 2 is a graph representing a spectrogram including abnormal noise; [ Fig. 3 ] there figure 3 represents steps of a diagnostic method according to a first embodiment of the invention; [ Fig. 4 ] there figure 4 is a graph comprising a curve of the frequency response of a microphone; [ Fig. 5 ] there figure 5 represents steps in post-classification data processing; [ Fig. 6 ] there figure 6 represents steps of a diagnostic method according to a second embodiment of the invention; [ Fig. 7 ] there figure 7 is a graph comprising a curve of a frequency representation obtained by implementing an FFT on the received audio signal; [ Fig. 8 ] there figure 8 is a graph comprising a spectrogram obtained for a received audio signal similar to that of the figure 7 . DETAILED DESCRIPTION OF THE INVENTION

[0031] In reference to the figure 1 , an audio-video decoder box 1 (or Set-Top Box, in English) allows the transmission of an audio-video stream Fav, produced by a source 2, to one or more external devices 3 which reproduce the audio-video stream Fav.

[0032] Source 2 is for example a broadcasting network. The broadcasting network can be a satellite television network, an Internet connection, a digital terrestrial television (DTT) network, a cable television network, etc. Source 2 can also be another device connected to the decoder box 1, and for example a CD, DVD or BlueRay player. Source 2 can also be a storage medium such as a USB key or a memory card connected to the decoder box 1, or even a storage medium internal to the decoder box 1.

[0033] External equipment 3 here includes a TV and external speakers.

[0034] The decoder box 1 firstly comprises a processing unit 4.

[0035] The processing unit 4 comprises one or more processing components, and for example any processor or microprocessor, general or specialized (for example a DSP, for Digital Signal Processor, or a GPU, for Graphics Processing Unit ), a microcontroller, or a programmable logic circuit such as an FPGA (for Field Programmable Gate Arrays ) or an ASIC (for Application Specific Integrated Circuit ).

[0036] Here, among this or these processing components, we find an NPU 5 (for Neural Processing Unit, which can be translated as artificial intelligence accelerator), which is a microprocessor specially adapted to the implementation of artificial intelligence applications.

[0037] The processing unit 4 also comprises memories 6. At least one of these memories 6 forms a computer-readable storage medium, on which at least one computer program is recorded comprising instructions which cause the processing unit 4 to execute at least some of the steps of the diagnostic method according to the invention. One of these memories 6 can be integrated in the NPU 5 or in another processing component.

[0038] The decoder box 1 comprises one or more internal speakers 7, and a sound transmission / reception module 8 which is here integrated into the processing unit 4. The decoder box 1 is thus capable of reproducing so-called “useful” sound signals Su from emitted audio signals Sae produced by the sound transmission module 8 and originating for example from the audio-video stream Fav. By “useful” sound signals, we mean sounds emitted voluntarily, for example music.

[0039] The emission of useful sound signals Su is possibly synchronized with the sound emission carried out by external equipment 3.

[0040] The decoder box 1 further comprises one or more internal microphones 10, which enable it to implement a voice recognition function. When the user pronounces a voice signal Sv comprising a keyword, this voice signal is captured by the internal microphones 10, which then produce a received audio signal Sar. The received audio signal Sar is analyzed by the sound transmission / reception module 8 which extracts voice commands from the received audio signal Sar. The decoder box 1 then performs the actions corresponding to the voice commands. Alternatively, the received audio signal Sar can be transmitted by the decoder box 1 to remote servers via the Internet. The remote servers analyze the received audio signal Sar, extract the voice commands, and transmit the commands contained in the voice signal Sv to the decoder box 1, which then performs the actions corresponding to the voice commands.

[0041] The decoder box 1 also integrates an echo cancellation module 11 (AEC, for Acoustic Echo Cancellation ), which is here integrated into the sound transmission / reception module 8. Indeed, when a user pronounces a voice signal Sv comprising a voice command and, simultaneously, the decoder box 1 reproduces a useful sound signal Su (music for example), the internal microphones 10 of the decoder box 1 capture the ambient sound signal Sa which comprises the voice signal Sv emitted by the user and the useful sound signal Su emitted by the internal speakers 7 (and possibly by the speakers of the external equipment 3). The received audio signal Sar produced by the internal microphones 10 therefore comprises both the voice signal Sv and the useful sound signal Su, which degrades the quality of voice recognition. The interpretation of the voice command is therefore likely to be disturbed by the echo phenomenon.

[0042] The echo cancellation module 11 therefore acquires the transmitted audio signal Sae and the received audio signal Sar produced by the internal microphones 10. The echo cancellation module 11 subtracts the transmitted audio signal Sae from the received audio signal Sar to provide a “cleaned” resulting signal Sr, as close as possible to the voice signal Sv (and not containing the useful sound signal Su). It is this resulting signal Sr which is analyzed to detect the keyword and to perform voice recognition.

[0043] To implement the functions that have just been described, and all the other functions performed by a “conventional” decoder box, the decoder box 1 comprises a plurality of electrical components 12 other than microphones or loudspeakers. The diagnostic method according to the invention aims to monitor these electrical components 12. The monitoring is based on the analysis of the parasitic sound signal Sp produced by these electrical components 12. The diagnostic method according to the invention therefore does not aim to analyze the transmission quality of the internal loudspeakers 7 or the reception quality of the internal microphones 10.

[0044] We know that components such as coils or capacitors can emit operating noise when current passes through them.

[0045] Any electronic device using coils, inductors, solenoids, self-inductors, transformers (these components are all based on a coil, and may have different names depending on their task on the circuit) and any other component involving the laws of electromagnetism described by Maxwell's equations, are affected by this phenomenon.

[0046] Parasitic noise is often of low amplitude and may be present during "normal" operation, but the phenomenon tends to increase as the equipment wears out. When the noise comes from a coil-based element, it is often called in English « c o il whine » (could be translated as “coil hiss”, although it is not always a hiss).

[0047] The noise of a coil is related to the principle of mechanical resonance produced in the coil at the frequency of the current flowing through it. From a technical point of view, a coil is a winding of a conductive wire around a core made of ferromagnetic material (also called a magnetostrictive bar). If the frequency of the signal (current) is in resonance with the wire around the core, this causes ultra-fast micro-vibrations of the coil due to magnetostriction (which is the phenomenon of deformation of a ferromagnetic body subjected to the influence of magnetic fields).

[0048] Noise can be constant during certain periods of operation, or it can be intermittent. For example, a suddenly high current draw can cause a one-off noise (a "tick" type noise). It can also happen that a "heavy" processing performed punctually and briefly by a processor, and which is repeated, causes a sequence of spaced noises ("tick...tick...tick").

[0049] For example, it may happen that the decoding of I (intra) frames of a video stream, due to the very large size of these frames, causes a one-off load call at the level of the SoC power stages (for " System On a Chip ", which can be translated as "system on chip"), due to intensive use of the arithmetic decoding units of the video stream (when CABAC type entropy coding is used). It is sometimes possible that this causes noise emitted by the coils at each I-frame decoding, which will cause a noise of almost fixed frequency, but intermittent.

[0050] These noises occur, in absolute terms, throughout the entire operation of the decoder box 1. However, they are most of the time inaudible to a human, either because their frequency is outside the audible spectrum (which goes from approximately 20Hz to approximately 20kHz), or because their amplitude is too low to be heard at a normal operating distance from the decoder box 1.

[0051] When the current varies, the electromagnetic field causes a variation in the forces applied to the coil, leading to a succession of deformations of the latter. In extreme cases, this can lead to the destruction of the coil ("coil breakdown"), but most of the time it simply causes vibration. The effect is all the more pronounced as the signal passing through the coil increases and approaches its magnetic resonance frequency.

[0052] A change in the operating noise over time, such as a different frequency or a higher amplitude, may indicate an upcoming failure of the decoder box 1. The fault may be linked to the electrical component 12 emitting the noise, but may also be linked to a failure or aging of another electrical component 12, leading to a change in operation of the component emitting the noise. Thus, a sound anomaly, present in a parasitic sound signal Sp emitted by one or more first electrical components (belonging to the electrical components 12), may result from a current or future failure of one or more second electrical components (also belonging to the electrical components 12) distinct from the first electrical component(s) 12.

[0053] Here, by "component failure" is meant non-nominal operation of a component, which potentially leads to present or future non-nominal operation of the set-top box 1. The component may remain operational, but it has at least one characteristic which no longer complies with its technical specification or the variation of which will ultimately make it non-compliant with its technical specification.

[0054] The failure can be current or future. The diagnostic process can therefore detect a failure that is present at the time of detection, or that will occur in the future.

[0055] We see for example on the figure 2 a spectrogram including an abnormal noise B produced by a coil. The frequency of this noise is close to 13kHz.

[0056] The diagnostic method consists of capturing the ambient sound signal Sa, and analyzing the parasitic sound signal Sp included in the ambient sound signal Sa and emitted by one or more of the electrical components 12, so as to detect a sound anomaly resulting from a failure of one or more electrical components 12 of the decoder box 1. By "parasitic sound signal", we therefore mean a sound signal emitted fortuitously, and therefore the emission of which is not desired. As we have just seen, the sound anomaly may be for example an increase in the intensity of the parasitic sound signal, but also for example a decrease in the amplitude of the parasitic sound signal.

[0057] First of all, a diagnostic method according to a first embodiment of the invention is described.

[0058] In reference to the figure 3 , the diagnostic method begins with an audio acquisition step (step E1). The internal microphones 10 capture the ambient sound signal Sa, and the sound transmission / reception module 8 of the processing unit 4 acquires the received audio signal Sar produced by the internal microphones 10.

[0059] For each internal microphone 10, it is preferable that the sampling rate be high (44.1kHz, 48kHz, or more), in order to allow the detection of high-frequency sounds. As is known in the field of signal processing, the sampling frequency should theoretically be twice the highest frequency that one wishes to capture. The sound transmission / reception module 8 includes low-pass filters to prevent aliasing, which have a transition band of a certain frequency width. It is therefore, in practice, only possible to correctly capture sounds up to a frequency slightly below half the sampling frequency. For example, it is estimated that a sampling frequency of 44.1kHz can capture sounds up to 20 (upper limit of undegraded human hearing) or 21kHz.A sampling frequency of 48kHz or more will therefore make it possible to capture sounds that are inaudible to a human, but which may still be representative of a component defect.

[0060] It is obviously necessary to ensure that these sounds are also within the sensitivity range of the internal microphones 10: if, for example, the internal microphones 10 can only pick up signals up to 23kHz, a sampling frequency of 88.2kHz would be useless, and we would then be limited to sampling at 48kHz.

[0061] The internal microphones 10 may be digital output microphones, and may therefore each incorporate an analog-to-digital converter. The internal microphones 10 may be of the MEMS type.

[0062] It is noted that the capture of the ambient sound signal Sa can also be carried out by means of one or more external microphones, which may be individual devices or which may be integrated into other devices allowing access to the recordings. The decoder box 1 then comprises means of connection to the external microphone(s), which comprise for example a jack or USB socket, or any type of means of communication with the devices integrating these external microphones.

[0063] Internal microphones 10 are however preferable, due to their immediate proximity to the noises / sounds that one wishes to capture.

[0064] It is common for the noise emitted by the electrical components 12 of the decoder box 1 to change depending on the use made of the decoder box 1 (the use influencing the current required).

[0065] Advantageously, the processing unit 4 therefore performs several captures and recordings of the ambient sound signal Sa, under different conditions of use, for example when the decoder box 1 is not used much and when the decoder box 1 is used more (example: decoding of several video streams and display of these, such as when the PiP mode, for Picture In Picture , is activated).

[0066] The duration of the recordings made must allow the use of detection modules (and therefore must not be too short), while avoiding the analysis being too "heavy" in terms of processing. A duration of approximately 1s is a good compromise, although other durations are possible.

[0067] The frequency of measurements is defined as follows. Since the aging of electrical components 12 is normally a very gradual phenomenon (with the notable exception of the imminent failure of a capacitor, causing a whistling noise very shortly before the breakdown), it is reasonable to carry out measurements fairly spaced out, for example once a week.

[0068] Many microphones have a frequency response that is not constant. For example, we see on the figure 4 the frequency response of a known microphone. This frequency response increases up to about 200Hz, then remains constant, then increases from about 3kHz.

[0069] Advantageously, the processing unit 4 therefore implements compensation for the received audio signal Sar by applying an equalization to the received audio signal Sar, in order to restore a more “flat” frequency response, particularly in the high frequencies (where component noise is most likely to occur): step E2.

[0070] The processing unit 4 then detects whether, at the time of capturing the ambient sound signal Sa, the decoder box 1 is or is not in the process of reproducing a useful sound signal Su: step E3.

[0071] If the decoder box 1 is reproducing a useful sound signal Su, via the internal speakers 7 or via a sound output (for example an analog or digital audio output, or via the HDMI signal intended for the television), it is preferable to eliminate the useful sound signal Su from the received audio signal Sar to better detect a potential sound anomaly in the received audio signal Sar.

[0072] The received audio signal Sar and the transmitted audio signal Sae are therefore applied to the input of the echo cancellation module 11, in order to eliminate as much as possible the transmitted audio signal from the received audio signal, and thus obtain a resulting audio signal Sr: step E4.

[0073] It should be noted that some echo cancellation modules are dedicated to further processing of voice, and are therefore limited in output to a signal sampled at 8kHz. It is therefore very likely that the decoder box 1 can pick up a wideband signal when the audio playback function is not active, but that it is satisfied with a more limited band signal when the audio playback function is active.

[0074] The data from the recording are initially temporal representations, of the PCM type ( Pulse Code Modulation ).

[0075] This type of representation is not very suitable for frequency analysis, and it is therefore appropriate, following step E3 or step E4 (depending on whether the echo cancellation module 11 is used or not), to transform the received audio signal (temporal) into a frequency representation: step E5.

[0076] The frequency representation normally has two components per frequency: phase and amplitude. Here, the phase does not provide useful information for the diagnostic process, and the processing unit 4 only retains the amplitude information as a function of frequency.

[0077] The processing unit 4 then extracts monitoring parameters from the received audio signal Sar, the monitoring parameters being representative of the parasitic sound signal Sp included in the ambient sound signal Sa and emitted by at least one of the electrical components 12: step E6. Then, the processing unit detects from the monitoring parameters a sound anomaly resulting from a failure of at least one electrical component 12 of the decoder box 1: step E7.

[0078] The monitoring parameters here form a spectrogram of the received audio signal Sar (or of the resulting signal Sr): for each recording made of the received audio signal, a spectrogram is therefore produced.

[0079] Sound anomaly detection from monitoring parameters is detection by classification based on spectrogram image.

[0080] Classification uses an inference model. However, it is necessary that the spectrogram, forming the input data of the inference model, always covers the same frequency range, otherwise the classification based on the spectrogram image cannot work correctly.

[0081] However, as we have just seen, it is possible that the echo cancellation module 11 was used to process the received audio signal Sar: this is the case if the decoder box 1 was in the process of reproducing a useful sound signal Su when capturing the ambient sound signal Sa (the monitoring parameters are then obtained from the resulting signal Sr).

[0082] However, if the echo cancellation module 11 has been activated, there are two possible types of resulting signal Sr at the output of the echo cancellation module 11: a signal having the same sampling frequency as the received audio signal Sar, but limited in bandwidth, or a signal having a reduced sampling frequency compared to the received audio signal Sar.

[0083] In the case where the resulting signal Sr is bandwidth-limited, no additional processing is required. On the other hand, in the case where the resulting signal Sr has a reduced sampling frequency, an additional step is necessary to obtain a correctly “classifiable” signal: the sampling frequency must be increased, to bring it back to that of the received audio signal (in order to always have signals with the same sampling frequency). In this case, the processing unit 4 resamples the resulting signal Sr, so as to obtain a resampled signal having a sampling frequency equal to that of the received audio signal: step E8. The monitoring parameters are then obtained from the resampled signal.

[0084] This step E8 is carried out before the calculation of the spectrogram (and therefore for example between step E5 and step E6), or after the calculation of the spectrogram (and therefore for example between step E6 and step E7) by then “completing” the image of the spectrogram (with blank data) in the high frequencies.

[0085] We are now particularly interested in step E7.

[0086] Sound anomaly detection consists of performing an inference of a previously trained machine learning model, using the monitoring parameters as input data. The monitoring parameters here are so-called "current" spectrograms, each current spectrogram being associated with a recording of the ambient sound signal Sa (and therefore of the received audio signal Sar or the resulting signal Sr). The sound anomaly is detected based on at least one output value obtained by performing said inference.

[0087] The classification here is a classification of images and not of audio or sound signals.

[0088] For example, we use the library TensorFlow Lite, which is a dedicated inference version, optimized for embedded environments.

[0089] Inference is run on the NPU 5 via APIs (for Application Programming Interface which we translate as application programming interface) dedicated (such as NNAPI under Android).

[0090] Using the library TensorFlow Lite, the transformation into a current spectrogram of the received audio signal Sar can for example be carried out using the following code: def get_spectrogram(waveform,padding=False, min_padding=48000): waveform = tf.cast(waveform, tf.float32) spectrogram = tf.signal.stft(waveform, frame_length=2048, frame_step=512, fft_length=2048) spectrogram = tf.abs(spectrogram) return spectrogram

[0091] This step then corresponds to the frequency transformation step E5.

[0092] In this section of code, tf.signal.stft allows you to perform a short-term Fourier transform ( Short Term Fourier Transform ), and tf.abs removes the phase information, keeping only the amplitude information.

[0093] The NPU 5 therefore applies an inference model to the current spectrograms. The inference model was obtained by pre-training the learning model.

[0094] Training requires a database. This database includes recordings obtained from a corpus of recordings of "healthy" devices and a corpus of recordings of "failing" devices. The failing devices include, for example, coil noises, as previously described, at different frequencies and amplitudes. The "failing" corpus can consist of recordings of truly failing devices and recordings of healthy devices to which a "failing" signal has been superimposed (a technique known as dataset augmentation). The corpus includes both static (or quasi-static) and transient / intermittent failure signals (as described above).

[0095] As seen, the data used to produce the current spectrograms can be obtained either from the received audio signals themselves (possibly processed), or from resulting signals Sr obtained by applying the echo cancellation module 11 to the received audio signals Sar.

[0096] It is therefore appropriate: either to have in the corpus, forming the training database, recordings with and without AEC, or to use two inference models, that is to say a model with use of the echo cancellation module 11, and a model without use of said module. Thus, if the received audio signal Sar has been applied as input to the echo cancellation module 11, the NPU 5 uses a first inference model. Otherwise, the NPU 5 uses a second inference model.

[0097] If we opt for this second possibility, it is no longer necessary to extend the spectrum of recordings made at the AEC output (resampling carried out in step E8). The quality of the detection will be better with this second possibility, the two models then being used systematically on data that better correspond to the characteristics of each of the training sets.

[0098] The trained model can be a custom model. It is also possible to use transfer training ( transfer learning ) on a pre-trained model.

[0099] The machine learning model here is an artificial neural network model.

[0100] The artificial neural network model used is, for example, of the convolutional neural network type (CNN, for Convolutional Neural Network ), For example MobileNet Or InceptionNet.

[0101] The model could also, for example, be of the multi-layer perceptron type (MPL, for MultiLayer Perceptron ) .

[0102] CNNs are more suitable for image classification than MLPs, but the latter are also very effective.

[0103] It is noted that it is not possible to directly use a “usual” pre-trained model, because the characteristics (types of sounds) that the invention aims to detect are not part of the “usual” sounds of the usual pre-trained models, and therefore the pre-trained models do not have a classifier for the defects that can be detected.

[0104] The obtained inference model is then converted into a model TensorFlow lite, in order to be able to be used on the set-top box 1. In order to improve the execution speed of the model and reduce its size, it may be advantageous to reduce its precision by requantizing its parameters, in order to reduce the necessary precision (for example, passing 32-bit floating-point values into 16-bit floating-point values, or into integer values).

[0105] The result of running the inference, on a current spectrogram corresponding to a recording of a particular Sar received audio signal, is therefore the result of a classification of said recording, i.e. a "probability" that the recording belongs to the "Healthy" category, and another "probability" of belonging to the "Faulty" category.

[0106] In reality, it is not a probability in the statistical / mathematical sense, but a "weight" indicating a more or less probable membership in the categories.

[0107] We thus obtain a pair of output values for the variables “Healthy” and “Failing”.

[0108] For example, the result of running the inference is: Sain -> 0.850647 Défaillant -> 0.060796

[0109] We see that the sum of the two output values is not equal to "1", as would be the case if the values expressed a mathematical probability.

[0110] The processing unit 4 can, to normalize the output values so that the sum of the values from the classifications is equal to 1, apply a function of type Softmax on the output values. The resulting values will then correspond to a probability.

[0111] We now describe how the output values are used: step E9.

[0112] It is possible to use only the output values from the last recording made to determine whether set-top box 1 should be considered faulty or at risk of failure soon.

[0113] Advantageously, however, the processing unit 4 uses a plurality of acquisitions and recordings of the received audio signal, and performs an inference of the learning model for each acquisition. Then, the processing unit 4 filters the output values obtained to obtain at least one filtered value. The processing unit 4 then compares the filtered value and / or a variation of the filtered value with predefined thresholds to detect the sound anomaly.

[0114] Using a plurality of recordings (e.g. 3 or 5) and filtering helps ensure that the noise is not coming from an intermittent external element.

[0115] The processing unit 4 can thus, for example, emit an alarm if the output value of the variable Défaillant, for at least M of the last N records, has a value greater than a predetermined threshold S (e.g., M = 2 and N = 3, or M = 3 and N = 5, and S = 0.2).

[0116] The processing unit 4 can also store output values obtained over a relatively long predefined period, for example equal to 1 month or 6 months, to enable the evolution of the output values to be analyzed.

[0117] The processing unit 4 analyses the evolution of the output values over the predefined duration, which makes it possible to detect a progressive degradation in the operation of one or more of the monitored components, which could possibly result in a future failure of the decoder box 1.

[0118] In reference to the figure 5 , processing unit 4 implements the following processing, which combines the strategies just described.

[0119] The output values are obtained following the execution of the inference of the machine learning model and therefore the application of the inference model (step E10).

[0120] These values are then filtered to obtain a filtered output value "filtered_failure_value": step E11. Many filtering strategies are possible, the ultimate goal being to reduce the potential impact of external noise captured during recording, which could influence the classification results.

[0121] Filtering possibly includes the steps of calculating a mean or median of output values.

[0122] Filtering can consist, for example, of keeping the K lowest values of the "Failed" output value (e.g., K = 2), then calculating an average of these K values, and assigning this average to the filtered value "filtered_failure_value". The advantage of this filtering is its simplicity.

[0123] Alternatively, filtering can consist of calculating the median of the K lowest values of the "Failed" output value, and assigning this median to the "filtered_failure_value" variable. This filtering helps to better reduce the impact of spurious captures.

[0124] The processing unit 4 then compares the filtered value with a first predefined threshold: step E12. The first predefined threshold is, for example, equal to 0.2. If the filtered value is less than or equal to the first predefined threshold, the processing unit 4 does not generate an alarm: step E13.

[0125] On the other hand, in step E12, if the filtered value is strictly greater than the first predefined threshold, the processing unit 4 compares the variation (here the increase) of the filtered value over a first predefined duration with a second predefined threshold: step E14. This second comparison makes it possible to avoid raising an alarm for low values (which will have a greater tendency to cause a significant relative variation). If the increase in the filtered value over the first predefined duration is strictly less than the second predefined threshold, the processing unit 4 does not generate an alarm: step E13. On the other hand, if the increase in the filtered value over the first predefined duration is greater than or equal to the second predefined threshold, the processing unit 4 generates an alarm: step E15. The second predefined threshold is for example equal to 25% and the first predefined duration to 30 days.

[0126] Note that the output values resulting from the execution of the inference are not absolute, but depend on the model itself. Thus, in the event of a modification or update of the model, output values obtained by a new model can no longer be directly compared to those obtained by the application of a previous model. Processing unit 4 must therefore not directly compare such heterogeneous data.

[0127] Two main ways can be used to avoid this pitfall: in case of model update, processing unit 4 only uses new data to make comparisons; in case of model update, if previous records have been kept, processing unit 4 submits these previous records to the new model, in order to obtain new resulting values.

[0128] The output values obtained can be stored locally, or can be sent by the decoder box 1 to a "head-end", i.e. to servers managed by the network operator. Data can be sent via known means, such as a TR-069 type protocol, or simply via an http request from the decoder box 1 to a server of the operator.

[0129] It is thus possible to analyze the results of the diagnostic process obtained for many decoder boxes, and, for example, to compare the results obtained for a particular decoder box with the overall aging of the fleet.

[0130] Optionally, the operator (or his technical support) is able to interrogate the decoder box 1 so that it transmits recordings used for detection, for various purposes such as for example: detection with more precise models; detection using different means; analysis by an expert; enrichment of the detection model.

[0131] In the event of a probable future breakdown (or exceeding an alarm threshold), it will then be possible to choose whether to communicate this information to the user or make it available to technical support.

[0132] Note that the detection inference model can be updated, for example by downloading from an operator server (via HTTP) or downloading from a carousel (DVB type data).

[0133] We now describe, with reference to the figure 6 , a second embodiment of the diagnostic method according to the invention. The detection of the sound anomaly is based on the evolution of the sound emission of the monitored components, by comparing the current sound spectrum with the sound spectrum of an old measurement (previous measurement), or of a “default” measurement, determined during the manufacture or design of the decoder box 1, representing that of a decoder box determined as “healthy” (reference measurement).

[0134] The diagnostic method begins with an audio acquisition step (step E20). The internal microphones 10 capture the ambient sound signal Sa, and the processing unit 4 acquires the received audio signal Sar produced by the internal microphones 10.

[0135] Advantageously, the processing unit 4 carries out a plurality of acquisitions and recordings of the received audio signal Sar, in order to ensure that the perceived noise does not come from an intermittent external element.

[0136] Advantageously, the processing unit 4 implements compensation of the received audio signal Sar by applying an equalization to the received audio signal Sar: step E21.

[0137] The processing unit 4 then detects whether, at the time of capturing the ambient sound signal Sa, the decoder box 1 is in the process of reproducing a useful sound signal Su: step E22.

[0138] If the decoder box 1 is transmitting a useful audio signal Su, the received audio signal Sar and the transmitted audio signal Sae are applied to the input of the echo cancellation module 11, in order to eliminate as much as possible the transmitted audio signal from the received audio signal: step E23.

[0139] Following step E22 or step E23 (depending on whether the echo cancellation module 11 is activated or not), for each acquisition and recording, the processing unit 4 decomposes the received audio signal Sar into elementary audio signals on frequency sub-bands: step E24. The processing unit 4 then calculates an elementary energy value of each elementary audio signal.

[0140] To do this, the processing unit 4 can apply a filter bank to the “original” received audio signal, i.e. to the samples, in the time domain, which constitute the received audio signal.

[0141] The processing unit 4 can also transform the audio signal received in the time domain into a signal in the frequency domain, in which case it is sufficient to group the coefficients according to the defined sub-bands. A larger number of sub-bands allows for a more detailed analysis, but makes the calculations more complex. Sub-bands with a width of 1 kHz or 2 kHz provide a good compromise.

[0142] For each acquisition, the processing unit 4 therefore obtains a list of frequency sub-bands each associated with an elementary energy value.

[0143] For each frequency sub-band, the processing unit 4 performs a second filtering on the elementary energy values associated with said frequency sub-band (and originating from the plurality of recordings) to produce a filtered elementary energy value associated with said frequency sub-band: step E25.

[0144] The monitoring parameters, obtained from the elementary energy values, are the filtered elementary energy values: step E26.

[0145] The second filtering can consist, for the processing unit 4, in using a predefined number (5 for example) of recordings spaced by a predefined duration (30 minutes for example). Then, for each frequency sub-band, the processing unit 4 calculates the median value of the elementary energy values, and only keeps this median value. This makes it possible to greatly reduce the influence of a "parasitic" capture, which will have only a very limited influence on the median (unlike an average, which would be much more influenced by potential non-representative values).

[0146] The second filtering may consist, for processing unit 4, of using a centered average that combines better resistance to noise than the median and better resistance to extreme values than the mean. For example, if processing unit 4 has made five recordings, processing unit 4 eliminates the minimum and the maximum and calculates the average of the other three measurements, or eliminates the two measurements furthest from the median and calculates the average of the three closest measurements.

[0147] Then, the processing unit 4 detects from the monitoring parameters a sound anomaly resulting from a failure of at least one electrical component 12 of the decoder box 1: step E27.

[0148] The detection step consists of comparing, for each frequency sub-band, the filtered elementary energy value with a third predefined threshold, and / or consists of comparing, for each frequency sub-band, a variation of the filtered elementary energy values over a second predefined duration with a fourth predefined threshold to detect the sound anomaly.

[0149] The processing unit 4 can therefore carry out either one of the two comparisons, or both, and detect a sound anomaly if only one threshold is exceeded, or if both thresholds are exceeded.

[0150] Following detection, the values obtained can be stored locally, or sent back by decoder box 1 to the head-end: step E28.

[0151] A third embodiment of the diagnostic method according to the invention is now described.

[0152] This time, processing unit 4 calculates a fast Fourier transform (or FFT, for Fast Fourier Transform ) of the received audio signal Sar or the resulting signal Sr, and only keeps the amplitude values of the signal in the frequency domain (the phase component of the FFT is not used here).

[0153] The processing unit 4 therefore performs the detection from a simple frequency representation of the received audio signal Sar or the resulting signal Sr, similar to the frequency representation visible on the figure 7 . On this figure 7 , we observe a noise at the frequency of 19kHz.

[0154] The processing unit 4 then detects peaks in the spectral representation. The processing unit 4 then keeps a list of detected peaks, as well as their value (which can be likened to a very simplified version of spectral envelope calculation).

[0155] The monitoring parameters are obtained from pairs each comprising a frequency or a sub-band of frequencies, and an amplitude of a peak at said frequency or in said sub-band of frequencies.

[0156] For example, processing unit 4 only keeps a predetermined number N of the most important peaks, for example the 3 or 5 most important peaks. Processing unit 4 therefore obtains a list of N [frequency; amplitude] pairs.

[0157] The obtained pairs are for example the following: [[8.24kHz; -72dB]; [12.65kHz; -43dB]; [16.64kHz; -68dB]].

[0158] The detection step again consists of comparing the monitoring parameters with third predefined thresholds and / or comparing variations in the monitoring parameters, over a second predefined duration, with fourth predefined thresholds to detect the sound anomaly.

[0159] Thus, for example, for each pair, the processing unit 4 compares the value of the amplitude of the peak with a third predefined threshold and detects a sound anomaly if said value is greater than the third predefined threshold.

[0160] Alternatively, for example, for each pair, the processing unit 4 compares the variation, over the second predefined duration, of the value of the amplitude of the peak with a fourth predefined threshold, and detects a sound anomaly if said variation is greater than the fourth predefined threshold.

[0161] These different detection methods can be applied to a simple frequency representation (a single value averaged per frequency for the entire duration of the captured sample), or to a frequency representation showing the temporal evolution, of the spectrogram type.

[0162] Such a spectrogram, obtained for a signal similar to that which made it possible to obtain the frequency representation visible on the figure 7 , is visible on the figure 8 .

[0163] To meet the detection objectives, the duration of the sample captured being relatively short (for example of the order of a second or less), and as the defects to be detected present relatively constant characteristics over such a period of time, a simple frequency representation will have the advantage, compared to a spectrogram, of reducing the calculations necessary for detection, and of carrying out time averaging allowing the values to be filtered (reduction of signal noise).

[0164] The processing of data from detection can take several forms.

[0165] The processing unit 4 can detect a sound anomaly and generate an alarm based solely on the last recording made: as soon as an anomaly signal is produced, the processing unit 4 detects a sound anomaly.

[0166] Preferably, the processing unit 4 uses several successive recordings, in order to ensure that the noise does not come from an intermittent external element.

[0167] For each frequency or sub-band of frequencies, the processing unit 4 performs a third filtering on the amplitudes of the peaks associated with said frequency or sub-band of frequencies to produce a filtered peak amplitude associated with said frequency or sub-band of frequencies, the monitoring parameters being the filtered peak amplitudes.

[0168] The processing unit can thus take into account a predefined number N of the latest recordings to detect a sound anomaly; for example, N=3 or 5.

[0169] Thus, if an anomaly signal is produced for each of the last N recordings, or for at least M recordings out of the last N recordings, an audio anomaly is detected. For example, the processing unit 4 generates an alarm if an anomaly signal is produced for at least 2 of the last 3 recordings (i.e. if one of the peaks has an amplitude greater than -50dB on 2 of the last 3 recordings).

[0170] The processing unit 4 can also detect an anomaly based on the evolution of the monitoring parameters.

[0171] The processing unit 4 can then store the peaks and frequencies over a long history of predefined duration, for example several weeks (or even several months). The processing unit 4 uses this long history of the values resulting from the detection, to then enable the analysis of their evolution.

[0172] The increases in monitoring parameters detected over time (evolution over several weeks) will make it possible to characterize a degrading decoder box, which may possibly be subject to imminent failure.

[0173] For a given date, the processing unit 4 keeps, for example, in the long history, for each frequency or sub-band of frequencies, an average of a predetermined number of the lowest filtered peak amplitudes.

[0174] Each frequency sub-band is for example defined by a small frequency variation, for example + / -500Hz, around a central frequency (i.e. a total range of 1000Hz).

[0175] For example, assume that processing unit 4 has made three recordings which give the following measurements: [[8.24kHz; -72dB] ; [12.65kHz; -43dB] ; [16.64kHz; -68dB]] [[7.11kHz; -78dB] ; [12.38kHz; -41dB] ; [13.25kHz; -30dB]] [[6.08kHz; -71dB] ; [12.42kHz; -43dB] ; [16.01 kHz; -65dB]]

[0176] The first set of values (around 7kHz) does not have at least 2 values in a range of + / -500Hz, so it is not kept.

[0177] The second set of values (around 12kHz) presents 3 values located in a range of + / -500Hz. Processing unit 4 will therefore retain the average of the 2 lowest values ([12.35kHz; -43dB] and [12.42kHz; -43dB]), and therefore [12.54kHz; -43dB]. Note that dB are logarithmic values, and that it is therefore not possible to directly calculate a linear average of values in dB. It is first necessary to linearize the values: X _ lin = 10 ∧ X _ dB / 10 , then average the linear values, then convert the average to dB: X _ dB = 10 * log X _ lin .

[0178] The third set of values only has 2 values located in a range of + / -500Hz. Processing unit 4 will therefore retain the average of these 2 values ([16.64kHz; - 68dB] and [16.01kHz; -65dB]), which gives [16.33kHz; - 66dB].

[0179] The third value ([13.25kHz; -30dB]), transient, is probably due to external noise (such as another device nearby), and is filtered by the selection method.

[0180] In this example, processing unit 4 will therefore retain the values in the long history: [[12.54kHz; -43dB]; [16.33kHz; -66dB]]

[0181] The processing unit 4 generates an alarm if, for example, one of the amplitudes of the peak associated with a frequency sub-band increases by more than X%.

[0182] In order to avoid raising an alarm for low values (which will tend to cause a large relative variation), a comparison with a threshold can be added.

[0183] For example, the processing unit 4 produces an alarm message if, for a given frequency band (for example + / -500Hz around a given central frequency), the amplitude of the peak in said frequency band increases by at least 10% over a period of 30 days.

[0184] Other strategies can be applied (instead or in addition), such as generating an alarm if a value (after applying a filtering method as previously described) is greater than -40dB.

[0185] Of course, the invention is not limited to the embodiments described but encompasses any variant falling within the scope of the invention as defined by the claims.

[0186] The electrical equipment in which the invention is implemented is not necessarily a decoder box, but can be any electrical equipment comprising or capable of being connected to one or more microphones: connected speaker, computer, smartphone, game console, etc.

[0187] The architecture of the electrical equipment may of course be different from that described here.

[0188] The NPU can perform only the application of the inference model, but also implement another or other steps of the diagnostic process. The processing unit does not necessarily include an NPU. The application of the inference model is therefore not necessarily performed in an NPU, but could be performed in any type of suitable component (DSP, GPU, etc.).

Claims

1. Diagnostic method of electrical equipment (1) which comprises: a processing unit (4); at least one internal microphone (10), and / or means for connecting to at least one external microphone; electrical components (12) other than microphones or speakers; the diagnostic method being implemented at least partially in the processing unit (4) and comprising the steps of: acquiring a received audio signal (Sar) produced from capturing an ambient sound signal (Sa), by the at least one internal microphone or by the at least one external microphone; producing monitoring parameters from the received audio signal, which are representative of an interfering sound signal (Sp) comprised in the ambient sound signal and emitted by at least one of the electrical components (12); detecting a sound anomaly resulting from a current or future failure of at least one electrical component (12) of the electrical equipment from the monitoring parameters, the method being characterized in that it further comprising the steps: of detecting if the electrical equipment (1), at the time of the acquisition step, is in the process of emitting a useful sound signal (Su), which comprises sounds voluntarily emitted; if this is the case, of applying the received audio signal (Sar) at the input of an acoustic echo cancellation module (11), to produce a resulting signal (Sr), the monitoring parameters being obtained from the resulting signal.

2. Diagnostic method according to claim 1, wherein the detection step comprises the steps of: executing an inference of a previously trained automatic training model, by using the monitoring parameters as input data, the automatic training model being a classification model; detecting the sound anomaly according to at least one output value obtained by executing said inference.

3. Diagnostic method according to claim 2, wherein the monitoring parameters form current spectrograms coming from the received audio signal, and wherein the automatic training model has been trained by using a database comprising images representing training spectrograms.

4. Diagnostic method according to any one of claims 2 or 3, wherein the automatic training model is an artificial neural network of the convolutional neural network or multilayer perceptron type.

5. Diagnostic method according to claim 1, further comprising the step, if the received audio signal (Sar) is applied at the input of the acoustic echo cancellation module (11), to increase a sampling frequency of the resulting signal (Sr), so as to obtain a resampled signal having a sampling frequency equal to that of the received audio signal, the monitoring parameters being obtained from the resampled signal.

6. Diagnostic method according to claim 1, comprising the step, if the received audio signal (Sar) has been applied at the input of the acoustic echo cancellation module (11), to use a first inference model, and otherwise to use a second inference model.

7. Diagnostic method according to any one of claims 2 to 6, further comprising the steps of: performing a plurality of acquisitions of the received audio signal (Sar); executing an inference of the training model for each acquisition; performing a first filtering of the output values to obtain at least one filtered value; comparing the filtered value with a first predefined threshold and / or a variation of the filtered value over a first predefined duration with a second predefined threshold to detect the sound anomaly.

8. Diagnostic method according to any one of claims 2 to 7, the processing unit (4) comprising an NPU (5), wherein at least one execution of the inference of the automatic training model is performed.

9. Diagnostic method according to claim 1, wherein the detection step consists of comparing the monitoring parameters with third predefined thresholds and / or of comparing variations of the monitoring parameters, over a second predefined duration, with fourth predefined thresholds to detect the sound anomaly.

10. Diagnostic method according to claim 9, further comprising the steps of: breaking down the received audio signal (Sar) into elementary audio signals on frequency sub-bands; calculating an elementary energy value of each elementary audio signal, the monitoring parameters being obtained from the elementary energy values.

11. Diagnostic method according to claim 10, further comprising the steps of: performing a plurality of acquisitions of the received audio signal; for each frequency sub-band, performing a second filtering on the elementary energy values associated with said frequency sub-band to produce a filtered elementary energy value associated with said frequency sub-band, the monitoring parameters being the filtered elementary energy values.

12. Diagnostic method according to claim 9, further comprising the steps of: producing a spectral representation of the received audio signal; detecting peaks in the spectral representation; the monitoring parameters being obtained from pairs each comprising a frequency or a frequency sub-band, and an amplitude of a peak at said frequency or in said frequency sub-band.

13. Diagnostic method according to claim 12, further comprising the steps of: performing a plurality of acquisitions of the received audio signal; for each frequency or frequency sub-band, performing a third filtering on the amplitudes of the peaks associated with said frequency or frequency sub-band to produce a filtered peak amplitude associated with said frequency or frequency sub-band, the monitoring parameters being the filtered peak amplitudes.

14. Diagnostic method according to claim 13, comprising the step of preserving in a long history, for each frequency or frequency sub-band, a mean of a predetermined number of the lowest filtered peak amplitudes.

15. Electrical equipment comprising: a processing unit (4) comprising an NPU (5); at least one internal microphone (10), and / or means for connecting to at least one external microphone; electrical components (12) other than microphones or speakers; the processing unit (4) being arranged to implement the diagnostic method according to claim 2, and the NPU being arranged to perform at least the execution of the inference of the automatic training model.

16. Electrical equipment according to claim 15, the electrical equipment being a set-top box.

17. Computer program comprising instructions which cause the processing unit (4) of the electrical equipment according to one of claims 15 or 16 to carry out the steps of the diagnostic method according to one of claims 1 to 14, when said program is executed on the processing unit.

18. Recording medium which can be read by a computer, on which the computer program according to claim 17 is recorded.