Security detection method, control device, and storage medium

Abstract

The present application relates to a safety detection method, a control device and a storage medium, wherein the method comprises: emitting a first sound wave signal; receiving a first return sound wave signal, wherein the first return sound wave signal is a reflected sound wave based on the first sound wave signal; obtaining the state of the personnel near the device based on the first return sound wave signal and a trained audio judgment system; and determining whether to open the child lock according to the personnel state and a preset first judgment standard. The present application can improve the safety of household washing machines, prevent safety accidents caused by child misuse, and provide better safety protection and user experience. Compared with the prior art, the present application can monitor the personnel near the device in real time, make intelligent judgments, and have stronger self-defined capabilities in combination with the first judgment standard, thereby improving higher safety and being more user-friendly.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of device security, and specifically provides a safety detection method, a control device and a storage medium. BACKGROUND

[0002] With the popularity and application of household washing machines, there are many use safety problems, such as waste caused by accidental triggering of the washing program, easy to cause faults caused by improper use of the washing machine, and danger caused by children easily getting into the washing machine.

[0003] In the prior art, a long-press unlocking child lock function is generally provided on the washing machine to prevent children from accidentally triggering the washing program or opening the washing machine. However, the prior art can only determine that the child lock is not opened by the child in a long-press unlocking manner, which is still prone to danger.

[0004] Correspondingly, there is a need in the art for a new child lock solution to solve the above problems. SUMMARY

[0005] In order to overcome the above defects, the present application is proposed to provide a solution or at least partially solve the problem of weak child lock recognition ability in the prior art.

[0006] In a first aspect, the present application provides a safety detection method, comprising: emitting a first sound wave signal; receiving a first return sound wave signal, wherein the first return sound wave signal is a reflected sound wave based on the first sound wave signal; obtaining a personnel state near the device based on the first return sound wave signal and a trained audio judgment system; and determining whether to open the child lock according to the personnel state and a preset first judgment standard.

[0007] Alternatively or additionally to the above solutions, in the method according to an embodiment of the present application, the preset first judgment standard is a system preset judgment standard and / or a user defined judgment standard.

[0008] Alternatively or additionally to the above solutions, in the method according to an embodiment of the present application, a feature extraction neural network and a recurrent neural network are used; the obtaining of the personnel state near the device based on the first return sound wave signal and the trained audio judgment system comprises: inputting the first return sound wave signal into the trained feature extraction neural network to obtain a first return sound wave feature vector; inputting the first return sound wave feature vector into the trained recurrent neural network to obtain a first return sound wave time sequence feature vector; obtaining a probability distribution of the personnel state near the device under different conditions based on the first return sound wave time sequence feature vector; and selecting a condition with the highest personnel state probability distribution near the device as the personnel state near the device.

[0009] As an alternative or supplement to the above scheme, in the method according to an embodiment of the present application, the obtaining of the personnel state near the device based on the first returned sound wave signal and the trained audio judgment system comprises: preprocessing the first returned sound wave signal to obtain a first returned sound wave vector; and inputting the first returned sound wave vector into the trained audio judgment system to obtain the personnel state near the device.

[0010] As an alternative or supplement to the above scheme, in the method according to an embodiment of the present application, the preprocessing of the first returned sound wave signal to obtain a first returned sound wave vector comprises: segmenting the first returned sound wave signal to obtain a segmented signal; performing window function processing on the segmented signal to obtain a window function signal; performing Fourier transform on the window function signal to obtain a frequency domain signal; and performing normalization on the frequency domain signal to obtain the first returned sound wave vector.

[0011] As an alternative or supplement to the above scheme, in the method according to an embodiment of the present application, further comprising: collecting environmental noise; and the obtaining of the personnel state near the device based on the first returned sound wave signal and the trained audio judgment system comprises: obtaining a second returned sound wave signal based on the environmental noise and the first returned sound wave signal; and obtaining the personnel state near the device based on the second returned sound wave signal and the trained audio judgment system.

[0012] As an alternative or supplement to the above scheme, in the method according to an embodiment of the present application, the obtaining of the second returned sound wave signal based on the environmental noise and the first returned sound wave signal comprises: performing Fourier transform on the collected environmental noise signal and the first returned sound wave signal respectively to obtain a corresponding first environmental noise signal and a frequency domain signal; subtracting the frequency spectrum of the first environmental noise signal from the frequency spectrum of the frequency domain signal to obtain a second returned sound wave frequency domain signal; performing inverse Fourier transform on the second returned sound wave frequency domain signal to obtain the second returned sound wave signal; and / or using the environmental noise signal as a reference signal of an adaptive filter; inputting the first returned sound wave signal into the filter to obtain the second returned sound wave signal.

[0013] As an alternative or supplement to the above scheme, in the method according to an embodiment of the present application, the environmental noise is collected before the first sound wave signal is emitted.

[0014] In a second aspect, a control device is provided, which comprises a processor and a storage device, the storage device being adapted to store a plurality of program codes, the program codes being adapted to be loaded and run by the processor to execute the safety detection method of any one of the technical solutions of the above safety detection method.

[0015] In a third aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored in the computer-readable storage medium, and the program codes are adapted to be loaded and run by a processor to execute the safety detection method according to any one of the technical solutions of the safety detection method described above.

[0016] The one or more technical solutions of the present application have at least one or more of the following advantages

[0017] Advantages:

[0018] In the implementation of the technical solutions of the present application, the state of the personnel near the device can be monitored in real time by emitting the first sound wave signal and receiving the first returned sound wave signal reflected thereby. The trained audio judgment system is used to analyze the returned sound wave signal to determine whether there is a child around the device. In combination with the preset first judgment standard, the system can automatically determine whether to open the child lock function. The present application can improve the safety of the household washing machine and prevent safety accidents caused by the misoperation of children, thereby providing better safety protection and use experience for the user. Compared with the prior art, the present application can monitor the personnel near the device in real time, make intelligent judgment, and has stronger self-defined ability in combination with the first judgment standard, thereby improving higher safety and being more friendly to the user experience. BRIEF DESCRIPTION OF DRAWINGS

[0019] The disclosure of the present application will become more apparent with reference to the drawings. It should be understood by those skilled in the art that the drawings are only for illustrative purposes, and are not intended to limit the scope of protection of the present application. In addition, similar numbers in the figures are used to represent similar components, wherein:

[0020] Figure 1 is a main step flow diagram of the safety detection method according to an embodiment of the present application;

[0021] Figure 2 is a main step flow diagram of the safety detection method according to an embodiment of the present application;

[0022] Figure 3 is a main step flow diagram of the safety detection method according to an embodiment of the present application;

[0023] Figure 4 is a main step flow diagram of the safety detection method according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] Some embodiments of the present application will be described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present application, and are not intended to limit the scope of protection of the present application.

[0025] In the description of the present application, "module" and "processor" can include hardware, software or a combination of both. A module can include hardware circuitry, various suitable sensors, communication ports, memory, and can also include a software portion such as program code, and can be a combination of software and hardware. The processor can be a central processor, a microprocessor, a graphics processor, a digital signal processor or any other suitable processor. The processor has data and / or signal processing functions. The processor can be implemented in software, hardware or a combination of both. The non-transitory computer readable storage medium includes any suitable medium that can store program code, such as a magnetic disk, a hard disk, an optical disk, a flash memory, a read-only memory, a random access memory, etc. The term "A and / or B" means all possible combinations of A and B, such as only A, only B, or A and B. The term "at least one of A or B" or "at least one of A and B" has a similar meaning to "A and / or B", and can include only A, only B, or A and B. The singular form of the term "one", "this" can also include the plural form.

[0026] Some terms related to the present application are explained here.

[0027] Short-Time Fourier Transform (STFT): STFT is a signal processing technique used to analyze the frequency components of a signal within different time windows. It works by dividing the signal into segments, and then performing a Fourier transform on each segment to convert the time-domain signal into a frequency-domain representation.

[0028] Mel-Frequency Cepstral Coefficients (MFCC): MFCC is an audio feature extraction method used to convert an audio signal into features that are easier to process and analyze. It mainly includes the following steps: calculating the power spectrum of the signal, converting the linear frequency scale to the Mel frequency scale, calculating the discrete cosine transform, and then extracting a certain number of cepstral coefficients.

[0029] Example 1

[0030] Referring to the accompanying Figure 1 , Figure 1 is a schematic diagram of the main steps of the safety detection method according to an embodiment of the present application. As shown in Figure 1 , the safety detection method in the embodiment of the present application mainly includes the following steps S10-S40.

[0031] Step S10: Emitting a first acoustic wave signal.

[0032] In this embodiment, the first acoustic wave signal is an acoustic wave signal emitted by the emission unit.

[0033] In one embodiment, the first sound wave signal is preferably an ultrasonic signal, and the ultrasonic signal is transmitted by a voice module (i.e., a speaker) on the laundry device, i.e., the first sound wave signal is emitted. For example, a high ultrasonic frequency (e.g., 40 kHz) is selected, so that the first sound wave signal can propagate in the air while avoiding damage to human hearing. The signal strength is kept within an appropriate range to ensure that the signal can be reflected by a person within a certain distance and still be detected, but will not have a negative impact on the surrounding people or the device.

[0034] By this method, the laundry device can use the existing voice module to transmit ultrasonic signals, thereby achieving detection of the state of the nearby person. Such a design can simplify the structure of the device, reduce dependence on other sensors, and improve the practicality and reliability of the device.

[0035] Step S20: receiving the first returned sound wave signal.

[0036] In this embodiment, the first returned sound wave signal is a reflected sound wave based on the first sound wave signal.

[0037] In one embodiment, the first returned sound wave signal is received using a voice module (i.e., a microphone) on the laundry device. That is, the voice module of the laundry device can be used not only to emit ultrasonic signals but also to receive reflected sound wave signals. In this way, the dependence on additional sensors can be further reduced, and the cost and complexity of the device can be reduced.

[0038] Step S30: inputting the first returned sound wave signal into a trained audio judgment system to obtain the state of the person near the device.

[0039] In this embodiment, the state of the person near the device refers to the situation of the person existing around a device (e.g., a smart home device, an electronic device, etc.). This state describes which types of people are around the device, for example, whether there are children, adults, or both. The device in this embodiment is a laundry device. It should be noted that the specific distance near the device is related to the signal strength of the first sound wave signal and the spatial configuration of the device. The "children" and "adults" in this embodiment are only a description for easy understanding of this embodiment, and in this embodiment, children refer to people with low height, and adults refer to people with high height.

[0040] In one embodiment, steps S301-S305 are performed as shown in FIG. 3. Figure 2

[0041] Step S301: data preprocessing.

[0042] ​In this embodiment, data preprocessing refers to processing the received first return acoustic signal. The main purpose of data preprocessing is to convert the raw data into a format suitable for model training and analysis, while improving the model's performance and robustness.

[0043] In one implementation, a short-time Fourier transform (STFT) is performed on the first returned acoustic signal to convert the time-domain signal into a frequency-domain representation. Then, Mel-frequency cepstral coefficients (MFCCs) are further extracted as features to convert the audio signal into a two-dimensional image in time-frequency representation. In this implementation, data preprocessing is achieved by segmenting the signal, applying window functions, calculating the STFT, calculating the MFCCs, and normalization, such as... Figure 3 As shown, the details are as follows:

[0044] Step S3011: The first returned acoustic signal is segmented to obtain a segmented signal.

[0045] In this embodiment, the audio signal is segmented into several segments, each with a relatively short duration. Segmenting the signal helps to focus on local features in subsequent processing while reducing computational complexity. Furthermore, the partial overlap between adjacent segments helps avoid boundary effects during subsequent recognition.

[0046] In this embodiment, the signal segmentation is explained using an example where the duration of the first returning acoustic signal is 0.5 seconds.

[0047] To better understand this technology, let's illustrate the sampling rate with an example. Assuming a sampling rate of 44100Hz, this means 44100 samples need to be taken per second. That is, a 0.5-second audio signal would contain 22050 samples.

[0048] The first step in obtaining the segmented signal is to determine the length of each segment (window size) and the overlap length. Specifically, each segment is set to contain T = 512 sampling points, with an overlap of 256 sampling points between adjacent segments. This means that the duration of each signal segment is (512 / 44100) seconds, and the interval between adjacent segments is (256 / 44100) seconds.

[0049] The second step in obtaining the segmented signal is to calculate the number of segments. Specifically, for a 0.5-second audio signal, the number of segments to be divided into is calculated. Since there is overlap between adjacent segments, the actual number of usable sampling points is the number of sampling points in each segment minus the number of overlapping sampling points, i.e., 512 - 256 = 256.

[0050] Therefore, the total number of segments N can be calculated using the following formula:

[0051] N = 1 + (Total number of sampling points - Window size) / (Window size - Overlap length)

[0052] N = 1 + (22050 - 512) / (512 - 256)

[0053] N ~ 85

[0054] In the example of the present scenario, the 0.5-second audio signal is segmented into about 85 segments.

[0055] The third step to obtain the segmented signal is the actual segmentation. Specifically, for each segment, the corresponding 512 sample points are extracted from the original audio signal. For example, the first segment contains sample points 0 to 511, the second segment contains sample points 256 to 767 (overlapping 256 sample points), and so on.

[0056] After the above steps, the 0.5-second audio signal is segmented into about 85 segments, each containing 512 sample points. The processed signal can be used for subsequent data preprocessing operations such as window function, STFT, and MFCC calculation.

[0057] Step S3012: Perform window function processing on the segmented signal to obtain a window function signal.

[0058] In the present embodiment, a window function is used to perform windowing processing on each segment of the signal to reduce spectral leakage caused by signal truncation. The window function smooths the edges of the signal, which helps to extract more accurate information in the frequency domain representation.

[0059] In one embodiment, again taking the 0.5-second first return sound wave signal as an example, the application of the window function includes selecting a window function, calculating a window function, and applying a window function, as follows:

[0060] Taking the 0.5-second first return sound wave signal as an example, the audio signal has been segmented into about 85 segments, each containing 512 sample points. Next, a window function will be applied for windowing processing.

[0061] Selecting a window function: There are various types of window functions, such as Hamming window, Hanning window, Blackman window, etc. These window functions can smooth the edges of the signal and reduce spectral leakage caused by truncation. In the present embodiment, the Hamming window is taken as an example:

[0062] Calculating the window function: The calculation formula of the Hamming window is:

[0063] w(n) = 0.54 - 0.46 x cos(2 x π x n / (N - 1))

[0064] where n is the index of the sample point, and N is the window size. In this embodiment, n is from 0 to 511, and N is 512. A 512-length Hamming window coefficient array is calculated using the formula of Hamming window.

[0065] Apply window function: Apply a window function to each segmented signal segment. The specific operation is to multiply each sample point of the signal segment with the corresponding window function coefficient point by point. For example, for the first signal segment, multiply sample point 0 with window function coefficient 0, multiply sample point 1 with window function coefficient 1, and so on until sample point 511 is multiplied with window function coefficient 511. Repeat this operation to apply the window function to all 85 signal segments.

[0066] After the window function processing, the corresponding window function signal is obtained, in which the edges of each window function signal segment become smoother, which can reduce the spectral leakage, so as to extract more accurate information in the frequency domain representation. Next, these windowed signal segments can be used for subsequent data preprocessing steps such as STFT and MFCC calculation.

[0067] Step S3013: Perform short-time Fourier transform on the window function signal to obtain a frequency domain signal.

[0068] In this embodiment: The time domain signal is converted into a frequency domain representation by short-time Fourier transform. This enables understanding of the characteristics of the signal at different times and frequencies, thereby helping to capture dynamic information in the audio signal.

[0069] In one embodiment, the first return sound wave signal is taken as an example of 0.5 seconds, and Fourier transform is performed for each signal segment. For each windowed signal segment, the fast Fourier transform (FFT) algorithm is used to calculate its Fourier transform. In this embodiment, 512-point FFT is used. The output of FFT is a complex number array, which represents the amplitude and phase information of the signal in the frequency domain.

[0070] Then the frequency and amplitude information is extracted. Since the signal is a real number, its Fourier transform has conjugate symmetry, and only the first half of the frequency components need to be retained. In this embodiment, the first 256 frequency components are retained. Next, the amplitude of each frequency component is calculated, i.e. the modulus of the complex number:

[0071] Amplitude = sqrt(real part^2 + imaginary part^2)

[0072] After extraction, the frequency domain complex representation of each signal segment is converted to amplitude representation.

[0073] After that, the time-frequency representation is constructed. The frequency amplitude information of all signal segments is combined to form a two-dimensional array. In this embodiment, the size of the two-dimensional array is 85x256, where 85 represents the number of signal segments (time dimension) and 256 represents the number of frequency components (frequency dimension). This two-dimensional array is the time-frequency representation of the audio signal, where each element represents the signal strength corresponding to the time and frequency.

[0074] After STFT processing, the time-domain audio signal is converted into a two-dimensional array of time-frequency representation. This array can be used for subsequent data preprocessing operations, such as normalization or calculation of Mel-frequency cepstral coefficients (MFCC).

[0075] Step S3014: Calculate MFCC.

[0076] In this embodiment, Mel-frequency cepstral coefficients are extracted, which can convert the audio signal into features that are easier to process and analyze.

[0077] In one embodiment, the power spectrum is first calculated. Specifically, the amplitude of each element in the time-frequency representation calculated by STFT is squared to obtain the signal power corresponding to the time and frequency. In the embodiment, the size of the power spectrum is 85x256.

[0078] Then apply the filter bank. For example, Gaussian filter bank or band-pass filter bank. These filter banks can help extract the energy features of ultrasonic signals in different frequency ranges. First, the number and center frequency of the filter bank need to be calculated. In this embodiment, the number of filters is set to 26. Next, the center frequencies of the filters are selected at equal intervals in the frequency range. Then construct the Gaussian filter or band-pass filter. Finally, perform inner product of the power spectrum with each filter to obtain the filter output.

[0079] Then apply the discrete cosine transform (DCT): perform discrete cosine transform on the filter output. When processing general sound waves, the first few DCT coefficients are retained. When processing ultrasonic signals, more DCT coefficients (such as the first 26) are retained to capture higher frequency information in the signal. These coefficients will constitute the ultrasonic frequency cepstral coefficients (UFCC).

[0080] In this embodiment, an 85x26 UFCC matrix will be obtained. This matrix will be used as the input of the neural network for identifying the state of the person near the device. By retaining more DCT coefficients, the high-frequency information of the ultrasonic signal can be better captured, thereby improving the performance of the audio judgment system.

[0081] Step S3015: Perform normalization.

[0082] In this embodiment, the purpose of normalization is to eliminate the dimensional influence between features, so that different features have similar scales. This helps the convergence and generalization ability of the neural network.

[0083] In this embodiment, it should be noted that after STFT processing, normalization operation can be directly performed. The reason for calculating the MFCC of the frequency domain signal is to exhibit more features and information.

[0084] The MFCC matrix (size 85x13) will be normalized. There are two common normalization methods, Min-Max Scaling and Standardization (Z-score normalization), which are described below. Specifically as follows:

[0085] Min-Max Scaling: Normalize each column (corresponding to an MFCC coefficient) of the MFCC matrix. The specific operation is as follows:

[0086] A1: Calculate the maximum and minimum values of each column.

[0087] A2: Perform the following operation on each element of each column: (element value - minimum value) / (maximum value - minimum value). In this way, the value of each element will be mapped to the interval [0, 1].

[0088] Standardization: Normalize each column (corresponding to an MFCC coefficient) of the MFCC matrix. The specific operation is as follows:

[0089] B1: Calculate the mean and standard deviation of each column.

[0090] B2: Perform the following operation on each element of each column: (element value - mean) / standard deviation. In this way, the value of each element will be centered at 0 with the same standard deviation.

[0091] After normalization, each feature (i.e. each column) of the MFCC matrix will have similar scales. This normalized MFCC matrix will be used as input features for neural network training and prediction. In this embodiment, the normalized MFCC matrix is the first return sound wave vector. The window function and normalization processing methods can reduce noise and interference and improve the robustness of the model.

[0092] In this embodiment, through data preprocessing, the accuracy, generalization ability and robustness of the model can be improved, so as to achieve better results in practical applications.

[0093] Step S302: input the first return acoustic wave signal into the trained feature extraction neural network to obtain a first return acoustic wave feature vector.

[0094] In this embodiment, the first return acoustic wave signal is a signal after data preprocessing. The trained feature extraction neural network can convert the first return acoustic wave signal into a feature vector.

[0095] In one embodiment, the preprocessed first return acoustic wave signal (e.g., UFCC matrix) is used as input data. The UFCC matrix here is a two-dimensional matrix that contains the time and frequency information of the first return acoustic wave signal. The first return acoustic wave signal is first passed through a convolutional layer. The convolutional kernels in the convolutional layer slide over the input data to extract features in local regions. For example, 3x3 or 5x5 convolutional kernels are used. Each convolutional kernel can learn to detect a specific type of feature, such as loudness or time. The result of the convolution operation is a new two-dimensional matrix called a feature map.

[0096] The feature map output by the convolutional layer is passed to a ReLU activation function. ReLU (Rectified Linear Unit) is a nonlinear function with the formula f(x) = max(0, x). The role of ReLU is to set all negative values to zero and keep positive values. The activation function enables the neural network to learn complex patterns and features.

[0097] The activated output is batch normalized. Batch normalization is a technique that normalizes the input data of each neuron to have similar numerical ranges. This helps to speed up the convergence of the neural network and improve the performance of the model. In specific operations, batch normalization calculates the mean and variance of the input data and normalizes the data using these statistics. Then, batch normalization also introduces learnable scaling and shifting parameters to make the neural network have more expressive power.

[0098] The data is reduced in dimension by the pooling layer. The main purpose of the pooling layer is to reduce the spatial dimension of the data, thereby reducing the amount of computation and the number of parameters. The most commonly used pooling operation is max pooling, which selects the maximum value from the local region of the input data as the output. The pooling layer can also improve the robustness of the model to small transformations. For example, max pooling can be performed on a 2x2 window, and the maximum value in each window is taken as the output.

[0099] By constructing multiple convolution, activation, batch normalization, and pooling layers, the neural network can automatically learn complex features in the audio signal. Finally, the CNN will output a high-dimensional feature vector containing useful information in the audio signal. This feature vector can be used for further analysis or passed to subsequent neural network layers. In the example of the CNN, the next step is to pass this feature vector to a recurrent neural network (RNN).

[0100] Step S303: input the first return sound wave feature vector into the trained recurrent neural network to obtain a first return sound wave time series feature vector.

[0101] In this embodiment, the trained recurrent neural network can further capture time-related information to generate the first return sound wave time series feature vector.

[0102] In one embodiment, this feature vector can be used for further analysis or passed to subsequent neural network layers. In the example of the CNN, the next step is to pass this feature vector to a recurrent neural network (RNN).

[0103] Fully connected layer and flattening: Before passing the data to the RNN, one or more fully connected layers are usually added. The fully connected layer further integrates the features extracted by the convolutional and pooling layers. In addition, before inputting the data into the RNN, the multi-dimensional feature vector needs to be flattened into a one-dimensional vector. The flattening operation converts the two-dimensional or three-dimensional data structure into a one-dimensional array for input to the RNN.

[0104] Recurrent neural network (RNN): RNN is a special type of neural network suitable for processing sequential data. Unlike traditional feedforward neural networks, RNN has a memory function that can capture long-term dependencies in time series data. In this embodiment, the RNN is used to capture time-related information in the ultrasonic signal. After the RNN, the first return sound wave time series feature vector is output.

[0105] Step S304: based on the first return sound wave time series feature vector, obtain the probability distribution of the state of the person near the device under different circumstances.

[0106] In this embodiment, the first return sound wave time series feature vector is input into the trained neural network to obtain the probability distribution of the state of the person near the device under different circumstances.

[0107] In one embodiment, after the RNN processing, the first return sound wave time series feature vector containing time series information is obtained. A fully connected layer is added after the RNN to map the first return sound wave time series feature vector to the pre-set circumstances. The goal of the fully connected layer is to associate the extracted features with the classification task.

[0108] Fully Connected Layer (FC): The fully connected layer contains a set of neurons that are connected to all neurons in the previous layer. In this example, the fully connected layer takes the first return acoustic wave time series feature vector outputted by the RNN as input, which contains time series information. The role of the fully connected layer is to integrate and map these features onto the classification task. Each neuron in the fully connected layer calculates the weighted sum of the input feature vector, plus a bias term, and then passes it through an activation function (e.g., ReLU) to introduce nonlinearity.

[0109] Output Layer: The output layer is the last layer of the neural network, which generates the final classification result. In this implementation, the output layer can be set to three neurons, representing the three classes of "no children", "only children", and "children and adults". The number of neurons in the output layer is equal to the number of classes in the classification task. Similar to the fully connected layer, the output layer also calculates the weighted sum of the input feature vector and adds a bias term.

[0110] Softmax Activation Function: To convert the weighted sum of the output layer into a probability distribution of each class, a Softmax activation function can be used. The Softmax function maps each weighted sum to a value between 0 and 1, ensuring that the sum of all class probabilities is 1. This allows for an intuitive understanding of the output result, for example, a probability of 0.8 represents an 80% confidence level.

[0111] In summary, after the fully connected layer and the output layer, the association degree of the first return acoustic wave time series feature vector with the "no children", "only children", and "children and adults" classes can be obtained. The Softmax activation function converts these association degrees into a probability distribution, facilitating the classification and decision-making of the acoustic wave signal.

[0112] Step S305: Select the case with the highest probability distribution of personnel state near the device as the personnel state near the device.

[0113] In one embodiment, after the Softmax activation function, a vector containing three probability values will be obtained, representing the predicted probability of each class. For example, the probability vector may be [0.1, 0.8, 0.1], indicating that the probability of "no children" is 10%, the probability of "only children" is 80%, and the probability of "children and adults" is 10%.

[0114] The class with the maximum probability value in this probability vector is found. In the above example, the maximum probability value is 0.8, which corresponds to the "only children" class. Therefore, it can be judged that the personnel state near the device is "only children". The class with the highest probability is taken as the personnel state near the device, that is, the personnel state near the device. In this example, "only children" is obtained as the output result.

[0115] In summary, first, the local features are extracted from the input first return acoustic wave signal using a convolutional neural network (CNN), and then the time-related information is captured through a recurrent neural network (RNN). Finally, the output layer generates the prediction result. In this process, the trained audio judgment system can automatically learn the complex patterns and features in the ultrasonic signal, thereby achieving accurate signal analysis and prediction.

[0116] Through the above steps, the personnel state near the device can be determined from the output of the neural network. Through the above scheme, it is possible to predict whether there are no children, only children, or children and adults near the device according to the input first return acoustic wave signal, so as to take corresponding safety measures.

[0117] Step S40: Determine whether to open the child lock according to the personnel state and the preset first judgment standard.

[0118] In this embodiment, the preset first judgment standard is a system preset judgment standard and / or a user customized judgment standard.

[0119] In one embodiment, the first judgment standard can be a system preset judgment standard or a user customized judgment standard. The system preset judgment standard is a set of default rules set by the device manufacturer or service provider. For example, the system preset judgment standard can be: if the personnel state is "only children" or "children and adults", automatically start the child lock. The user customized judgment standard is that the user of the device can customize the judgment standard according to his own needs and preferences. For example, the user may wish not to open the child lock when the personnel state is "children and adults", because the user believes that adults can ensure the safety of children according to their own situation. In this case, the user can modify the judgment standard so that the child lock is only opened when the personnel state is "only children".

[0120] The device can automatically decide whether to open the child lock according to these preset first judgment standards. For example, when the device detects that there are children in the surrounding environment, it can automatically enable the child lock function to ensure the safety of children. At the same time, the user can adjust these judgment standards according to his own needs and environmental conditions, so as to realize more flexible and personalized safety measures.

[0121] In summary, according to the state of the person and the preset first judgment standard, the device can automatically determine whether to open the child lock to ensure the safety of children and adults near the device. Users can also customize these judgment standards according to individual needs and preferences, providing more flexible safety control options.

[0122] Example 2

[0123] In this embodiment, the technology in embodiment 2 is substantially the same as in embodiment 1, except that in this embodiment, the environmental noise is collected in advance, and the influence of the environmental noise is excluded, as shown in Figure 4 , specifically as follows:

[0124] Step S501: Collect environmental noise.

[0125] In one embodiment, the environmental noise is collected through the voice module of the laundry device. Preferably, in one embodiment, before the first sound wave signal is emitted, the voice module of the laundry device is started, and its microphone is used to record the environmental noise for a period of time. This period of time can be adjusted according to actual needs, for example, it can be set to 1-5 seconds. Then the collected environmental noise data is stored in the memory of the device for subsequent processing and analysis.

[0126] Step S502: Based on the environmental noise and the first return sound wave signal, the second return sound wave signal is obtained.

[0127] In this embodiment, the second return sound wave signal is a sound wave signal after the environmental noise is eliminated.

[0128] In one embodiment, the received return sound wave signal is compared and analyzed with the previously collected environmental noise data, so as to eliminate or reduce the influence of noise on signal detection. In this embodiment, two specific methods are given for signal elimination, namely using an adaptive filter method and a spectral subtraction method for signal elimination.

[0129] Among them, the spectral subtraction method is specifically:

[0130] The collected environmental noise signal and the first return sound wave signal are respectively subjected to Fourier transform to obtain the corresponding first environmental noise signal and the frequency domain signal. The frequency domain signal is the frequency domain signal of the first return sound wave signal after frequency domain, which is similar to step S3013, and will not be repeated here. In this embodiment, the first environmental noise signal is converted to the frequency domain using fast Fourier transform (FFT).

[0131] The spectrum of the received frequency domain signal is then subtracted by the spectrum of the first ambient noise signal to obtain a second return sound frequency domain signal. The second return sound signal eliminates the influence of noise. The second return sound frequency domain signal is then inverse Fourier transformed (IFFT) to obtain a second return sound signal. The second return sound signal converts the signal from the frequency domain back to the time domain.

[0132] Another method is to adaptively filter the method specifically is:

[0133] The ambient noise signal is used as the reference signal of the adaptive filter, and the first return sound signal is input into the filter to obtain a second return sound signal.

[0134] It should be noted that the adaptive filter is a filter that can automatically adjust the filter coefficients according to the input signal and the reference signal. When the filter coefficients are optimal, the output signal of the filter will be as close as possible to the original signal without the influence of ambient noise. Although this output signal is not exactly the original signal, it can be a relatively good approximation.

[0135] Step S503: Based on the second return sound signal and the trained audio judgment system, the state of the person near the device is obtained.

[0136] In this embodiment, this step is similar to step S30, which will not be described here.

[0137] It should be noted that although the above embodiments describe each step in a specific order, those skilled in the art can understand that in order to achieve the effect of the present application, different steps do not necessarily have to be executed in this order, they can be executed simultaneously (in parallel) or in other order, these changes are within the protection scope of the present application.

[0138] Those skilled in the art can understand that all or part of the processes in the method of the above-mentioned embodiment of the present application can also be completed by a computer program instructing the relevant hardware, and the computer program can be stored in a computer readable storage medium. When the computer program is executed by a processor, the steps of the above-mentioned various method embodiments can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms, etc. The computer readable storage medium can include any entity or device, medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunication signal and software distribution medium, etc. that can carry the computer program code. It should be noted that the contents included in the computer readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable storage medium does not include electrical carrier signals and telecommunication signals.

[0139] Further, the present application also provides a control device. In an embodiment of the control device according to the present application, the control device comprises a processor and a storage device, the storage device can be configured to store a program for executing the safety detection method of the above-mentioned method embodiments, and the processor can be configured to execute the program in the storage device, which includes but is not limited to the program for executing the safety detection method of the above-mentioned method embodiments. For the convenience of description, only the parts related to the embodiments of the present application are shown, and the specific technical details not disclosed are referred to the method part of the embodiments of the present application. The control device can be a control device equipment formed by various electronic devices.

[0140] Further, the present application also provides a computer readable storage medium. In an embodiment of the computer readable storage medium according to the present application, the computer readable storage medium can be configured to store a program for executing the safety detection method of the above-mentioned method embodiments, which can be loaded and run by a processor to implement the above-mentioned safety detection method. For the convenience of description, only the parts related to the embodiments of the present application are shown, and the specific technical details not disclosed are referred to the method part of the embodiments of the present application. The computer readable storage medium can be a storage device equipment formed by various electronic devices, and optionally, the computer readable storage medium in the embodiments of the present application is a non-transitory computer readable storage medium.

[0141] Further, it should be understood that, since the setting of each module is only for illustrating the functional units of the device of the present application, the physical device corresponding to the module can be the processor itself, or a part of software, a part of hardware or a part of combination of software and hardware in the processor. Therefore, the number of each module in the figure is only illustrative.

[0142] Those skilled in the art can understand that the various modules in the device can be adaptively split or combined. Such splitting or combining of specific modules does not cause the technical solution to deviate from the principles of the present application, and therefore, the technical solutions after splitting or combining will fall within the protection scope of the present application.

[0143] So far, the technical solutions of the present application have been described in combination with the preferred embodiments shown in the drawings, but those skilled in the art can easily understand that the protection scope of the present application is obviously not limited to these specific embodiments. Those skilled in the art can make equivalent changes or replacements to the related technical features without deviating from the principles of the present application, and the technical solutions after such changes or replacements will fall within the protection scope of the present application.

Claims

1. A security detection method characterized by, The method comprises: emitting a first sound wave signal; receiving a first return sound wave signal, wherein the first return sound wave signal is a reflected sound wave based on the first sound wave signal; obtaining a personnel state near the device based on the first return sound wave signal and a trained audio judgment system, wherein the personnel state describes whether there are children, adults, or both around the device; judging whether to open a child lock according to the personnel state and a preset first judgment standard; the audio judgment system comprises a feature extraction neural network and a recurrent neural network; and obtaining the personnel state near the device based on the first return sound wave signal and the trained audio judgment system comprises: inputting the first return sound wave signal into the trained feature extraction neural network to obtain a first return sound wave feature vector; inputting the first return sound wave feature vector into the trained recurrent neural network to obtain a first return sound wave time sequence feature vector; obtaining a probability distribution of the personnel state near the device under different conditions based on the first return sound wave time sequence feature vector; and selecting a condition with the highest personnel state probability distribution near the device as the personnel state near the device.

2. The safety detection method of claim 1, wherein: the preset first judgment standard is a system preset judgment standard and / or a user defined judgment standard.

3. The safety detection method according to claim 1 or 2, characterized in that, obtaining the personnel state near the device based on the first return sound wave signal and the trained audio judgment system comprises: preprocessing the first return sound wave signal to obtain a first return sound wave vector; and inputting the first return sound wave vector into the trained audio judgment system to obtain the personnel state near the device.

4. The security detection method of claim 3, wherein, The preprocessing of the first return sound wave signal to obtain the first return sound wave vector comprises: segmenting the first return sound wave signal to obtain a segmented signal; performing window function processing on the segmented signal to obtain a window function signal; performing Fourier transform on the window function signal to obtain a frequency domain signal; and normalizing the frequency domain signal to obtain the first return sound wave vector.

5. The security detection method of claim 1 or 2, wherein, The method further comprises: collecting environmental noise; obtaining the personnel state near the device based on the first return sound wave signal and the trained audio judgment system comprises: obtaining a second return sound wave signal based on the environmental noise and the first return sound wave signal; and obtaining the personnel state near the device based on the second return sound wave signal and the trained audio judgment system.

6. The security detection method of claim 5, wherein, The obtaining of the second return sound wave signal based on the environmental noise and the first return sound wave signal comprises: performing Fourier transform on the collected environmental noise signal and the first return sound wave signal respectively to obtain a corresponding first environmental noise signal and a frequency domain signal; subtracting the frequency spectrum of the first environmental noise signal from the frequency spectrum of the frequency domain signal to obtain a second return sound wave frequency domain signal; performing inverse Fourier transform on the second return sound wave frequency domain signal to obtain the second return sound wave signal; and / or using the environmental noise signal as a reference signal of an adaptive filter; and inputting the first return sound wave signal into the filter to obtain the second return sound wave signal.

7. The security detection method of claim 6, wherein, The ambient noise is acquired before the first sound wave signal is emitted.

8. A control device comprising a processor and a storage device, the storage device being adapted to store a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by the processor to execute the safety detection method of any one of claims 1 to 7.

9. A computer readable storage medium having stored therein a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by the processor to execute the safety detection method of any one of claims 1 to 7.

Images