Monitoring method for performing equipment alert and mistaken entry prevention by using voice recognition technology

Abstract

The invention discloses a monitoring method for performing equipment alert anti-mistaken-entry by using a sound recognition technology, which is applied to the field of anti-mistaken-entry monitoring, and aims to solve the problem that a traditional anti-mistaken-entry recognition method only depends on a single sensor or mark for recognition and cannot cope with the variability of the environment. The problems of low monitoring precision, long response time, insufficient mistaken entry protection capability and the like exist; according to the invention, autonomous learning and adaptive adjustment are realized through real-time analysis and mode recognition of environmental data. Through continuous data acquisition and processing, the system can continuously optimize self monitoring capability and accuracy. The method with the functions of self-learning, high adaptability, accurate monitoring, safety warning and mistaken entry prevention can greatly improve the safety of equipment and promote the safe operation of a working place.

Description

Technical Field

[0001] The invention belongs to the field of sound recognition monitoring, and in particular relates to a misentry monitoring technology. Background Art

[0002] In dimly lit industrial environments, such as mines and tunnels, it is very important to ensure the warning and prevention of accidental entry of dangerous equipment. However, traditional anti-accidental entry identification methods rely only on a single sensor or marker for identification, which cannot cope with the variability of the environment and have problems such as low monitoring accuracy, long response time, and insufficient protection against accidental entry. At the same time, when people or objects enter the equipment area by mistake, there is a lack of timely warnings and effective prevention measures.

[0003] Therefore, a self-learning and adaptable method is needed that can accurately monitor according to environmental changes and provide safety warnings and prevent misentry. Summary of the invention

[0004] In order to solve the above technical problems, the present invention proposes a monitoring method for equipment warning and preventing accidental entry by using sound recognition technology. In industrial sites with dim light and changing environment, the sound positioning technology is used to monitor equipment warning and preventing accidental entry, so as to avoid workers from approaching dangerous equipment and reduce the occurrence rate of safety accidents.

[0005] The technical solution adopted by the present invention is: a monitoring method for equipment warning and preventing accidental entry using sound recognition technology, comprising:

[0006] S1. synchronously collect sound data and environmental data; the environmental data at least includes spatial location information;

[0007] S2. Preprocess the collected environmental data to obtain a security level;

[0008] S3, preprocessing the sound data to obtain characteristic values ​​of the sound data;

[0009] S4, using the characteristic value of the sound, the security level and the corresponding spatial position information to mark the characteristic value of the sound data;

[0010] S5, constructing a sound recognition model; after the known sound data and environmental data are processed in steps S1-S4, they are input into the constructed sound recognition model for training;

[0011] S6. After being processed in steps S1-S4, the sound data and environmental data to be detected are input into the trained sound recognition model to obtain the recognition result.

[0012] Beneficial effects of the present invention: The present application collects sound data and environmental data at the same time; calculates the security level of the current detection area based on the environmental data, and calculates the threshold of the sound data based on the security level, so as to make a preliminary judgment on abnormal events; pre-processes the sound data, divides it into a normal spectrum feature data set and an abnormal feature data set, and annotates the sound data in the data set in combination with the corresponding security level and spatial location information; trains a sound recognition model based on the annotated sound data; thereby obtaining a more accurate misentry recognition result.

[0013] The present invention calculates the security level by combining environmental data (such as sensor data), and dynamically sets the threshold of sound data accordingly, thereby effectively reducing false alarms caused by environmental noise interference; at the same time, by utilizing spatial location information, the present invention can more accurately distinguish normal sound events from abnormal sound events, and can capture important abnormal events in complex environments, reducing the possibility of missed reports; in addition, the use of environmental data to assist in sound recognition, combined with model training based on labeled sound data, significantly improves the ability to distinguish normal sounds from abnormal sounds in noisy environments, thereby improving the overall recognition accuracy; finally, by setting a sound data threshold based on the security level, the present invention can quickly filter out irrelevant audio data, simplify the processing flow, and effectively shorten the processing delay time. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 A block diagram of the monitoring system on which the method of the present invention is based;

[0015] Figure 2 A sound preprocessing flow chart provided for an embodiment of the present invention;

[0016] Figure 3 This is a structural diagram of the sound recognition model used in the present invention;

[0017] Figure 4 Flow chart of the method of the present invention. DETAILED DESCRIPTION

[0018] To facilitate those skilled in the art to understand the technical content of the present invention, the present invention is further explained below with reference to the accompanying drawings.

[0019] First, the method of the present invention is introduced. Figure 1 As shown, the monitoring system based on the present invention includes: an acquisition module, a sound preprocessing module, an identification and analysis module, a control module, a warning control module, a data storage module and a display module; below, in order to facilitate ordinary technicians in this field to understand and implement the present invention, the specific situation of each module of the above method is further described in detail:

[0020] Acquisition module: It is mainly used to collect data from multiple different types of sensors at a preset fixed frequency and accurately, and use the microphone array to synchronously capture the audio information of the device's surrounding environment in order to comprehensively monitor the device status and surrounding environment. The collected data and sound information are then effectively integrated and transmitted to other related modules in the system for further processing and decision-making. The specific implementation process is as follows:

[0021] Step 1: Sensor Layout and Installation

[0022] According to the physical structure and size characteristics of the equipment, carefully plan the sensor deployment plan to ensure that it covers the key monitoring areas and does not affect the normal operation of the equipment. Select suitable locations on the edge of the equipment for continuous and uniform sensor installation to achieve all-round and no-dead-angle data collection. The main types of sensors used include but are not limited to:

[0023] Laser Radar (LiDAR): Provides accurate three-dimensional distance information, suitable for obstacle detection, contouring and environment modeling, and helps devices achieve obstacle avoidance navigation and spatial positioning;

[0024] Photoelectric sensor: By detecting changes in optical signals such as light intensity, color, and infrared characteristics, it is used to identify the presence of objects, motion detection, flame alarms, and other scenarios, enhancing the sensitivity of the device to environmental changes;

[0025] Millimeter wave sensor: uses millimeter wave radar technology to detect the distance, speed and angle information of distant targets, which is particularly suitable for target tracking and identification in severe weather conditions;

[0026] Vibration sensor: monitors the vibration of the equipment itself or the surrounding structure, which can be used for fault diagnosis, structural health monitoring and earthquake early warning, etc., to ensure the safety and stability of equipment operation;

[0027] Ultrasonic sensor: uses the characteristics of ultrasonic waves propagating in the air to detect and measure the distance of close objects. It is often used in obstacle avoidance, liquid level monitoring, flow control and other fields.

[0028] In particular, in response to the need for all-round monitoring in the horizontal direction, one or more sensor arrays evenly distributed along the horizontal plane are designed and installed to ensure that the equipment achieves seamless and continuous detection coverage within the horizontal 360° range. The spacing and angles between sensors need to be carefully calculated to eliminate blind spots and ensure the integrity and consistency of monitoring data.

[0029] Step 2: Install the audio capture device microphone

[0030] Based on the physical structure and size characteristics of the device, installing multiple microphones around the device to form an array can help improve the accuracy of sound positioning and better handle complex sound environments.

[0031] Environmental Assessment: Analyze the sound environment around the device to identify the main audio sources that need to be captured and possible sources of interference.

[0032] Location selection: Choose a suitable location around the device to install the microphone according to the working environment and characteristics of the device. If the device works in an open environment, it is recommended to evenly distribute the microphones around the device to ensure that all-round sound information can be captured.

[0033] Quantity configuration: The number of microphones is determined based on monitoring requirements and the complexity of the environment. If sound localization is required, at least two microphones are required; for more complex environments, the number of microphones can be increased and formed into an array.

[0034] Spacing and Angle: Ensure that there is sufficient spacing between the microphones to reduce mutual interference, and determine the optimal angle and spacing based on the specific layout of the equipment to achieve the best sound capture effect.

[0035] Step 3: Data fusion and security level assessment

[0036] The acquisition module receives and integrates the raw data streams from various sensors in real time, and uses advanced data fusion algorithms to conduct comprehensive analysis of these multi-source, heterogeneous data, extract key features, eliminate redundancy and noise, and improve the effectiveness and reliability of the data. The specific steps include:

[0037] Data synchronization: ensure that data from all sensors are collected at the same timestamp to avoid information inconsistency caused by data sampling time differences;

[0038] Data calibration: calibrate and correct the data of different sensors to eliminate individual differences and environmental influences between sensors and ensure data comparability;

[0039] In order to ensure that the data of multiple distance measuring sensors of the same type (such as ultrasonic sensors) are consistent and comparable in the same environment, we need to calibrate and correct them. The following is an explanation of the calibration and correction of ultrasonic sensor data:

[0040] Assume that there are three ultrasonic sensors A, B and C, which are used to measure the distance of the same target. This embodiment calibrates and corrects their data through the following steps.

[0041] Step a: Calibration

[0042] Goal: Determine the intrinsic error of each sensor, which is the deviation between the sensor reading and the actual distance under ideal conditions.

[0043] Method: Use a standard distance as a reference point for measurement. For example, place a target at a known distance, such as 1 meter, and record the readings from each sensor.

[0044] Example:

[0045] Sensor A measures a target at 1 meter and reads 0.98 meters.

[0046] Sensor B measures the target at 1 meter and reads 1.02 meters.

[0047] Sensor C measures a target at 1 meter and reads 1.01 meters.

[0048] Calculation formula:

[0049] Correction value = actual distance - sensor reading

[0050] Calculation process:

[0051] For sensor A: correction value_A = 1-0.98 = 0.02 m

[0052] For sensor B: correction value_B = 1-1.02 = -0.02 meters

[0053] For sensor C: correction value_C = 1-1.01 = -0.01 m

[0054] Step b: Environmental impact compensation

[0055] Objective: Considering the impact of factors such as temperature and humidity on sensor readings, environmental impact compensation is required.

[0056] Method: You can consult the sensor's technical manual to understand the impact of environmental factors on its accuracy and compensate accordingly.

[0057] Example:

[0058] Assuming that all sensors are affected by temperature, the reading error increases by 0.001 meters for every 1°C increase.

[0059] Assume that the current temperature is 25℃ and the standard temperature is 20℃, which means the temperature has risen by 5℃.

[0060] Calculation formula:

[0061] Environmental impact compensation value = (current temperature - standard temperature) x error change coefficient

[0062] Calculation process:

[0063] The temperature rises by 5°C, so the error changes to: 5 x 0.001 = 0.005 meters

[0064] Total correction value:

[0065] The total calibration value of sensor A = calibration value_A + environmental impact compensation value_A. In this embodiment, the total calibration value of sensor A is: 0.02+0.005=0.025 meters

[0066] The total correction value of sensor B = correction value_B + environmental impact compensation value_B. In this embodiment, the total correction value of sensor B is: -0.02+0.005=-0.015 meters

[0067] The total correction value of sensor C = correction value_C + environmental impact compensation value_C. In this embodiment, the total correction value of sensor C is: -0.01+0.005=-0.005 meters

[0068] Step c: Error correction

[0069] Goal: Apply the calculated total correction value to the raw reading to obtain a more accurate result.

[0070] Calculation formula:

[0071] Corrected reading = original reading + total correction value

[0072] Calculation process:

[0073] Corrected reading of sensor A: 0.98+0.025=1.005 meters

[0074] Sensor B corrected reading: 1.02-0.015=1.005 meters

[0075] Corrected reading of sensor C: 1.01-0.005=1.005 meters

[0076] By following the above steps, the readings from the three ultrasonic sensors can be made more consistent and reliable, so that the processed data can be better used in subsequent analysis and decision-making.

[0077] Feature extraction: Perform mathematical transformation and feature extraction on the original data to generate feature vectors that are easy to analyze, such as distance, speed, acceleration, etc. The specific process is as follows:

[0078] 1. Expression of original data

[0079] Assuming that the ranging sensor collects data every time interval △t, the data collected for the i-th time can be defined as:

[0080] d i (t) = f(t i )

[0081] Among them, d i(t) represents the distance measured for the i-th time; t i is the time point of the ith measurement; f(t i ) is the output function of the ranging sensor, which represents the i Measured distance.

[0082] 2. Representation of feature vectors

[0083] Distance feature

[0084] The simplest feature is the distance itself, which can be obtained directly from the raw data:

[0085] d(t i ) = d i

[0086] Speed ​​characteristics

[0087] The speed can be estimated by the distance change and time interval between two adjacent measurements. The instantaneous speed v(t) can be approximated as:

[0088]

[0089] Acceleration characteristics

[0090] Acceleration can be calculated from the rate of change of velocity. The instantaneous acceleration a(t) can be approximated as:

[0091]

[0092] Combination of eigenvectors

[0093] Combine the above features into a feature vector X(t i ), which can be expressed as:

[0094] X(t i )=[d i ,v(t i ),a(t i )] T

[0095] Among them, d i is the distance feature; v(t i ) is the velocity characteristic; a(t i ) is the acceleration feature; T represents the transpose operation, which arranges the vectors in columns.

[0096] Fusion calculation: Use weighted average, Kalman filtering, Bayesian network, deep learning and other methods to fuse the extracted features and form a unified cognitive model of the current monitoring environment;

[0097] The following is the specific calculation method and the feature representation after fusion.

[0098] 1. Weighted average

[0099] Weighted averaging is a simple method that can be used to combine feature values ​​from different sources by giving each feature a certain weight to reflect its importance.

[0100] The determination of the specific weight threshold of each feature is mainly to use the knowledge of domain experts to set the initial weight, and then adjust the weight through experiments, observe the system performance, and continuously iterate and optimize. Before actual deployment, it should be fully verified in a test environment to ensure that the weight setting is reasonable.

[0101] Suppose there are n features x1, x2, ..., x n , each feature has a corresponding weight w1,w2,...,w n . Then the weighted average can be expressed as:

[0102]

[0103] in, is the fused eigenvalue, and To ensure that the sum of the weights is 1.

[0104] 2. Kalman Filter

[0105] Kalman filter is a recursive Bayesian filter, which is often used for state estimation in dynamic systems. For ranging sensors, Kalman filter can be used to estimate state variables such as distance, velocity and acceleration.

[0106] Assume that the state of the system is represented by X k It means that in the process from time step k to k+1, the state transfer equation is:

[0107] x k+1 =Ax k +Bu k +w k

[0108] Where A is the state transfer matrix; u k is the control input; w k is the process noise, which is generally assumed to be zero-mean Gaussian white noise;

[0109] The observation equation is:

[0110] z k =Hx k +v k

[0111] Among them, z k is the observation vector; H is the observation matrix; v kis the observation noise, which is also assumed to be zero-mean Gaussian white noise.

[0112] The Kalman filter algorithm consists of a prediction step and an update step. The prediction step involves predicting a priori estimates of the next state, while the update step corrects this prediction based on new observations.

[0113] 3. Bayesian Network

[0114] A Bayesian network is a probabilistic graphical model that can be used to represent conditional dependencies between random variables. For ranging sensor data, a Bayesian network can be constructed to estimate the relationship between various features.

[0115] Suppose there are multiple features X1,X2,...,X n , the Bayesian network is constructed by the joint probability distribution P(X1,X2,...,X n ) to model the relationship between them. The network can be represented as a series of conditional probability tables (CPTs) and dependencies between nodes.

[0116] 4. Deep Learning

[0117] Deep learning can automatically learn complex relationships between features through neural networks. For ranging sensor data, a multilayer perceptron (MLP) or convolutional neural network (CNN) can be built to fuse features.

[0118] Assume there are m sets of eigenvectors x 1 ,x 2 ,...,x m , each vector has a length of n, and the deep learning model can be expressed as:

[0119] y=f(x 1 ,x 2 ,...,x m )

[0120] Where f is the forward propagation function of the neural network, and y is the fused feature vector.

[0121] Feature representation after fusion

[0122] For all these methods, the fused features can be represented as a vector y:

[0123] -Weighted average:

[0124] - Kalman filter: y = x k|k (A posteriori estimate of the state vector)

[0125] - Bayesian network: y can be the joint probability distribution obtained from the Bayesian network

[0126] - Deep learning: y is the output vector of the last layer of the neural network

[0127] The y here can be further refined according to the specific application scenario, for example, including estimated values ​​of state variables such as distance, speed, and acceleration.

[0128] Based on the fused data, the acquisition module uses a predefined evaluation algorithm or machine learning model to calculate the safety level of the current monitoring area. The safety level may be determined based on a variety of factors, such as obstacle distance, potential collision risk, environmental abnormality indicators, equipment vibration conditions, etc., and is presented in the form of numerical values, levels or color codes. The calculation process of the monitoring area safety level is as follows:

[0129] Assume that the data fusion process has been completed and a fused feature vector y is obtained, which contains features such as distance, speed, acceleration, vibration amplitude, etc. Now, the safety level of the current monitoring area is calculated based on these features.

[0130] 1. Define the evaluation algorithm or model

[0131] You can use a simple evaluation algorithm or a trained machine learning model to perform the calculation. Here we assume that a simple evaluation algorithm is used.

[0132] 2. Calculate the safety level

[0133] The security level can be determined based on the following factors:

[0134] ·Obstacle distance: The closer you are to an obstacle, the higher the danger level.

[0135] Speed: The faster you move, the higher the risk of potential collision.

[0136] Acceleration: Sudden acceleration or deceleration may cause instability.

[0137] Vibration Amplitude: Abnormally high vibrations may indicate equipment failure.

[0138] A simple security level calculation formula is defined:

[0139] SafetyLevel=α·Distance+β·Speed+γ·Acceleration+δ·VibrationSafety

[0140] Among them, α, β, γ, and δ are weight coefficients used to indicate the importance of each factor;

[0141] Distance, Speed, Acceleration, and Vibration are features extracted from the fused feature vector y, which are distance, speed, acceleration, and vibration amplitude respectively.

[0142] Assume that the weight coefficients are: α = 0.4, β = 0.3, γ = 0.2, δ = 0.1;

[0143] 3. Application Examples

[0144] Assume that the fused feature vector y contains the following information:

[0145] Distance to the nearest obstacle = 0.5 meters

[0146] Current speed Speed ​​= 0.2 m / s

[0147] Current acceleration = 0.1 m / s

[0148] Equipment vibration amplitude Vibration = 0.05 meters

[0149] Substitute the formula to calculate the safety level:

[0150] SafetyLevel=0.4·0.5+0.3·0.2+0.2·0.1+0.1·0.05=0.2+0.06+0.02+0.005=0.285S

[0151] 4. Representation of security level

[0152] Security levels can be expressed in different forms, for example:

[0153] Numerical form: Directly displays the calculated safety level value 0.285.

[0154] ·Grade format: Different levels are divided according to the safety level, such as low risk (0-0.3), medium risk (0.3-0.6), and high risk (0.6-1.0). In this example, the safety level 0.285 belongs to the low risk level.

[0155] Color code: Use colors to indicate safety levels, such as green for low risk, yellow for medium risk, and red for high risk. In this example, a safety level of 0.285 can be represented by green.

[0156] Through the above method, the safety level of the current monitoring area can be calculated based on the fused data and displayed in an intuitive way to help staff quickly understand the current status.

[0157] After completing data fusion and security level calculation, the acquisition module will promptly and accurately pass the processed data packets (including sensor data, security level information, and synchronously captured audio data) to the subsequent sound preprocessing module through a standardized interface or a dedicated communication protocol. The sound preprocessing module will further perform operations such as noise reduction and speech recognition on the audio data, extract sound events, keywords or emotional features, and combine them with security level information to provide rich multimodal input for the system's decision support, alarm triggering, human-computer interaction and other functions.

[0158] Sound preprocessing module: mainly used to receive data from the acquisition module, preprocess the acquired sound data to extract the characteristic value of the sound, and pass the data to other modules of the method of the present invention. Figure 2 The detailed method is as follows:

[0159] Step 1: The algorithm for converting time domain signals to frequency domain is mainly Fourier transform. Fourier transform is a mathematical method for converting signals from time domain to frequency domain. It can decompose complex time domain signals into a superposition of a series of sine waves.

[0160] For discrete signals, the present invention uses Discrete Fourier Transform (DFT) or its fast version, Fast Fourier Transform (FFT). These algorithms convert time domain signals into frequency domain representations by calculating the amplitude and phase of different frequency components in the signal. The algorithm formula for Fourier transform is:

[0161] X[k]=Σ[x[n]*e^(-j*2πnk / N)]

[0162] Where X[k] represents the complex coefficient in the frequency domain, x[n] represents the sampling value of the time domain signal, e is the base of the natural constant, j represents the imaginary unit, π represents the circumference of a circle, n represents the index of the time domain sample, k represents the frequency index in the frequency domain, and N represents the total number of time domain sampling points.

[0163] Step 2: Determine whether the sound exceeds the threshold based on the current security level. If it does not exceed the threshold, it is defined as a normal spectrum feature dataset, otherwise it is an abnormal sound feature.

[0164] Example: Determination of sound threshold and abnormality judgment

[0165] Assume that the safety level of the current monitoring area has been calculated to be 0.285 (low risk level), and what you want to monitor is the operating sound of the equipment.

[0166] 1. Determine the threshold and its basis

[0167] Assume that a sound threshold calculation method based on the safety level is defined, which can take into account the safety level and other relevant factors (such as equipment type, ambient noise level, etc.).

[0168] 2. Threshold calculation method

[0169] Define a simple threshold calculation formula: Threshold = Base + Level Factor

[0170] Among them: Base is the basic threshold, which can be determined according to the normal operating sound range of the device; Level is the current security level; Factor is an adjustment factor used to adjust the impact of the security level on the threshold.

[0171] Assume that the basic threshold Base determined in this embodiment is 50 dB, and the adjustment factor Factor is 10 dB.

[0172] 3. Application Examples

[0173] Substitute the current security level 0.285 to calculate the volume threshold:

[0174] Threshold=50+0.285·10=50+2.85=52.85dB

[0175] 4. Determine whether the sound is abnormal

[0176] Assume that the real-time monitoring result of the sound intensity of the device is 51dB.

[0177] ●Judgment: Because 51dB<52.85dB, the current sound intensity does not exceed the threshold.

[0178] ●Result: The sound belongs to the normal spectral feature dataset.

[0179] If the detected sound intensity is 55dB:

[0180] Judgment: Because 55dB>52.85dB, the current sound intensity exceeds the threshold.

[0181] Result: The sound was considered to have an abnormal sound signature.

[0182] Step 3: Filter out the normal spectrum features from the current sound using spectral subtraction to obtain pure abnormal sound features. The gain function H(k) formula is:

[0183] H(k)=max(1-|N(k)| / |Y(k)|,0)

[0184] Where N(k) represents the amplitude of the noise feature at a certain frequency index k, Y(k) represents the amplitude of the source signal feature at a certain frequency index k, |N(k)| and |Y(k)| represent the absolute value or amplitude of N(k) and Y(k), respectively, which are used to measure the signal strength or energy at the frequency index.

[0185] Spectral subtraction uses a gain function (H(k)) to adjust the amplitude spectrum of the noisy speech to obtain an estimated amplitude spectrum of the clean speech W(k). The formula for spectral subtraction is:

[0186] W(k)=|Y(k)|*H(k)

[0187] Where Y(k) represents the amplitude of the source signal feature at a certain frequency index k, and H(k) represents the relative strength or weight of the sound feature at a certain frequency index k. |Y(k)| represents the absolute value or amplitude of Y(k), which is used to measure the signal strength or energy at the frequency index.

[0188] Step 4: Pass the security level data and sound feature values ​​to the recognition and analysis module for further analysis and processing.

[0189] Recognition and analysis module: mainly used to receive data from the sound preprocessing module, train the sound recognition model, confirm the results based on the sound recognition and security level, and pass the data to other modules of this method. The details are as follows:

[0190] Step 41: Preliminarily determine whether an abnormal event occurs, such as the approach of a person, based on the sound characteristics of the sound preprocessing module. Common determination methods include:

[0191] ①Threshold judgment: Set a volume threshold. When the volume of the sound signal exceeds the threshold, it is considered that an abnormal event may have occurred. At the same time, combined with the spectrum characteristics and time characteristics, further confirm whether it is the sound of a person approaching (such as footsteps).

[0192] The volume threshold calculated in step 2 is T. When the volume of the sound signal exceeds the threshold T, it is considered that an abnormal event may have occurred. Next, the spectrum characteristics and time characteristics will be combined to further confirm whether it is the sound of a person approaching.

[0193] 1. Spectral characteristics analysis

[0194] Footsteps typically have the following spectral characteristics:

[0195] Low-frequency components: Footsteps usually contain strong low-frequency components (such as between 100Hz and 500Hz).

[0196] Periodicity: Footsteps often show a certain periodicity, that is, they repeat regularly within a certain frequency range.

[0197] 2. Time characteristics analysis

[0198] Footsteps typically have the following timing characteristics:

[0199] Duration: Footsteps usually last for a short time, usually within a few hundred milliseconds.

[0200] ●Interval time: There is a certain interval time between consecutive footsteps, usually between a few hundred milliseconds to a few seconds.

[0201] Repeatability: If it is a series of footsteps, the time intervals between them are relatively stable.

[0202] 3. Judgment process

[0203] Assume that we have detected a sound signal whose volume exceeds the threshold T. Next, we will analyze the spectral characteristics and time characteristics:

[0204] Spectrum Analysis:

[0205] Calculate the Fourier transform of the sound signal to obtain the spectrum.

[0206] Analyze the peaks in the spectrum to see if there are strong low-frequency components (such as between 100Hz and 500Hz).

[0207] Observe whether there is any obvious periodicity in the spectrum.

[0208] Time characteristics analysis:

[0209] Calculate the duration of the sound signal to see if it matches the duration characteristics of footsteps.

[0210] If multiple sound signals are detected continuously, calculate the time interval between them to see if they meet the time interval characteristics of footsteps.

[0211] Analyze the repetitiveness of the sound signal to see if it exhibits the characteristics of continuous footsteps.

[0212] 4. Application Examples

[0213] Assume that the spectrum and time characteristics of the sound signal have been calculated:

[0214] Spectral characteristics:

[0215] There is a clear peak between 100Hz and 500Hz.

[0216] There is a certain periodicity in the spectrum.

[0217] Time characteristics:

[0218] The duration of the sound signal is 500 milliseconds.

[0219] If it is a continuous sound signal, the time interval between them is about 1 second.

[0220] The sound signal shows continuous repetitiveness.

[0221] 5. Conclusion

[0222] Based on the above analysis of spectrum characteristics and time characteristics, we can conclude that:

[0223] Spectral characteristics: There are strong low-frequency components and periodicity, which are consistent with the spectral characteristics of footsteps.

[0224] Time characteristics: Duration, interval time and repetition all conform to the time characteristics of footsteps.

[0225] In summary, we can determine that the sound signal is most likely the sound of footsteps caused by a person approaching.

[0226] By combining the spectral characteristics and time characteristics for analysis, we can effectively determine whether the sound signal is the sound of a person approaching, thereby helping the monitoring system to make accurate judgments and responses.

[0227] ② Pattern recognition: Use machine learning algorithms (such as support vector machines, neural networks, etc.) to train sound features and learn the difference between normal sounds and abnormal sounds (such as people approaching). When the newly collected sound features match the abnormal sound patterns in the trained model, the system will sound an alarm.

[0228] For example, the acquisition module obtains a sound signal. After preprocessing, the system extracts the volume, spectral characteristics, and time characteristics of the sound. The system finds that the volume suddenly increases, and the spectral characteristics match the pre-trained "footsteps" pattern. At the same time, the sound signal shows periodic fluctuations, which is consistent with the walking rhythm of a person. Based on these characteristics, the system determines that an abnormal event has occurred, that is, someone is approaching, and triggers the corresponding alarm or response measures.

[0229] Step 42, match the sound recognition result with the security level. If the security level indicates that an object is approaching or entering the warning area when the sound event occurs, the accuracy of the sound recognition result is further confirmed. On the contrary, if the security level does not show that an object is approaching, it may be a misjudgment of the sound recognition, and further analysis or ignoring the result is required. At the same time, the recognition model is supervised and trained using the result, as follows;

[0230] a. In the early stage of system operation, the sound feature data is annotated according to the results of the security level. In the long-term operation process, a data set of normal sound and abnormal sound samples is gradually formed for training and testing the sound recognition model;

[0231] b. Construct a sound recognition model, mainly using convolutional neural networks (CNN) and recurrent neural networks (RNN) to automatically learn useful features in sound data, so as to better distinguish different sound sources. The convolutional neural network uses the Leaky ReLU activation function, which not only retains the probability but also has more robustness. The sound recognition model structure used in this embodiment is as follows: Figure 3 As shown, it includes input layer, convolution layer, pooling layer, circulation layer, fully connected layer and output layer;

[0232] The input layer is responsible for receiving the original sound data, usually the spectrogram or Mel-frequency cepstral coefficients (MFCCs) of the sound signal and other features; the input data is represented in the form of a two-dimensional array: (number of time frames, number of frequency channels); if the sound signal is converted into Mel-frequency cepstral coefficients, then the input data can be represented as (100, 40), where 100 represents the number of time frames and 40 represents the number of frequency channels.

[0233] The convolution layer slides on the input data through multiple filters (convolution kernels) to extract local features; the Leaky ReLU activation function is usually used to retain more information and improve the robustness of the model; its input data is represented in the form of a three-dimensional array: (batch size, number of time frames, number of frequency channels); if the input data is (100, 40) and the batch size is 32, the convolution layer input data is represented as (32, 100, 40); the convolution layer output data is represented in the form of a three-dimensional array: (batch size, new number of time frames, new number of frequency channels). Assuming that the convolution layer uses 64 filters, the convolution layer output data can be represented as (32, 98, 64).

[0234] The pooling layer includes downsampling and smooth output. The downsampling reduces the spatial dimension and reduces the computational complexity. The smooth output reduces overfitting and improves the generalization ability of the model. The input data representation is the same as the output of the convolutional layer, which is a three-dimensional array. The output data representation is a three-dimensional array: (batch size, smaller number of time frames, fewer number of frequency channels). If the pooling layer adopts maximum pooling and the pooling window size is 2x2, the output data is represented as: (32, 49, 32).

[0235] The recurrent layer is used to capture the time series dependencies in the sound signal and to maintain the information of the previous time step, which is very important for processing time series data. The input data of the recurrent layer is represented as a three-dimensional array: (batch size, number of time steps, number of frequency channels); the output data is represented as a three-dimensional array: (batch size, number of time steps, number of hidden units). Assuming that the RNN layer uses 128 hidden units, the output data is represented as (32, 49, 128).

[0236] The fully connected layer is used to integrate features, specifically the features learned by the previous layers, for classification or regression tasks. Its input data is represented as a two-dimensional array: (batch size, number of features). If the output shape of the RNN layer is (32, 49, 128), the output of the last time step needs to be flattened to (32, 128); its output data is represented as a two-dimensional array: (batch size, number of categories). If the classification task has 10 categories, the output shape is (32, 10).

[0237] The output layer is used to generate the final prediction. Specifically, the output layer usually uses a softmax function (if it is a classification task) or a linear activation function (if it is a regression task) to generate the final prediction result. Its input data representation is the same as the output of the fully connected layer, which is a two-dimensional array; its output data representation is a two-dimensional array: (batch size, number of categories). If the classification task has 10 categories, the output data set is represented as (32,10).

[0238] By combining the above layers, an efficient sound recognition model can be built to distinguish different sound sources.

[0239] The input, output, and iteration stop conditions during the sound recognition model training process are as follows:

[0240] Input: The training dataset contains multiple audio files and their corresponding labels (i.e., sound source categories); in each training step, audio files are read in batches, converted into spectrograms or preprocessed waveforms, and passed to the model as input.

[0241] Output: The model outputs the probability distribution of each audio file belonging to each sound source category; the cross entropy loss function is used to calculate the error between the model output and the true label.

[0242] Iteration stopping condition: reaching the preset number of iterations (epoch). The loss or accuracy on the validation set does not improve significantly in multiple consecutive iterations (early stopping method).

[0243] Monitor overfitting during training and stop training early if performance on the training set improves significantly while performance on the validation set stagnates or decreases.

[0244] c. As the data set increases, the collected sound data will be repeatedly tested and trained, and the algorithm will be continuously optimized over the long term. Ultimately, the system will rely on sound recognition as the main basis for judgment.

[0245] Step 43: Based on the results of the voice recognition and the security level confirmation, the recognition result is transmitted to the control module for use.

[0246] Control module: mainly used to receive data from the identification and analysis module and match it with the preset threshold of the system to generate corresponding control instructions.

[0247] 1. Data reception and analysis

[0248] a. Standardization of data interface The control module has a communication interface compatible with the identification and analysis module, such as serial port, Ethernet, CAN bus, wireless communication, etc., to ensure real-time and accurate reception of data packets after analysis and processing. Data packets usually contain various sensor monitoring indicators, abnormal event identification, risk level and other information, and are transmitted in standard or customized data formats, such as JSON, XML, binary encoding, etc. The control module needs to have efficient data parsing capabilities to convert the received raw data into a structured internal representation for subsequent processing.

[0249] b. Data integrity and validity verification After receiving the data, the control module performs strict data integrity checks, such as checksum, cyclic redundancy check (CRC), message sequence number, etc., to ensure that the data has not been tampered with or lost during transmission. At the same time, the validity of the data is judged, such as whether the data is within the valid range, whether the timestamp is reasonable, whether the key fields are missing, etc., to eliminate invalid or erroneous data and ensure that subsequent decisions are based on reliable information.

[0250] 2. Data matches the preset threshold

[0251] a. Threshold setting The thresholds preset by the system are determined based on multiple factors such as equipment operation characteristics, safety specifications, historical data analysis, etc., covering various key monitoring indicators. These thresholds may include but are not limited to vibration amplitude thresholds, fault warning levels, etc. The threshold setting should have a certain dynamic adjustment mechanism to adapt to changes in equipment operating conditions or be updated as the system learns and optimizes.

[0252] b. Real-time comparison and trigger judgment The control module compares the analyzed real-time monitoring data with the preset thresholds item by item. For continuous monitoring indicators, numerical comparison is used; for discrete events or state changes, logical judgment is performed. When the monitoring data exceeds (or is lower than) the threshold, it is considered that the trigger condition is met and may trigger the corresponding control action. The control module needs to implement complex logical operation functions to make accurate judgments.

[0253] 3. Control instruction generation

[0254] a. Control strategy Based on the result of matching data with thresholds, the control module generates corresponding instructions according to the predefined control strategy. The control strategy usually includes automatic adjustment instructions in normal operation mode (such as adjusting equipment operating parameters to maintain optimal efficiency or extend life), fault warning or response instructions (such as triggering alarms, degrading operation, automatic shutdown, etc.), and preventive maintenance suggestions (such as recommending replacement of parts, arranging regular inspections, etc.). The strategy design should take into account factors such as equipment safety, production efficiency, resource consumption, etc., and follow the principle of fail-safe.

[0255] b. Instruction encoding and sending The generated control instructions are encoded according to the instruction format specified by the system to ensure that the downstream execution module can correctly interpret and execute them. The encoding process may include steps such as instruction type identification, operation parameter packaging, and checksum calculation. After the encoding is completed, the control module sends the instructions through the communication interface connected to the execution module using the determined communication protocol (such as Modbus, Profinet, OPC UA, etc.).

[0256] c. Feedback and closed-loop control The control module has the ability to receive feedback information from the execution module to form a closed-loop control system. After receiving feedback information from the execution module about the execution results of the instructions, the control module adjusts the subsequent instructions or triggers compensation control actions according to the feedback content to ensure the stable operation of the system and the accurate achievement of the target state. For complex systems, it may also be necessary to integrate advanced control algorithms, such as PID controllers, model predictive control (MPC), etc., to achieve more sophisticated dynamic control.

[0257] Warning control module: mainly used to control the color changes of multi-color spotlights around the equipment, so as to intuitively and eye-catchingly mark the dangerous conditions of the equipment and realize safety warning and status visualization.

[0258] Step A1, spotlight layout design and installation: According to the physical size and structural characteristics of the equipment, accurately plan the layout of multi-color spotlights. Select a suitable location on the edge of the equipment to install the spotlight to ensure that the light is evenly distributed and there is no blind spot. The spotlight should be set to a vertical projection direction, that is, the light is directly on the ground and evenly arranged along the contour of the equipment to form a circle of continuous light bands to ensure 360° coverage of the area around the equipment. In addition, considering the working environment of the equipment and possible obstruction factors, appropriately increase the number of spotlights or adjust the irradiation angle to ensure that the warning effect is not affected.

[0259] Step A2, safety level signal connection: closely integrated with the equipment control system or independent safety monitoring system, receiving the safety level signal output by the processing module in real time. These signals reflect the current operating status of the equipment, fault warning, abnormality detection and other key information, and are the basis for dynamically adjusting the color of the spotlight.

[0260] Step A3, color coding and response rule setting: Establish a clear and easily identifiable color coding system to correspond different safety levels to specific colors. For example, green represents normal operation of the equipment, yellow indicates minor abnormalities or warnings, orange indicates serious warnings, and red indicates emergency shutdowns or major dangerous conditions. Develop detailed response rules to clarify the target colors that the spotlights should switch to when signals of different safety levels are triggered, as well as the timing and speed of color changes, to ensure accurate communication of warning information.

[0261] Step A4, real-time dynamic control: When receiving the security level signal change sent by the processing module, the warning control module immediately starts the corresponding color control process. By precisely controlling the color output of each spotlight, the overall color of the light band around the device is synchronously changed to quickly and accurately reflect the current safety status of the device. During the color change process, a smooth transition is maintained to avoid visual misleading caused by sudden changes.

[0262] Step A5, fault self-check and alarm: The warning control module has a self-monitoring function, and regularly checks the working status of the spotlight, such as light source life, power supply, communication link, etc., to ensure that it is always in good working condition. Once a spotlight failure or communication interruption is found, the internal alarm is immediately triggered, and the fault information is reported to the superior control system for timely maintenance or replacement to ensure the stable operation of the warning system.

[0263] Step A6, user customization and expansion: Support users to personalize parameters such as color coding, response rules, brightness adjustment, etc. according to actual needs. At the same time, reserve interfaces with other security systems or smart devices to facilitate future system upgrades or function expansions, and achieve higher levels of linkage warnings and emergency responses.

[0264] Display module: As the core component of the system's human-computer interaction, it is committed to presenting complex system parameters, real-time sensor data, and alarm information to users in an intuitive and easy-to-understand manner, improving operational convenience and system monitoring efficiency.

[0265] 1. Visual configuration of system parameters

[0266] a. Parameter classification and layout The display module scientifically classifies system parameters, such as device status parameters, operating parameters, setting parameters, etc., and clearly lays them out on the interface. Use paging, tabs, folding panels, etc. to organize a large number of parameters to ensure that users can quickly locate the required information. Parameters are displayed in the form of lists, tables, tree structures, etc., with search and filtering functions to facilitate user query and management.

[0267] b. Visual editing and configuration For configurable system parameters, the display module provides an intuitive graphical editing interface. For example, using controls such as sliders, numeric input boxes, and selection lists, users can adjust parameter values ​​directly on the interface. For parameters with strong correlation or constraints, users can be assisted in making reasonable configurations through linkage effects, dynamic prompts, and real-time previews. Configuration changes support operations such as save, undo, and redo, and are synchronized and updated to the control system in the background.

[0268] c. The parameter trend and statistical analysis display module integrates data visualization components, such as line charts, bar charts, pie charts, heat maps, etc., to convert historical parameter data into charts to show the trend of parameter changes over time, periodic laws, correlation analysis, etc. In addition, it provides statistical report functions to summarize statistical indicators such as the average value, maximum value, minimum value, standard deviation, etc. of key parameters to help users deeply understand system performance and make decisions.

[0269] 2. Display of real-time sensor data

[0270] a. Real-time data refresh and dynamic display

[0271] The display module is tightly integrated with the data acquisition system to receive and update sensor data in real time. It uses animation effects, color changes, value flashing and other means to intuitively reflect the dynamic changes of data. For important or frequently changing parameters, special display methods such as data scroll bars and data flow views are set to ensure that users can capture key information in a timely manner.

[0272] b. Multi-dimensional data display and linkage

[0273] For multi-sensor, multi-dimensional monitoring data, the display module supports simultaneous display of multiple views in the same interface, such as distributed equipment status on the map, multi-parameter evolution on the timeline, temperature distribution of various parts inside the equipment, etc. Through interactive operations such as hovering, clicking, and dragging the mouse, linkage browsing between different views is achieved, which facilitates users to fully grasp the system status from macro to micro.

[0274] c. Abnormal data labeling and early warning prompts

[0275] When sensor data exceeds the preset threshold or shows abnormal fluctuations, the display module highlights the abnormal data points by changing the data color, adding warning icons, flashing borders, etc. At the same time, a data warning area is provided to list the current alarm information in real time, including details such as alarm level, occurrence time, associated sensors, trigger conditions, etc.

[0276] 3. Voice alarm function

[0277] a. Alarm trigger logic

[0278] The display module is deeply integrated with the alarm system to generate alarm events based on system status, sensor data, and preset alarm rules. The alarm trigger logic takes into account factors such as alarm level, delayed confirmation, and repeated alarm suppression to ensure the accuracy and effectiveness of the alarm.

[0279] b. Speech synthesis and broadcasting

[0280] For critical alarms or alarm events that users are concerned about, the display module uses text-to-speech (TTS) technology to convert alarm information into natural and smooth voice output. The voice content includes elements such as alarm level, alarm cause, and recommended operations. The speech speed, volume, and tone can be dynamically adjusted according to the urgency of the alarm. It supports multiple language settings to meet the needs of different users.

[0281] Data storage module: As an important infrastructure of the system, it is responsible for safely and efficiently storing the large amount of data collected synchronously by the ranging equipment and the sound recognition system, ensuring that this information can be used for subsequent historical data analysis, query and application.

[0282] 1. Data types and structured storage

[0283] a. Distance measurement data

[0284] The ranging data usually contains the distance measurement value, timestamp, ranging device ID, target object identification (if any), ranging method (such as laser, ultrasonic, radar, etc.), measurement accuracy, environmental factors (such as temperature, humidity, etc.), and other related information. The data storage module stores these data in a structured manner according to a predefined data model, such as creating a dedicated ranging data table that contains corresponding fields to ensure the standardization and consistency of the data.

[0285] b. Voice recognition data

[0286] The data collected synchronously by voice recognition includes original audio files, audio feature vectors, recognition results (such as text transcription, keyword extraction, sentiment analysis, etc.), timestamps, microphone positions, recognition engine versions, recognition confidence, background noise levels, etc. These data also need to be stored in a structured manner, which may involve multiple associated tables such as the voice data table and the recognition result table to support complex query and analysis requirements.

[0287] 2. Efficient data writing and real-time guarantee

[0288] a. Data stream processing and batch writing

[0289] In the face of the large amount of ranging and voice recognition data that is continuously generated, the data storage module uses data stream processing technologies, such as message queues and stream computing platforms, to achieve data buffering, peak load shifting, and asynchronous processing to ensure the stability and efficiency of the data writing system. At the same time, it supports batch writing operations, reduces the number of database I / O times, and improves writing performance.

[0290] b. Transaction management and data consistency

[0291] When writing data from multiple sources simultaneously, the data storage module implements a strict transaction management mechanism to ensure that the ranging data and the corresponding sound recognition data are submitted atomically within the transaction to avoid data inconsistency. Technologies such as optimistic locking, pessimistic locking, and distributed locking are used to prevent concurrent writing conflicts and maintain data integrity.

[0292] 3. Storage optimization and capacity management

[0293] a. Tiered storage and separation of hot and cold data

[0294] The data storage module adopts a tiered storage strategy, such as storing frequently accessed hot data on high-speed SSD media, and moving long-term archived cold data to lower-cost HDD or cloud storage services. Regularly perform data heat analysis, dynamically adjust the distribution of data at different storage levels, and optimize storage costs and access performance.

[0295] b. Data compression and deduplication

[0296] Efficiently compress the numerical data in the distance measurement data and the audio files in the sound recognition data to reduce the storage space occupied. For repeated distance measurement scenes or sound clips, implement data deduplication strategy, store only unique data instances, and retain the reference relationship pointing to the actual data, further saving storage resources.

[0297] 4. Data indexing and query performance

[0298] a. Multidimensional index construction

[0299] To support efficient historical data query, the data storage module establishes multi-dimensional indexes for key fields of ranging data and sound recognition data, such as timestamp, device ID, target object, recognition keywords, etc. B+ tree, hash index, bitmap index, spatial index (for geographic location data) and other technologies are used to accelerate the query process.

[0300] b. Query interface and complex query support

[0301] It provides standardized query interfaces (such as SQL and RESTful API), allowing users to flexibly combine query conditions and perform complex operations such as range query, aggregation query, and association query. It supports spatiotemporal query (such as ranging data within a specified time period, sound events within a specific geographic fence) and full-text search (such as keyword search based on sound transcription text).

[0302] 5. Data backup and recovery

[0303] a. Regular backup and incremental backup

[0304] Implement regular full backup and incremental backup strategies to ensure data security. Full backup covers all data to prevent large-scale data loss; incremental backup only records changes since the last backup, reducing backup frequency and storage costs. Backup data is stored off-site or uploaded to a cloud backup service to increase disaster recovery redundancy.

[0305] b. Snapshot and point-in-time recovery

[0306] Supports data snapshot function, quickly creates a copy of the data status at a certain moment, which is convenient for data analysis or rollback to a specific time point. When data is accidentally deleted or needs to be restored to a certain historical state, data can be quickly restored from the snapshot to reduce business interruption time.

[0307] In order to facilitate those skilled in the art to further understand the monitoring method of the present invention for equipment warning and preventing accidental entry by using sound recognition technology, the following is a detailed description in the form of a flow chart. Figure 4 As shown:

[0308] 1. The acquisition module collects data from multiple sensors at a fixed frequency and simultaneously captures the sounds around the device. By comprehensively analyzing the sensor data, it accurately calculates the security level of the current monitoring area, and then passes the security level data and sound to the sound preprocessing module.

[0309] 2. After receiving the security level and sound data from the acquisition module, the sound preprocessing module preprocesses the sound data and extracts its feature values. The security level data and sound feature values ​​are then passed to the recognition and analysis module for further analysis and processing.

[0310] 3. After receiving the security level and sound feature value data from the sound pre-processing module, the recognition and analysis module immediately inputs the sound feature value and security level into the pre-trained sound recognition model to perform accurate sound signal recognition. After the recognition is completed, the processing module matches the sound recognition result with the previously calculated security level, and uses the data of the ranging sensor to assist in training and optimizing the sound recognition model to further improve its performance, and at the same time passes the recognition result to the control module.

[0311] The steps to use ranging sensor data to assist in training and optimizing the sound recognition model are as follows:

[0312] Data collection and integration

[0313] Step 1: Synchronous data collection: Ensure that the sound preprocessing module, ranging sensor and sound recognition system can collect data synchronously. This means that the measurement data of the ranging sensor at the same time point needs to be recorded while the sound is being collected.

[0314] Step 2: Data labeling: The collected sound data is labeled using the sound feature value, security level, and corresponding spatial location information (obtained through the ranging sensor). These labels will be used in the subsequent training and verification process.

[0315] Data preprocessing

[0316] Step 1: Spatial Position Encoding: Convert the data from the ranging sensor into a form that can be understood and utilized by the sound recognition model. This may include converting the distance into a relative position encoding, angle information, or other spatial feature representations.

[0317] Step 2: Feature fusion: Fuse the sound feature values ​​with the spatial position information to form a feature vector that contains more contextual information. This can be achieved through concatenation, weighting, or more complex feature fusion techniques.

[0318] Model training and optimization

[0319] Step 1: Modify model structure: Based on the fused feature vector, the structure of the sound recognition model may need to be adjusted to better handle inputs containing spatial information.

[0320] Step 2: Train the model: Use the fused dataset to train the sound recognition model. During the training process, the model will learn how to associate sound features, security levels, and spatial location information to improve recognition accuracy.

[0321] Step 3: Performance Evaluation: Evaluate the performance of the model by leaving out a test set or using methods such as cross-validation.

[0322] Special attention is paid to the performance of the model when the spatial position changes, in order to verify the effect of spatial information on improving recognition accuracy.

[0323] Model Optimization

[0324] Step 1: Hyperparameter Tuning: Based on the performance of the model on the test set, adjust the model’s hyperparameters (such as learning rate, batch size, regularization strength, etc.) to further optimize the model’s performance.

[0325] Step 2: Feature selection and weight adjustment: Analyze the contribution of different features and spatial information to model performance, and optimize the model by adjusting feature weights or selecting more representative features.

[0326] Step 3: Continuous Learning: As new data is continuously collected, the model is regularly updated and retrained to capture new sound patterns and spatial variations, keeping the model current and accurate.

[0327] 4. The control module stores the data related to the processing results, displays them on the display screen, and transmits them to the warning control module to achieve accurate control of the color of the multi-color spotlights.

[0328] 5. If the predetermined danger level is reached, the control module automatically controls the equipment to execute the shutdown operation to ensure safety.

[0329] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the scope of protection of the present invention is not limited to such specific statements and embodiments. For those skilled in the art, the present invention may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A monitoring method for equipment warning and preventing accidental entry using sound recognition technology, characterized in that: include: S1, synchronously collect sound data and environmental data; The environmental data at least includes spatial location information; S2. Preprocess the collected environmental data to obtain a security level; S3, preprocessing the sound data to obtain characteristic values ​​of the sound data; S4, using the characteristic value of the sound, the security level and the corresponding spatial position information to mark the characteristic value of the sound data; S5, constructing a sound recognition model; after the known sound data and environmental data are processed in steps S1-S4, they are input into the constructed sound recognition model for training; S6. After being processed in steps S1-S4, the sound data and environmental data to be detected are input into the trained sound recognition model to obtain the recognition result.

2. According to claim 1, a monitoring method for equipment warning and preventing accidental entry using sound recognition technology is characterized in that: The environmental data in step S1 is collected based on laser radar, photoelectric sensor, millimeter wave sensor, vibration sensor, and ultrasonic sensor.

3. The method for monitoring equipment by using sound recognition technology to prevent accidental entry according to claim 2, characterized in that: The preprocessing described in step S2 specifically includes sequentially performing data synchronization, data calibration, feature extraction, and data fusion processing on different sensor data to obtain a fused feature vector.

4. The method for monitoring equipment by using sound recognition technology to prevent accidental entry according to claim 3 is characterized in that: In step S2, the security level is calculated based on the fused feature vector.

5. The method for monitoring equipment by using sound recognition technology to prevent accidental entry according to claim 4 is characterized in that: The safety level calculation formula is: SafetyLevel=α·Distance+β·Speed+γ·Acceleration+δ·VibrationSafety Among them, α, β, γ, δ are weight coefficients, and Distance, Speed, Acceleration, and Vibration are features extracted from the fused feature vector y, which are distance, speed, acceleration, and vibration amplitude, respectively.

6. The method for monitoring equipment by using sound recognition technology to prevent accidental entry according to claim 5, characterized in that: The preprocessing process in step S3 includes the following steps: S31, converting the current sound data from the time domain signal to the frequency domain; S32, calculating a threshold of the current sound data based on a security level calculated from environmental data collected synchronously with the current sound data; S33, if the intensity corresponding to the current sound data is greater than the threshold calculated in step S32, it is determined that there is an abnormal sound feature, and step S34 is executed; otherwise, the sound data is classified as a normal spectrum feature data set, and then step S4 is executed; S34, using spectral subtraction to filter out normal frequency spectrum features from the current sound data processed in step S31, so as to obtain pure abnormal sound features, and then executing step S4.

7. The method for monitoring equipment by using sound recognition technology to prevent accidental entry according to claim 6, characterized in that: The sound recognition model in step S5 includes an input layer, a convolutional layer, a pooling layer, a recurrent layer, a fully connected layer and an output layer; the input layer is used to receive the sound data annotated by step S4, and the convolutional layer is used to extract local features of the sound data received by the input layer; the pooling layer downsamples and smoothes the local features output by the convolutional layer; the input of the recurrent layer to the pooling layer is the output of the pooling layer, which is used to capture the time series dependency in the sound signal; the input of the fully connected layer is the output of the recurrent layer; the output of the fully connected layer is used as the input of the output layer, and the output layer uses a softmax function or a linear activation function to generate the final prediction result.

8. The method for monitoring equipment by using sound recognition technology to prevent accidental entry according to claim 7, characterized in that: The output of the sound recognition model is the probability distribution of each sound source category; the cross entropy loss function is used during training to calculate the error between the model output and the true label.

Images