Method and system for regional automatic sound wave expelling of violation personnel

Abstract

The invention relates to the technical field of automatic expelling, and discloses a method and a system for regional automatic sound wave expelling of violation personnel, which realize accurate identification, behavior pattern judgment and intelligent expelling of violation personnel through technologies of multi-modal sensor fusion, deep learning target detection, time sequence behavior analysis, self-adaptive sound wave expelling and the like. The system overcomes the defects of monitoring precision, violation recognition accuracy and expelling effect in the prior art, can be widely applied to airports, factories, military bases, dangerous operation areas and other scenes, and improves the intelligent level of safety management.

Description

Technical Field

[0001] The present invention relates to the field of automatic expulsion technology, and in particular to a method and system for regional automatic sonic expulsion of illegal personnel. Background Art

[0002] Currently, in many security application scenarios, such as airports, factories, military bases, etc., manual patrols, traditional video surveillance or access control systems are usually used to prevent unauthorized entry. However, manual patrols have high time and labor costs, and cannot achieve all-weather monitoring without blind spots; traditional video surveillance relies on the observation of monitoring personnel, and it is easy for intrusion incidents to not be discovered in time due to human negligence; access control systems are only applicable to closed areas. For large open areas or semi-enclosed areas, it is difficult to effectively prevent intrusion by outsiders. In addition, existing intrusion detection methods mainly rely on video analysis technology, which is greatly affected by lighting conditions and is difficult to provide reliable monitoring effects at night or in bad weather conditions. Summary of the invention

[0003] In view of this, the purpose of the present invention is to provide a regional automatic sound wave method and system to drive away illegal persons, so as to solve the problem that it is difficult to effectively and timely prevent the intrusion of outsiders in the current security management of open areas or semi-enclosed areas.

[0004] The first aspect of the present invention discloses a regional automatic sound wave method for driving away illegal personnel, the method comprising the following steps: S1. Deploy a multimodal sensor in the target area to monitor the personnel entering the perimeter of the target area in real time to obtain real-time monitoring data; the multimodal sensor includes an infrared thermal imaging sensor, a millimeter wave radar sensor, an identity recognition device, and a visible light camera; S2. Using the deep learning target detection model to identify the personnel entering the perimeter of the target area as normal workers or violators based on the monitoring data, and after identifying the violators, combining the time series analysis model to determine the behavior patterns of the violators; S3, based on the behavior pattern of the violator, determine the type of violation behavior of the violator through the violation determination strategy; S4. Select a corresponding expulsion mode to expel the violator according to the type of violation behavior of the violator.

[0005] Furthermore, after acquiring the real-time monitoring data and before executing step S02, the method further includes: A sensor data fusion operation is performed on the acquired real-time monitoring data to obtain first monitoring data; the data fusion operation includes a time alignment operation on the data based on Kalman filtering.

[0006] Furthermore, the expulsion mode includes a low-level warning mode, a medium-level expulsion mode, and a high-level emergency expulsion mode; wherein, The low-level warning mode includes emitting low-frequency sound waves to warn and drive away violators; The medium-level expulsion mode includes emitting medium-frequency sound waves to the violators and driving them away in combination with warning lights; The high-level emergency expulsion mode includes emitting high-frequency sound waves to the violators, combined with flashes to interfere with the violators; The sound waves are emitted by a directional sound wave emitter.

[0007] Furthermore, the types of violation behaviors include mistaken entry violation, tentative violation, intentional intrusion violation, latent violation and forced intrusion violation; wherein, When a violation is determined to be a tentative violation, a low-level warning mode is used; When a violation is determined to be an intentional trespassing violation, a medium-level removal mode is adopted; When a potential violation is identified, a high-level emergency evacuation mode will be adopted; When a forced invasion violation is determined, a high-level emergency eviction mode is adopted and the linkage security mechanism is triggered for handling by security personnel.

[0008] Furthermore, the violation determination strategy is based on a multi-task learning model and combines the behavior parameters in the behavior pattern to determine the violation behavior type of the violator; The behavior parameters include the violator's moving speed, moving direction, stay time, stay location, and distance from normal operating personnel.

[0009] Furthermore, the process of transmitting sound waves to the violators is adaptive sound wave transmission, which specifically includes: Obtain the location of the target violator, adjust the acoustic wave transmission power of the acoustic wave transmitter based on the location of the target violator, and adaptively adjust the acoustic wave intervention strategy according to the environmental noise interference level; When the expulsion mode is a high-level emergency expulsion mode, adjusting the sound wave transmission power of the sound wave transmitter based on the position of the target violator specifically includes: The sonic emission power of the sonic transmitter is adjusted based on the location of the target violator and the hearing sensitivity range to cause the violator to reach maximum discomfort.

[0010] Furthermore, the method further includes determining the violation area based on the positions of the violating personnel and the normal operating personnel, and after determining the violation area, using phased array ultrasonic technology to adjust the transmission array phase of the sound wave transmitter so that the sound wave acts only on the violation area; When the violator moves, the multiple angle-adjustable sound wave units of the sound wave transmitter automatically adjust the sound wave emission angle according to the violator's moving path.

[0011] Furthermore, the step S2 specifically includes the following sub-steps: S21. Perform target detection on the first monitoring data through a pre-trained deep learning target detection model, identify all personnel entering the perimeter of the target area, and extract image features of the personnel; S22. Perform identity matching based on the image characteristics of the personnel through the personnel database. If there is no match in the personnel database, the person is determined to be a violator, and step S23 is executed after the person is determined to be a violator; S23. Collect behavioral parameters of the violators based on the first monitoring data, input the behavioral parameters into a time series analysis model, and determine the behavioral patterns of the violators through the time series analysis model.

[0012] Furthermore, the step S23 includes the following sub-steps: S231. Collecting behavioral parameters of violators based on the first monitoring data; S232. Based on the Transformer model of the self-attention mechanism, the behavioral parameters of the violators are modeled in time series to extract the behavioral characteristics of the violators; S233. Setting the criteria for determining the violation behavior patterns, the violation behavior patterns include abnormal stay pattern, rapid approach to sensitive area pattern, long-term wandering pattern, detour entry pattern, and high-intensity abnormal pattern; S234. The behavioral characteristics of the violators are matched with the criteria for determining the violation behavior pattern through the Transformer model, and the behavior pattern of the violators is determined when the behavioral characteristics of the violators are consistent with the violation behavior pattern.

[0013] The second aspect of the present invention discloses a regional automatic sound wave system for driving away illegal personnel. The system is implemented based on the method disclosed in the first aspect. The system includes a data acquisition module, an identification module, a behavior analysis module, and a driving away module; wherein: The data acquisition module is used to deploy multimodal sensors in the target area to monitor the personnel entering the perimeter of the target area in real time and obtain real-time monitoring data; the multimodal sensors include infrared thermal imaging sensors, millimeter wave radar sensors, identity recognition devices and visible light cameras; The recognition module is used to identify the personnel entering the perimeter of the target area as normal operating personnel or illegal personnel based on the monitoring data through the deep learning target detection algorithm; The behavior analysis module is used to determine the behavior pattern of the violator in combination with the time series analysis model after the violator is identified, and determine the type of violation behavior of the violator through the violation determination strategy based on the result of the violation pattern determination; The expulsion module is used to select the corresponding expulsion mode to expel the violators according to the type of their violation behavior.

[0014] Compared with the prior art, the present invention has the following beneficial effects: The present invention uses a multimodal sensor to monitor the target area in real time, and uses Kalman filtering to perform data fusion, thereby improving the time alignment capability of sensor data, and can maintain a high accuracy of personnel detection in complex environments (such as low light and bad weather), effectively reducing false alarms and missed reports. Secondly, by combining the deep learning target detection algorithm with the time series analysis model, it can not only accurately identify the identity of the person entering the area, but also distinguish the types of violations such as probing, deliberate intrusion, and lurking behavior through behavioral pattern analysis, making the determination of violations more intelligent and targeted. In addition, the present invention sets an adaptive acoustic wave expulsion strategy. After the violation is identified, according to the severity of the violation, a low-frequency, medium-frequency or high-frequency acoustic wave intervention mode is adopted, and combined with phased array ultrasonic technology, the acoustic wave energy is focused on the violation area, and the violators are accurately intervened to avoid interference with normal operating personnel. The present invention further dynamically adjusts the acoustic wave emission angle in combination with the moving path of the violators to make the expulsion effect more efficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of the economic application, and do not constitute a limitation of the embodiments of the present invention. In the drawings: Figure 1 The present invention is a flowchart of a method for automatically using sound waves to drive away offenders in a regional manner disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0016] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Example

[0017] The first aspect of the present invention discloses a regional automatic sound wave method for driving away illegal personnel, see Figure 1 , Figure 1 The present invention discloses a method for automatically using sound waves to drive away illegal personnel in a regional manner, which method includes the following steps: S1. Deploy multimodal sensors in the target area to monitor people entering the perimeter of the target area in real time and obtain real-time monitoring data. The multimodal sensors include infrared thermal imaging sensors, millimeter wave radar sensors, identity recognition devices, and visible light cameras.

[0018] S2. Use the deep learning target detection model to identify people entering the perimeter of the target area as normal workers or violators based on monitoring data, and after identifying them as violators, use the time series analysis model to determine the behavior patterns of violators.

[0019] S3. Determine the result based on the behavior pattern of the violator and determine the type of violation behavior of the violator through the violation determination strategy.

[0020] S4. Select a corresponding expulsion mode to expel the violator according to the type of violation behavior of the violator.

[0021] It should be noted that the method of the present invention is mainly used in security management scenarios, such as airports, factories, military bases, and dangerous work areas, preferably open areas or semi-enclosed areas.

[0022] Specifically, the purpose of the present invention to select infrared thermal imaging sensors, millimeter wave radar sensors, identity recognition devices and visible light cameras as monitoring equipment is to improve the accuracy of personnel identification, environmental adaptability and data redundancy, and ensure that the system can work stably under different lighting, weather and occlusion conditions. The infrared thermal imaging sensor can detect the thermal radiation characteristics of objects in the target area, and can still effectively detect human targets in visible light restricted environments such as night, smoke, rain and fog, ensuring that it still has the ability to identify people in low visibility environments. The millimeter wave radar sensor can provide high-precision distance, speed, and motion trajectory information based on the reflection characteristics of electromagnetic waves, and still has strong detection capabilities for targets wearing concealed objects, which can make up for the shortcomings of infrared thermal imaging in long-distance detection, and at the same time provide key data such as target movement speed and direction for behavior pattern analysis. The identity recognition device is used to authenticate the identity of people entering the area. It can match authorized entrants based on technologies such as RFID, Bluetooth, facial recognition, and iris recognition to avoid misjudging normal operating personnel as illegal personnel and improve the recognition accuracy of the system. The specific recognition technology adopted in identity authentication is not specifically limited in the embodiments of the present invention. Visible light cameras are used to provide high-resolution personnel image data, which can be combined with deep learning target detection models to achieve more accurate personnel detection, posture recognition, and violation determination, and can be integrated with other sensor data to enhance the system's comprehensive recognition capabilities.

[0023] Through the combination of the above-mentioned multimodal sensors, the present invention can not only accurately detect human targets in complex environments, but also integrate the data advantages of different sensors, reduce possible misjudgments of a single sensor, and improve the accuracy of personnel identification, behavior pattern analysis, and violation judgment, thereby ensuring the reliability and intelligence level of the system in various application scenarios.

[0024] After acquiring the real-time monitoring data and before executing step S02, a sensor data fusion operation is performed on the acquired real-time monitoring data to obtain first monitoring data; the data fusion operation includes a time alignment operation on the data based on Kalman filtering.

[0025] Specifically, due to the different sampling rates and delays of infrared thermal imaging sensors, millimeter wave radar sensors, identity recognition devices, and visible light cameras, directly using raw data may result in mismatched detection results of the target object at different time points. In an embodiment of the present invention, in order to ensure the consistency of multimodal sensor data in time and space and improve the recognition accuracy of the subsequent deep learning target detection model, before performing personnel identification (step S2), the acquired real-time monitoring data is first fused to obtain the first monitoring data. This process includes using the Kalman Filter algorithm for time alignment to ensure that data from different sensors can be calculated and analyzed under the same time reference.

[0026] Preferably, a time synchronization model is constructed based on the Kalman filter algorithm, and the historical state and predicted state of the sensor data are used for optimal estimation, thereby eliminating the time deviation of the data and making the measurement results of different sensors consistent under the same time reference. Specifically, the execution process of the Kalman filter is as follows: State initialization: define the state vector of the tracked target (the person entering the perimeter of the area), including the target position (X, Y, Z), speed (Vx, Vy, Vz), direction (θ), and initialize the state covariance matrix; Time update (prediction): Based on the state and motion model of the previous moment (such as the acceleration model of a person walking or the radar speed measurement model), the estimated state of the target at the current time is predicted; Measurement update (correction): Compare the current measurement data of each sensor with the predicted state, calculate the measurement error, and then correct the target state through the Kalman gain to obtain the optimal estimate; Timing alignment: After the state estimation is completed, interpolation is used to adjust the timestamps of different sensor data to correspond to the same time point, thereby ensuring data consistency.

[0027] The first monitoring data after Kalman filtering can provide more stable, accurate and time-consistent personnel status information, ensuring that in the subsequent step S2, the deep learning target detection model can identify personnel based on the aligned data, improve the recognition accuracy, and avoid recognition errors caused by the asynchrony of sensor data time. In addition, the time series alignment process can also optimize the time series analysis model's judgment on the behavior pattern of illegal personnel, making the detection of illegal behavior patterns more accurate and reliable.

[0028] Furthermore, the expulsion mode includes a low-level warning mode, a medium-level expulsion mode, and a high-level emergency expulsion mode; wherein, The low-level warning mode includes emitting low-frequency sound waves to warn and drive away violators; The medium-level expulsion mode includes emitting medium-frequency sound waves to the violators and driving them away in combination with warning lights; The high-level emergency evacuation mode includes emitting high-frequency sound waves at violators and combining them with flashes to interfere with them.

[0029] The present invention adopts a graded expulsion mode to select appropriate expulsion means according to the severity of the behavior of the violator and the potential threat level, so as to reduce the impact of misjudgment, improve expulsion efficiency, and reduce safety risks. The low-level warning mode uses low-frequency sound waves for gentle prompts, which is suitable for mistaken or tentative violations to reduce unnecessary disturbances and accidental injuries. The medium-level expulsion mode combines medium-frequency sound waves and warning lights to effectively attract the attention of violators. It is suitable for deliberate intrusion violations, making them aware of the consequences of the violations and actively evacuating. The high-level emergency expulsion mode uses high-frequency sound waves and flash interference to cause stronger physiological discomfort to long-term latent or forced invasion violators, and quickly force them to evacuate in response to high-risk security incidents. Through a hierarchical expulsion strategy, the present invention can reduce interference with normal personnel while improving the effectiveness of violators and achieving more accurate regional security management.

[0030] Furthermore, the types of violation behaviors include mistaken entry violation, tentative violation, intentional intrusion violation, latent violation and forced intrusion violation; among them, When a violation is determined to be a tentative violation, a low-level warning mode is used; When a violation is determined to be an intentional trespassing violation, a medium-level removal mode is adopted; When a potential violation is identified, a high-level emergency evacuation mode will be adopted; When a forced invasion violation is determined, a high-level emergency eviction mode is adopted and the linkage security mechanism is triggered for handling by security personnel.

[0031] Furthermore, the sound waves are emitted by a directional sound wave transmitter, and the process of emitting sound waves to the violators is adaptive sound wave emission, which specifically includes: The location of the target violator is obtained, the acoustic wave transmission power of the acoustic wave transmitter is adjusted based on the location of the target violator, and the acoustic wave intervention strategy is adaptively adjusted according to the environmental noise interference level.

[0032] Preferably, when the expulsion mode is a high-level emergency expulsion mode, the sound wave transmission power of the sound wave transmitter is adjusted based on the location of the target violator and the hearing sensitivity range to make the violator reach maximum discomfort.

[0033] The present invention uses an adaptive sound wave emission strategy in the sound wave expulsion process to achieve precise intervention on violators while reducing interference with normal workers around them. The emission power and angle of the sound wave transmitter are adjusted based on the real-time location data of the violators to ensure that the sound wave intervention effectively covers the violators' area.

[0034] Specifically, in the normal expulsion mode, the optimal sound wave propagation path is calculated based on the location information of the target violator, and the transmission power is adjusted according to the distance attenuation characteristics of the sound wave, ensuring that the violator perceives the sound wave expulsion signal while avoiding excessive power output affecting the surrounding environment. In the high-level emergency expulsion mode, the frequency, power and directivity of the sound wave are dynamically adjusted in combination with the auditory sensitivity range of the target violator (based on the general human hearing threshold and the physiological influence range of the sound wave frequency), ensuring that the violator evacuates as soon as possible without reaching the maximum value, while avoiding exceeding the safe sound pressure range to prevent hearing damage.

[0035] In addition, the embodiment of the present invention also dynamically adjusts the sound wave intervention strategy through real-time monitoring of the environmental noise interference level. The sound wave intervention strategy includes: Dynamic frequency adjustment: When the ambient noise is high, the sound wave frequency is increased or switched to pulse sound wave mode to enhance the auditory stimulation of violators and improve the expulsion effect.

[0036] Directional control: Use phased array ultrasonic technology to adjust the transmission angle so that the sound waves can focus on the area where the violators are located, reducing the impact on other areas.

[0037] Power adaptive control: Dynamically adjust the sound wave transmission power based on the environmental noise measurement data to ensure that the effective expulsion effect can be maintained under different noise backgrounds, and the sound wave will not fail due to excessive environmental noise.

[0038] Through comprehensive calculation of factors such as the location of the violator, hearing sensitivity range, ambient noise level, etc., the power, frequency and directionality of the sound waves are dynamically adjusted to achieve precise, intelligent and adaptive sound wave expulsion, thereby improving the effectiveness and safety of the expulsion.

[0039] Preferably, when executing the directionally controlled acoustic wave intervention strategy, the violation area is determined based on the positions of the violating personnel and the normal operating personnel, and after the violation area is determined, the phase of the transmitting array of the acoustic wave transmitter is controlled by using the phased array ultrasonic technology to make the acoustic wave act only on the violation area; When the violator moves, the multiple angle-adjustable sound wave units of the sound wave transmitter automatically adjust the sound wave emission angle according to the violator's moving path.

[0040] Specifically, in the sonic expulsion process of the present invention, phased array ultrasonic technology is used to control the emission array phase of the sonic transmitter to achieve high-precision, dynamically adjustable directional sonic intervention.

[0041] First, a mechanism for determining the violation area is set up. This mechanism determines the dynamically adjustable sound wave intervention area based on factors such as the real-time location of the violator, the violation behavior, the location and activity range of the normal operating personnel, etc. Then, the phase of the transmitting array of the sound wave transmitter is controlled through phased array ultrasonic technology, and the directivity of the ultrasonic beam is adjusted to make the sound wave energy highly focused in the violation area. The core of phased array ultrasonic technology is to use phase control technology to make the ultrasonic waves emitted by each transmitting unit form beam synthesis interference in space, thereby enhancing the sound wave intensity in the designated area and reducing the sound wave intensity in the non-target area, ensuring that the violator receives the most inappropriate sound wave signal, while the normal operating personnel are minimally affected.

[0042] Secondly, during the movement of the violators, the moving path of the violators is determined, and the sound wave emission angle is dynamically adjusted in combination with the multiple adjustable angle sound wave units of the sound wave transmitter. Specifically, by real-time monitoring of the violators' moving speed, direction changes, detour strategies and other parameters, and updating according to the violators' positions, the angles and emission intensities of different emission units are controlled to ensure that the violators are always within the sound wave intervention range no matter how they move, so as to achieve continuous expulsion.

[0043] As a further preferred method, reinforcement learning is combined to optimize the expulsion parameters, that is, a deep reinforcement learning algorithm (such as DQN or PPO) is used to train the optimization model so that the emission angle, power, frequency and intervention time of the sound waves can be adaptively adjusted in actual scenarios. The model parameters are continuously optimized through historical expulsion data to minimize interference with normal personnel, maximize the discomfort of violators, reduce the length of stay of violators, and further improve the expulsion effect.

[0044] Through phased array ultrasonic technology and a violation area determination mechanism, combined with the violator's movement path and an adjustable angle sound wave unit adjustment mechanism, the embodiment of the present invention achieves highly accurate, dynamically adaptive directional sound wave intervention, effectively improving the effect of removing violators while minimizing the impact on normal operating personnel.

[0045] Furthermore, step S2 specifically includes the following sub-steps: S21. Perform target detection on the first monitoring data through a pre-trained deep learning target detection model, identify all personnel entering the perimeter of the target area, and extract image features of the personnel; S22. Perform identity matching based on the image characteristics of the personnel through the personnel database. If there is no match in the personnel database, the person is determined to be a violator, and after the person is determined to be a violator, step S23 is executed; S23. Collect behavioral parameters of the violators based on the first monitoring data, input the behavioral parameters into a time series analysis model, and determine the behavioral patterns of the violators through the time series analysis model.

[0046] Further, step S23 includes the following sub-steps: S231. Collecting behavioral parameters of violators based on the first monitoring data; S232. Based on the Transformer model of the self-attention mechanism, the behavioral parameters of the violators are modeled in time series to extract the behavioral characteristics of the violators; S233. Setting the criteria for determining the violation behavior patterns, the violation behavior patterns include abnormal stay pattern, rapid approach to sensitive area pattern, long-term wandering pattern, detour entry pattern, and high-intensity abnormal pattern; S234. The behavioral characteristics of the violators are matched with the criteria for determining the violation behavior pattern through the Transformer model, and the behavior pattern of the violators is determined when the behavioral characteristics of the violators are consistent with the violation behavior pattern.

[0047] Preferably, the pre-trained deep learning target detection model adopts the Faster R-CNN target detection network, takes the first monitoring data as input, and outputs a set of detected target persons:

[0048] Each target person Contains the following information:

[0049] Indicates the target person The bounding box (center coordinates ( ),width and height ); Represents the image characteristics of a person, including but not limited to facial features, clothing, body shape, posture, hairstyle, and height. ResNet is used to extract feature vectors:

[0050] in, The target person is cropped from the visible light camera of the image.

[0051] Through the personnel database Perform identity matching and calculate the detected person image features With the known personnel characteristics set in the database Similarity:

[0052] like , they will be judged as violators. Preset threshold for minimum similarity.

[0053] In the present invention, the Faster R-CNN target detection network is mainly used to detect and identify people entering the perimeter of the target area and extract the image features of the violators. The deep learning target detection model can quickly identify people and extract key image features to automatically and accurately detect people entering the area, which can effectively improve the recognition accuracy, reduce false alarms and missed alarms, and improve the system's adaptability to different lighting, angles and occlusion conditions.

[0054] Furthermore, the behavior parameters of the offenders are extracted based on the sensor data. The behavior parameter set is defined as follows:

[0055] in, is the moving speed of the offender at the current moment; The moving direction of the offender at the current moment; The length of stay for the offender; The location of the offender at the current moment; It is the distance between the violator and the normal operating personnel at the current moment.

[0056] The Transformer model based on the self-attention mechanism is used to perform time series modeling to predict the behavior patterns of violators. The input of the Transformer model is the historical behavior parameter sequence of the target person. :

[0057] Among them, the value of t ranges from 1 to T, and T is the observation time window.

[0058] Use the self-attention mechanism to extract the long-term behavior of violators:

[0059] in, is the query matrix; is the key matrix; is the value matrix; To calculate the similarity between the query vector and the key vector; is the dimension of the key vector, This is for scaling operations to prevent values ​​from being too large and affecting gradient stability.

[0060] The final output of Transformer is the behavioral characteristics of the offender , including but not limited to: 1. Spatial motion characteristics Movement speed change rate: whether there is abnormal acceleration or sudden stop; Movement direction mode: whether there is abnormal direction switching, such as a sudden change of direction after a rapid approach; Path trajectory pattern: whether it is similar to the historical violation path; Changes in habitual behavior: Are there extreme changes in position in a short period of time, such as sudden running or jumping?

[0061] 2. Stay and approach characteristics Stay time: whether to stay in sensitive areas for a long time; The way of approaching sensitive areas: whether to approach gradually (exploratory) or to break in directly; Circumvention behavior: Are there any signs of circumventing the monitoring area or approaching in a covert manner?

[0062] 3. Interaction Features Distance from normal operating personnel: whether to try to follow or blend into the normal crowd; Whether a coordinated mode is formed with other personnel: whether the person moves synchronously or cooperates with other violators.

[0063] Set the criteria for determining violation patterns:

[0064] Abnormal stay mode (such as staying in a non-operating area for a long time); A mode for quickly approaching sensitive areas (e.g. approaching important facilities in a short period of time); Long-term wandering mode (such as repeatedly patrolling the target area); Entering in a detour mode (e.g. trying to avoid the normal passage); It is a high-intensity abnormal mode (high-intensity abnormal behavior).

[0065] Match the behavioral characteristics of violators with the set of violation behavior patterns:

[0066] in, is the probability distribution of the violation pattern predicted by Transformer, and the value of k ranges from 1 to 5; if , then determine the behavior pattern of the offender, The preset minimum matching threshold.

[0067] It is understandable that the behavior of violators is a sequence data that changes over time, such as movement speed, direction, and dwell time. Traditional time series methods (such as LSTM and GRU) have the problem of long-term dependencies that are difficult to capture, while the Transformer model is based on the self-attention mechanism, which can focus on the global relationship in the entire time series data, thereby more accurately modeling the changing pattern of violations. For example, someone stays briefly in the target area → moves slowly → quickly approaches the sensitive area. Transformer can capture these behavioral changes and output the corresponding violation behavior features, and effectively analyze complex violation patterns such as long-term stay, wandering, and detours, avoiding false positives or omissions due to abnormal short-term behavior. In addition, the hierarchical structure of Transformer allows it to extract the behavior patterns of violators at different time scales. For example, it can simultaneously detect rapid movement behavior in a short period of time and long-term continuous stay behavior, thereby improving the accuracy of violation classification.

[0068] Furthermore, the violation determination strategy is based on a multi-task learning model and combines the behavior parameters in the behavior pattern to determine the type of violation behavior of the violator. The behavior parameters include the violator's moving speed, moving direction, stay time, stay location, and distance from normal workers.

[0069] Specifically, the core of the violation determination strategy of the present invention is to simultaneously learn the determination criteria of multiple violation behavior categories, so that the violation types can be accurately identified in different environments and scenarios.

[0070] By adopting the multi-task learning model (MTL), multiple parallel violation classification tasks are established to simultaneously learn the characteristics of different violation categories and make classification decisions. The model structure is as follows: Shared layer: Input the behavioral parameters of the offender and perform feature extraction through multiple shared layers.

[0071] Task specific layers: Task 1: Detect stray entry violations; Task 2: Detection of exploratory violations; Task 3: Detect intentional intrusion violations; Task 4: Detect latent violations; Task 5: Detect forced intrusion violations.

[0072] Specific rules for determining the types of violations: If the moving speed and moving path are normal, it is determined to be a stray entry violation; If the moving speed is low, the stay time is long, and it is close to sensitive areas, it will be judged as a latent violation; If the movement speed is high, the direction is clear, and it is close to the target area, it is determined to be an intentional intrusion violation; If the movement speed is irregular and close to the boundary area, it is considered a tentative violation; If it is accompanied by intense movement (such as sprinting, jumping over obstacles), it will be considered a forced intrusion violation.

[0073] Preferably, stray-entry violations correspond to abnormal stay modes; exploratory violations correspond to rapid approach to sensitive areas modes; deliberate intrusion violations correspond to long-term wandering modes; lurking violations correspond to detour entry modes; and forced intrusion violations correspond to high-intensity abnormal behaviors. Example

[0074] The second aspect of the present invention discloses a regional automatic sound wave system for driving away illegal personnel, the system comprises a data acquisition module, an identification module, a behavior analysis module, and a driving away module; wherein: The data acquisition module is used to deploy multimodal sensors in the target area to monitor the personnel entering the perimeter of the target area in real time and obtain real-time monitoring data; the multimodal sensors include infrared thermal imaging sensors, millimeter wave radar sensors, identity recognition devices and visible light cameras; The recognition module is used to identify the personnel entering the perimeter of the target area as normal operating personnel or illegal personnel based on the monitoring data through the deep learning target detection algorithm; The behavior analysis module is used to determine the behavior pattern of the violator in combination with the time series analysis model after the violator is identified, and determine the type of violation behavior of the violator through the violation determination strategy based on the result of the violation pattern determination; The expulsion module is used to select the corresponding expulsion mode to expel the violators according to the type of their violation behavior.

[0075] Furthermore, the system also includes a data fusion module, which is used to perform a sensor data fusion operation on the acquired real-time monitoring data after acquiring the real-time monitoring data to obtain the first monitoring data. The data fusion operation includes performing a time sequence alignment operation on the data based on Kalman filtering.

[0076] Furthermore, the expulsion mode includes a low-level warning mode, a medium-level expulsion mode, and a high-level emergency expulsion mode; wherein, The low-level warning mode includes emitting low-frequency sound waves to warn and drive away violators; The medium-level expulsion mode includes emitting medium-frequency sound waves to the violators and driving them away in combination with warning lights; The high-level emergency evacuation mode includes emitting high-frequency sound waves at violators and combining them with flashes to interfere with them.

[0077] The sound waves are emitted by a directional sound wave emitter.

[0078] Furthermore, the types of violation behaviors include mistaken entry violation, tentative violation, intentional intrusion violation, latent violation and forced intrusion violation; among them, When a violation is determined to be a tentative violation, a low-level warning mode is used; When a violation is determined to be an intentional trespassing violation, a medium-level removal mode is adopted; When a potential violation is identified, a high-level emergency evacuation mode will be adopted; When a forced invasion violation is determined, a high-level emergency eviction mode is adopted and the linkage security mechanism is triggered for handling by security personnel.

[0079] Furthermore, the violation determination strategy is based on a multi-task learning model and combines the behavior parameters in the behavior pattern to determine the violation type of the violator. The behavior parameters include the violator's moving speed, moving direction, stay time, stay location, and distance from normal workers.

[0080] Furthermore, the process of transmitting sound waves to the violators is adaptive sound wave transmission, which specifically includes: Obtain the location of the target violator, adjust the acoustic wave transmission power of the acoustic wave transmitter based on the location of the target violator, and adaptively adjust the acoustic wave intervention strategy according to the environmental noise interference level; When the expulsion mode is a high-level emergency expulsion mode, adjusting the sound wave transmission power of the sound wave transmitter based on the position of the target violator specifically includes: The sonic emission power of the sonic transmitter is adjusted based on the location of the target violator and the hearing sensitivity range to cause the violator to reach maximum discomfort.

[0081] Furthermore, the expulsion module includes a determination unit for determining the violation area based on the positions of the violators and normal operating personnel, and after determining the violation area, using phased array ultrasonic technology to adjust the transmission array phase of the sound wave transmitter so that the sound wave acts only on the violation area.

[0082] When the violator moves, the multiple angle-adjustable sound wave units of the sound wave transmitter automatically adjust the sound wave emission angle according to the violator's moving path.

[0083] Furthermore, the identification module specifically performs the following operations: S21. Perform target detection on the first monitoring data through a pre-trained deep learning target detection model, identify all personnel entering the perimeter of the target area, and extract image features of the personnel; S22. Perform identity matching based on the image features of the person through the personnel database, and determine the person as a violator if there is no matching result in the personnel database.

[0084] Furthermore, the behavior analysis module specifically performs the following operations: S231. Collecting behavioral parameters of violators based on the first monitoring data; S232. Based on the Transformer model of the self-attention mechanism, the behavioral parameters of the violators are modeled in time series to extract the behavioral characteristics of the violators; S233. Setting the criteria for determining the violation behavior patterns, the violation behavior patterns include abnormal stay pattern, rapid approach to sensitive area pattern, long-term wandering pattern, detour entry pattern, and high-intensity abnormal pattern; S234. The behavioral characteristics of the violators are matched with the criteria for determining the violation behavior pattern through the Transformer model, and the behavior pattern of the violators is determined when the behavioral characteristics of the violators are consistent with the violation behavior pattern.

[0085] It should be noted that the specific implementation process of the second embodiment is similar to that of the first embodiment and will not be repeated in the second embodiment.

[0086] Finally, it should be noted that the regional automatic sound wave method and system for driving away violators disclosed in the embodiments of the present invention only discloses the preferred embodiments of the present invention, which are only used to illustrate the technical solutions of the present invention, rather than to limit them. Although the present invention has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for regional automatic sonic wave expulsion of illegal personnel, characterized in that: The method comprises the following steps: S1. Deploy a multimodal sensor in the target area to monitor the personnel entering the perimeter of the target area in real time to obtain real-time monitoring data; the multimodal sensor includes an infrared thermal imaging sensor, a millimeter wave radar sensor, an identity recognition device, and a visible light camera; S2. Using the deep learning target detection model to identify the personnel entering the perimeter of the target area as normal workers or violators based on the monitoring data, and after identifying the violators, combining the time series analysis model to determine the behavior patterns of the violators; S3, based on the behavior pattern of the violator, determine the type of violation behavior of the violator through the violation determination strategy; S4. Select a corresponding expulsion mode to expel the violator according to the type of violation behavior of the violator.

2. The method of regional automatic sonic wave to drive away violators according to claim 1, characterized in that: After acquiring the real-time monitoring data and before executing step S02, the method further includes: A sensor data fusion operation is performed on the acquired real-time monitoring data to obtain first monitoring data; the data fusion operation includes a time alignment operation on the data based on Kalman filtering.

3. The method of regional automatic sonic wave to drive away illegal personnel according to claim 1, characterized in that: The expulsion mode includes a low-level warning mode, a medium-level expulsion mode, and a high-level emergency expulsion mode; wherein, The low-level warning mode includes emitting low-frequency sound waves to warn and drive away violators; The medium-level expulsion mode includes emitting medium-frequency sound waves to the violators and driving them away in combination with warning lights; The high-level emergency expulsion mode includes emitting high-frequency sound waves to the violators, combined with flashes to interfere with the violators; The sound waves are emitted by a directional sound wave emitter.

4. The method of regional automatic sonic wave to drive away violators according to claim 3 is characterized in that: The types of violations include accidental entry violations, tentative violations, intentional intrusion violations, latent violations, and forced intrusion violations; among them, When a violation is determined to be a tentative violation, a low-level warning mode is adopted; When a violation is determined to be an intentional trespassing violation, a medium-level removal mode is adopted; When a potential violation is identified, a high-level emergency evacuation mode will be adopted; When a forced invasion violation is determined, a high-level emergency eviction mode is adopted and the linkage security mechanism is triggered for handling by security personnel.

5. The method of regional automatic sonic wave expulsion of illegal personnel according to claim 4 is characterized in that: The violation determination strategy is based on a multi-task learning model and combines the behavior parameters in the behavior pattern to determine the violation behavior type of the violator; The behavior parameters include the violator's moving speed, moving direction, stay time, stay location, and distance from normal operating personnel.

6. The method for regional automatic sonic wave expulsion of violators according to any one of claims 3 to 5, characterized in that: The process of emitting sound waves to violators is adaptive sound wave emission, which specifically includes: Obtain the location of the target violator, adjust the acoustic wave transmission power of the acoustic wave transmitter based on the location of the target violator, and adaptively adjust the acoustic wave intervention strategy according to the environmental noise interference level; When the expulsion mode is a high-level emergency expulsion mode, adjusting the sound wave transmission power of the sound wave transmitter based on the position of the target violator specifically includes: The sonic emission power of the sonic transmitter is adjusted based on the location of the target violator and the hearing sensitivity range to cause the violator to reach maximum discomfort.

7. The method of regional automatic sonic wave expulsion of illegal personnel according to claim 6 is characterized in that: The method further includes determining the violation area based on the positions of the violating personnel and the normal operating personnel, and after determining the violation area, using phased array ultrasonic technology to adjust the transmission array phase of the sound wave transmitter so that the sound wave acts only on the violation area; When the violator moves, the multiple angle-adjustable sound wave units of the sound wave transmitter automatically adjust the sound wave emission angle according to the violator's moving path.

8. The method of regional automatic sonic wave to drive away offenders according to claim 1, characterized in that: The step S2 specifically includes the following sub-steps: S21. Perform target detection on the first monitoring data through a pre-trained deep learning target detection model, identify all personnel entering the perimeter of the target area, and extract image features of the personnel; S22. Perform identity matching based on the image characteristics of the personnel through the personnel database. If there is no match in the personnel database, the person is determined to be a violator, and step S23 is executed after the person is determined to be a violator; S23. Based on the first monitoring data, behavioral parameters of the violators are collected, the behavioral parameters are input into a time series analysis model, and the behavioral patterns of the violators are determined through the time series analysis model.

9. The method of regional automatic sonic wave expulsion of illegal personnel according to claim 8, characterized in that: The step S23 includes the following sub-steps: S231. Collecting behavioral parameters of violators based on the first monitoring data; S232. Based on the Transformer model of the self-attention mechanism, the behavioral parameters of the violators are modeled in time series to extract the behavioral characteristics of the violators; S233. Setting the criteria for determining the violation behavior patterns, the violation behavior patterns include abnormal stay pattern, rapid approach to sensitive area pattern, long-term wandering pattern, detour entry pattern, and high-intensity abnormal pattern; S234. The behavioral characteristics of the violators are matched with the criteria for determining the violation behavior pattern through the Transformer model, and the behavior pattern of the violators is determined when the behavioral characteristics of the violators are consistent with the violation behavior pattern.

10. A regional automatic sound wave system for driving away offenders, the system is implemented based on the method described in any one of claims 1 to 9, characterized in that: The system includes a data acquisition module, an identification module, a behavior analysis module, and an expulsion module; wherein, The data acquisition module is used to deploy multimodal sensors in the target area to monitor the personnel entering the perimeter of the target area in real time and obtain real-time monitoring data; the multimodal sensors include infrared thermal imaging sensors, millimeter wave radar sensors, identity recognition devices and visible light cameras; The recognition module is used to identify the personnel entering the perimeter of the target area as normal operating personnel or illegal personnel based on the monitoring data through the deep learning target detection algorithm; The behavior analysis module is used to determine the behavior pattern of the violator in combination with the time series analysis model after the violator is identified, and determine the type of violation behavior of the violator through the violation determination strategy based on the result of the violation pattern determination; The expulsion module is used to select the corresponding expulsion mode to expel the violators according to the type of their violation behavior.

Images