An intelligent rat prevention and control monitoring method and system based on visual identification and directional sound waves

Abstract

The application discloses an intelligent rat damage prevention monitoring method and system based on visual identification and directional sound waves, comprising the following steps: collecting video images of a monitoring area through a visible light monitoring camera, and performing real-time target detection on the video images; performing category identification on the targets in the video images to obtain target categories, target confidence levels, target bounding boxes and skeleton key points; when the target confidence level of an identified rat target reaches a preset detection threshold and the rat target is detected in N continuous video images, the rat target is confirmed and enters a tracking state; morphological features of the rat target are constructed according to the bounding box geometric parameters, skeleton key points, visual appearance features and motion features of the rat target, target correlation of the rat target in continuous video images is performed, and the motion state of the rat target is estimated and predicted based on Kalman filtering.

Description

Technical Field

[0001] This invention relates to the field of monitoring equipment technology, and in particular to an intelligent rodent control monitoring method and system based on visual recognition and directional sound waves. Background Technology

[0002] With the widespread adoption of intelligent monitoring equipment in homes, warehouses, restaurants, and other environments, the automatic detection and control of rodents using monitoring devices is gradually becoming the development direction of rodent control technology. Traditional rodent control methods mainly include physical methods such as mousetraps, glue boards, and mouse cages; chemical methods such as rodenticide application; and ultrasonic rodent repellents. Physical methods usually rely on rodents actively entering or triggering the traps, and require manual cleanup afterwards. When using rodenticides, there is a risk of accidental ingestion by children or pets, and rodents may die in hidden locations that are difficult to clean up. Conventional ultrasonic rodent repellents typically use a wide-range or omnidirectional sound wave output, lacking the ability to accurately locate and track specific rodent targets, and their sustained effectiveness is easily affected when using fixed or similar sound wave parameters over a long period.

[0003] Currently, technical solutions combining image recognition, monitoring equipment, and rodent control have emerged. For example, Chinese utility model patent CN219698853U discloses a "data center computer room rodent control system based on recognition training," which detects rodent targets through image acquisition and recognition, and processes the targets by combining trajectory calculation, pan-tilt-zoom (PTZ) devices, and corresponding blinding and stunning devices. This solution can improve the automation level of rodent target detection and processing in data center environments. However, it is mainly geared towards data center applications and uses thermal imaging and corresponding directional devices. In environments such as homes and restaurants where people and pets are frequently active and there are many furniture and appliances, there is still a need to further improve its versatility, target tracking accuracy, and safety control capabilities during the process.

[0004] For example, Chinese invention patent CN108259830B discloses a "cloud server-based intelligent rodent monitoring system and method," which monitors, analyzes, and manages rodent infestations through rodent information collection equipment and a cloud server, thereby improving the informatization and remote management capabilities of rodent monitoring. However, this type of technical solution mainly focuses on the monitoring, statistics, and management of rodent information. After detecting rodent targets, it lacks a complete closed-loop control process for continuous locking, future location prediction, and directional active intervention based on the real-time movement status of the rodent targets.

[0005] For example, Chinese invention patent application CN113229270A discloses "a novel method for repelling rodents," which uses corresponding methods to drive rodents away. Although existing acoustic rodent repelling technologies can reduce the use of traditional chemical agents or mechanical rodent traps, their main purpose is usually to make rodents leave a specific area. There is a lack of precise correspondence between the direction of sound wave action, the target object, and the parameters and the real-time state of the target, making it difficult to continuously lock onto, aim ahead of, and control the directional sound beam for a single moving rodent target.

[0006] Therefore, existing technologies still have at least the following shortcomings: On the one hand, there is still room for improvement in the coordination between target detection, target tracking, and subsequent action devices in existing intelligent rodent monitoring systems. When rodents move quickly, are briefly obscured, or multiple rodent targets appear simultaneously, it can easily affect the continuous locking and accurate tracking of the target. On the other hand, existing technologies usually process the rodent target based on its current location, without fully considering the impact of image processing delay, actuator response delay, and propagation delay of the action medium on the aiming position of the moving target. In addition, existing acoustic rodent repelling devices usually lack the ability to dynamically adjust acoustic parameters based on the target distance, movement speed, and target status, and also lack a launch window control mechanism that comprehensively judges the status of personnel, pets, target obstruction, and launch module status before and during acoustic emission.

[0007] Therefore, it is necessary to provide an intelligent rodent control and monitoring system and method based on visual recognition and directional sound waves to improve the ability to detect, lock, track and predict the future location of rodent targets, and to coordinate the directional sound wave emission with the target status, the safety status of the sound beam path and the working status of the emission module, so as to achieve more complete closed-loop monitoring and directional control. Summary of the Invention

[0008] To address the problems existing in the prior art, this invention provides an intelligent rodent control and monitoring method and system based on visual recognition and directional sound waves.

[0009] To achieve the above objectives, the present invention provides an intelligent rodent control and monitoring method based on visual recognition and directional sound waves, comprising the following steps: S1. Acquire video images of the monitored area using a visible light surveillance camera, and perform real-time target detection on the video images; S2. Perform category recognition on the target in the video image to obtain the target category, target confidence, target bounding box and skeleton key points. When the target confidence of the identified rodent target reaches the preset detection threshold and the rodent target is detected in N consecutive video images, confirm the rodent target and enter the tracking state. S3. Construct the morphological features of the rodent target based on the bounding box geometric parameters, skeleton key points, visual appearance features, and motion features of the rodent target; perform target association on the rodent targets in continuous video images; estimate and predict the motion state of the rodent targets based on Kalman filtering; obtain the motion trajectory and predicted position of the rodent targets at future times; and obtain the target distance of the rodent targets; the target distance is obtained by a time-of-flight sensor or determined based on calibrated camera parameters and target spatial geometric relationships. S4. Determine the launch window based on the target confidence level, target distance, target motion state, beam path safety status of the directional sound wave, angle error between the sound wave emission direction and the target direction, target obstruction status, and working status of the directional sound wave emission module. Generate a launch permit when all launch conditions are met. S5. Based on the predicted position of the rodent target and the time delay from video image processing to the arrival of the sound wave at the target position, control the gimbal to drive the directional sound wave emission module to rotate, so that the emission direction of the directional sound wave emission module is pointed to the predicted position of the rodent target at a future time. S6. Determine the sound wave emission parameters based on the target distance and movement speed of the rodent target, form a directional sound beam through a parametric array transducer array, and emit the sound wave in a directional manner towards the predicted position of the rodent target according to the sound wave emission parameters. S7. After the sound wave is emitted, continue to acquire video images of the rodent target, and evaluate the sound wave effect based on the motion and posture of the rodent target after the emission; when the sound wave effect reaches the preset effect condition, stop the emission; when the sound wave effect does not reach the preset effect condition and the emission window is satisfied again, perform directional sound wave emission again within the preset maximum number of emission times. S8. During the execution of steps S4 to S7, personnel and pets are detected in the sound beam path. When personnel or pets are detected entering the sound beam path, sound wave emission is prohibited or interrupted.

[0010] Preferably, in step S2: When the target confidence P of the detected rodent target is greater than or equal to the preset detection threshold P0 and the rodent target is detected in N consecutive video images, the rodent target is confirmed; False alarms are filtered based on various features of the rodent target, including its size ratio, aspect ratio, movement speed, movement intermittency, and consistency of continuous frame detection. Auxiliary verification information for candidate rodent targets is obtained using at least one of a pyroelectric infrared sensor and a time-of-flight sensor. When the pyroelectric infrared sensor is used, infrared response information corresponding to the candidate rodent target is obtained to help determine whether it has infrared response characteristics corresponding to a live target. When the time-of-flight sensor is used, distance information corresponding to the candidate rodent target is obtained. When the pyroelectric infrared sensor is used and the obtained infrared response information does not characterize the infrared response characteristics corresponding to a live target, or when the time-of-flight sensor is used and the obtained distance information is abnormal, the directional acoustic emission process for the candidate rodent target is not triggered.

[0011] Preferably, in step S3: The morphological characteristics of the rodent target are represented as follows: F = [B, K, R, M]; Wherein, B represents the bounding box geometric parameters of the rodent target, K represents the skeleton key points of the rodent target, R represents the re-identification visual appearance features of the rodent target, and M represents the motion features of the rodent target in continuous video images; A comprehensive matching score is calculated based on the detected rodent targets in the current video image and the existing tracked targets: Score=α·IoU+β·Sim(ReID)+γ·Sim(Pose)+δ·Sim(Motion); Where IoU represents the intersection-union ratio of bounding boxes, Sim(ReID) represents the similarity of visual appearance features in re-identification, Sim(Pose) represents the similarity of skeleton key points, and Sim(Motion) represents the similarity of motion features. α, β, γ and δ are the corresponding weight coefficients, and α+β+γ+δ=1. When the overall matching score reaches the preset matching threshold, the mouse target detected in the current video image is associated with the corresponding existing tracked target as the same target.

[0012] Preferably, in step S3: The motion state of the mouse target, including its position, velocity, and acceleration, is estimated and predicted based on Kalman filtering. When the mouse target is lost due to occlusion, a search area is established near the last location of the mouse target. Candidate targets in subsequent video images are re-matched based on the re-identified visual appearance features. When a re-match is successful within a preset number of frames, the original tracking identifier of the mouse target is restored.

[0013] Preferably, in step S4, the emission window simultaneously satisfies the following conditions: The target confidence P of the rodent target is greater than or equal to the emission confidence threshold P1, wherein the emission confidence threshold P1 is higher than the preset detection threshold P0 in step S2; The target distance d of the rodent target is within the preset effective range; The moving speed v of the rodent target is less than or equal to the preset maximum speed vmax, or the prediction hit error e of the rodent target is less than or equal to the preset maximum hit error emax; The sound beam path does not overlap with the areas where the detected people and pets are located; The angular error between the sound wave emission direction and the target direction is less than or equal to the preset maximum angular error θmax; The rodent target was not obstructed by any obstacle, and the sound beam propagation path was unobstructed; Furthermore, the directional acoustic wave transmitting module is in a working state where transmission is permitted.

[0014] Preferably, in step S5: The predicted time Δt of the future time relative to the current time is determined according to the following formula: Δt=t_proc+t_gimbal+t_sound; Where t_proc represents video image processing latency, t_gimbal represents gimbal response latency, and t_sound represents sound wave propagation latency; The sound wave propagation delay satisfies: t_sound=d / c; Where d represents the distance from the directional sound wave emitting module to the rodent target, and c represents the speed of sound; The predicted position of the rodent target at time t+Δt is obtained based on the predicted time Δt, and forward aiming is performed based on the predicted position; The servo control parameters of the gimbal are dynamically adjusted according to the movement speed of the rodent target.

[0015] Preferably, in step S6: At least two ultrasonic carrier signals are generated and the parametric array transducer array is driven. The phase delay of each transducer unit in the parametric array is determined based on the direction and distance of the rodent target, and the sound beam is electronically scanned and focused by controlling the phase delay of each transducer unit. The at least two ultrasonic carrier signals generate difference frequency sound waves during the nonlinear propagation of the air.

[0016] Preferably, in step S6: Based on the distance range to which the target distance of the rodent target belongs, the sound pressure level, pulse width, frequency sweep range, and transmission duration at the target location are jointly determined. As the target distance of the rodent target increases, the sound pressure level at the target location is increased and / or the emission duration is increased, and the angular accuracy requirement of the sound wave emission direction is improved; When the rodent target is in a high-speed movement state, wait for the rodent target to enter a low-speed movement state before emitting sound waves, or emit sound waves in advance based on the predicted position of the rodent target in the future. When the historical sound wave effect does not meet the preset effect conditions and step S6 is executed again, at least one of the sound wave frequency combination, pulse interval, and sweep frequency mode is adjusted.

[0017] Preferably, in steps S7 and S8: The sound wave effect is determined based on the speed change and static duration of the rodent target. When the speed change of the rodent target reaches a preset speed change threshold and the static duration reaches a preset static time threshold, the sound wave effect is determined to meet the preset effect condition. If the effect of the sound wave does not meet the preset effect condition, the directional sound wave will be emitted again when the emission window is met again, and the total number of sound wave emissions will not exceed the preset maximum number. When a person or pet enters the sound beam path, the power amplifier used to drive the parametric array transducer array is turned off, and the pan-tilt unit used to adjust the emission direction of the directional sound wave emission module is locked.

[0018] The present invention also provides an intelligent rodent control and monitoring system based on visual recognition and directional sound waves, including a visible light monitoring camera, a target detection and recognition module, a shape locking and motion tracking module, a transmission window determination module, a pan-tilt servo control module, a directional sound wave transmission module, and a control processing module; The visible light surveillance camera is used to capture visible light video images of the monitored area; The target detection and recognition module is used to perform target detection and category recognition on the visible light video image. When a rodent target is detected, it outputs the target confidence score, target bounding box, and skeleton key points of the rodent target. It is also used to identify human and pet targets in the visible light video image and to obtain the target distance of the rodent target. The target distance is obtained by a time-of-flight sensor or determined based on calibrated camera parameters and target spatial geometry. The morphology locking and motion tracking module is used to perform target association and motion tracking of the rodent target based on the target bounding box, skeletal key points, visual appearance features and motion features of the rodent target, and to obtain the motion trajectory and predicted position of the rodent target at future time through Kalman filtering; The launch window determination module is used to determine the launch window based on the target confidence level, target distance, target motion state, sound beam path safety state, angular error between the sound wave emission direction and the target direction, target obstruction state, and the working state of the directional sound wave emission module, and to generate a launch permit when all launch conditions are met. The gimbal servo control module is used to drive the directional acoustic wave emission module to change the direction of acoustic wave emission, and to make the directional acoustic wave emission module point to the predicted position of the rodent target in the future time according to the predicted position of the rodent target in the future time, as well as the video image processing delay, gimbal response delay and acoustic wave propagation delay. The directional acoustic wave emission module includes a waveform generator, a power amplifier, a parametric array transducer array, and a phase control circuit. The waveform generator is used to generate at least two ultrasonic carrier signals. The power amplifier is used to drive the parametric array transducer array. The phase control circuit is used to control the phase delay of each transducer unit in the parametric array transducer array according to the direction and distance of the rodent target, so as to electronically scan and focus the sound beam. The control processing module is connected to the target detection and recognition module, the shape locking and motion tracking module, the launch window determination module, the gimbal servo control module, and the directional sound wave emission module, respectively, and is used to coordinate target detection, target tracking, launch window determination, gimbal aiming, sound wave parameter determination, directional sound wave emission, sound wave effect result evaluation, and safety interruption.

[0019] The technical solution of this invention has the following beneficial effects: This invention uses a visible light surveillance camera to acquire video images of the monitored area and confirms rodent targets through target confidence and multiple consecutive frames. It can utilize visible light surveillance equipment commonly found in homes, warehouses, and restaurants to detect rodent targets, which helps reduce the system's dependence on dedicated thermal imaging equipment and improves the system's applicability in different application environments.

[0020] This invention constructs morphological features based on the bounding box geometric parameters, skeleton key points, re-identification visual appearance features, and motion features of rodent targets, and performs cross-frame target association based on the comprehensive matching results between multiple features, which can improve the ability of rodent targets to maintain their identity in continuous video images; combined with Kalman filtering to estimate and predict the target position, velocity, and acceleration, and to re-match after the target is briefly occluded, it is beneficial to improve the stability of continuous tracking of moving rodent targets.

[0021] This invention does not immediately emit sound waves after detecting a rodent target. Instead, it determines the emission window by comprehensively considering the target confidence level, target distance, target motion state, sound beam path safety status, angular error between the sound wave emission direction and the target direction, target obstruction status, and the working status of the directional sound wave emission module. A emission permit is generated only when the corresponding conditions are met, which helps to reduce the possibility of false and invalid emission.

[0022] This invention incorporates video image processing latency, gimbal response latency, and sound wave propagation latency into the prediction time of the target's future position. It controls the gimbal to perform forward aiming based on the predicted future position of the rodent target, rather than aiming only at the target's current detection position. This reduces the target position deviation caused by system processing latency and sound wave propagation latency, thereby improving the aiming accuracy of moving targets.

[0023] This invention uses a parametric array transducer array to generate a directional sound beam, and controls the phase delay of each transducer unit according to the direction and distance of the rodent target to achieve electronic scanning and focusing of the sound beam. This allows the sound wave energy to act more concentratedly on the target area. Compared with large-area or omnidirectional sound wave emission methods, this is beneficial to improve the utilization efficiency of sound wave energy and reduce the impact on non-target areas.

[0024] This invention can determine the corresponding sound wave emission parameters based on the distance and movement speed of the rodent target, and can adjust the sound pressure level, pulse width, frequency sweep range and emission duration according to different distance ranges. When the historical sound wave effect results do not meet the preset conditions, the frequency combination, pulse interval or frequency sweep mode can also be changed, thereby avoiding the long-term use of a single fixed sound wave parameter and enhancing the system's adaptability to different target states and different action distances.

[0025] After completing one directional sound wave emission, this invention continues to acquire video images of the rodent target. Based on information such as the rodent target's speed change, posture state, and duration of stillness, it judges the result of the sound wave action. If the result of the action does not meet the preset conditions and the emission window is met again, it performs a limited number of re-emissions. This forms a closed-loop control process of "visual detection - directional emission - visual feedback - re-judgment", reducing the continuous output of sound waves without confirming the result of the action.

[0026] This invention continuously detects whether personnel or pets enter the sound beam path before and during launch. When personnel or pets are detected entering the sound beam path, sound wave emission is prohibited or interrupted, and output can be stopped by turning off the power amplifier and locking the pan-tilt unit. At the same time, it combines the target obstruction status, transducer temperature, power supply status, and sound pressure feedback status for control, thereby improving the system's operational safety and control reliability in environments where personnel, pets, and other objects coexist. Attached Figure Description

[0027] Figure 1 This is an overall flowchart of the intelligent rodent control and monitoring method of the present invention; Figure 2 This is a flowchart of the target detection, shape locking, and motion tracking process of the present invention; Figure 3 This is a flowchart of the launch window determination process of the present invention; Figure 4 This is a flowchart of the coordinated control process for gimbal aiming and directional acoustic wave emission in this invention. Figure 5 This is a schematic diagram of the directional acoustic wave transmitting module structure of the present invention; Figure 6 This is a schematic diagram of the parametric array transducer array arrangement of the present invention; Figure 7 This is a flowchart of the adaptive control of acoustic emission parameters according to the present invention; Figure 8 This is a flowchart of the acoustic wave effect evaluation and safety interruption control of the present invention; Figure 9 This is a block diagram of the overall structure of the intelligent rodent control and monitoring system of the present invention. Detailed Implementation

[0028] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0029] This invention provides an intelligent rodent control and monitoring method based on visual recognition and directional acoustic waves. It acquires video images of the monitored area using a visible light monitoring camera, detects and identifies rodent targets in the video images, and performs morphological locking and continuous tracking based on the target's bounding box, skeletal key points, visual appearance features, and motion characteristics. The method predicts the future position of the rodent based on its motion state and determines the transmission window by comprehensively considering target confidence, target distance, target motion state, acoustic beam path safety status, transmission direction angle error, target occlusion status, and the operational status of the directional acoustic wave transmission module. When the transmission window is met, the method controls the pan-tilt unit to drive the directional acoustic wave transmission module for forward aiming based on the predicted future position of the rodent. The system accurately emits sound waves at the predicted location of the rodent target via a parametric array transducer array. After the sound waves are emitted, the system continues to acquire the motion and posture of the rodent target through visible light video images, evaluates the effect of the sound waves, and determines whether to stop the emission or restart the directional sound wave emission when the emission window is met again. Simultaneously, the system continuously monitors personnel and pets during the emission window determination, gimbal aiming, sound wave emission, and effect evaluation. When personnel or pets enter the sound beam path, the sound wave emission is prohibited or interrupted, thus forming a closed-loop control process of target detection, target confirmation, shape locking, motion tracking, position prediction, emission window determination, advanced aiming, directional sound wave emission, effect evaluation, and safety interruption.

[0030] Specifically, this implementation method includes the following steps.

[0031] S1. Acquire video images of the monitored area using a visible light surveillance camera, and perform real-time target detection on the video images.

[0032] The monitoring area can be a living room, kitchen, corridor, storage room, warehouse, dining room, or other areas requiring rodent monitoring. The visible light monitoring camera can be an existing visible light security monitoring camera to obtain continuous video images.

[0033] In one specific implementation, the image acquisition frame rate of the visible light surveillance camera can be no less than 15 FPS, and the image resolution can be no less than 1080p. The above parameters are only one specific implementation method. Other image acquisition frame rates and resolutions can also be used if the requirements of target recognition, continuous tracking, and future position prediction can be met.

[0034] The video images captured by the camera can first undergo preprocessing such as scaling and normalization before being input into the target detection and recognition module for real-time target detection. The target detection and recognition module can be set in the AI ​​processing unit inside the camera, or it can be set in an embedded AI chip or edge computing device that communicates with the camera.

[0035] The object detection and recognition module can employ a deep learning object detection model trained on rodent target samples. In one specific implementation, YOLOv8 or other object detection models with the same object detection capabilities can be used. During model training, images of rodent targets in different indoor environments can be acquired, and information such as target bounding boxes, skeleton key points, and pose of the rodent targets in the images can be labeled. The labeled samples are then used to train the object detection model, and the trained model is then deployed to an AI inference device.

[0036] S2. Perform category recognition on the targets in the video image to obtain the target category, target confidence, target bounding box and skeleton key points, and perform continuous frame confirmation and false alarm filtering on the detected candidate rodent targets.

[0037] The target detection and recognition module performs target detection on each frame of video image and outputs the target category and target confidence score P corresponding to the detected target. When the target category is identified as a rodent target, the target confidence score P is compared with the preset detection threshold P0.

[0038] In one specific implementation, the preset detection threshold P0 can be set in the range of 0.70 to 0.85.

[0039] To avoid accidental single-frame false detections that could lead to incorrect triggering of subsequent control processes, a continuous frame confirmation method is adopted. When the target confidence P of a detected rodent target is greater than or equal to a preset detection threshold P0, and the corresponding rodent target is detected in N consecutive video frames, the rodent target is confirmed and the tracking state is entered.

[0040] N can be set to 3 to 5 frames. For example, in one specific implementation, P0 can be set to 0.80 and N can be set to 3 frames.

[0041] After the target is confirmed, the target bounding box B of the mouse target is output. The target bounding box B can be represented as: B = (x, y, w, h); Where x and y represent the position coordinates of the target bounding box, w represents the width of the target bounding box, and h represents the height of the target bounding box.

[0042] Simultaneously, the skeletal key points K of the rodent target are obtained. The skeletal key points may include multiple key locations such as the nose tip, both ears, the center of the trunk, and the tail root. The morphology and posture of the rodent target can be further described through the skeletal key points.

[0043] To reduce false positives caused by cats, dogs, toy mice, and other similar targets, a comprehensive false positive filtering can be performed based on various features, including the size ratio of candidate mouse targets, the aspect ratio of the target bounding box, movement speed, movement intermittent features, low-light enhancement results, and consistency of continuous frame detection.

[0044] Among them, the intermittent motion feature can be obtained based on the position and velocity changes of the rodent target in consecutive video frames. For example, the motion pattern of the rodent target can be determined based on its movement and pause states in consecutive frames, and the candidate rodent targets can be further confirmed by combining the size ratio, aspect ratio, and consecutive frame detection results.

[0045] For candidate targets that appear only in a single frame or a small number of video frames, or whose size, aspect ratio, and motion state are significantly different from the preset features corresponding to rodent targets, the subsequent directional sound wave emission process may not be triggered.

[0046] In an alternative implementation, candidate rodent targets can also be further verified using at least one of a pyroelectric infrared sensor and a time-of-flight sensor.

[0047] When a pyroelectric infrared sensor is used, infrared response information corresponding to the area where the candidate rodent target is located is obtained, and the infrared response information is used to help determine whether the candidate rodent target has the infrared response characteristics corresponding to a live target; when a time-of-flight sensor is used, distance information corresponding to the candidate rodent target is obtained.

[0048] When the pyroelectric infrared sensor is used and the acquired infrared response information does not characterize the infrared response features corresponding to the live target, or when the time-of-flight sensor is used and the acquired distance information is abnormal, the directional acoustic emission process for the candidate rodent target will not be triggered.

[0049] S3. Based on the bounding box geometric parameters, skeleton key points, visual appearance features, and motion features of the mouse targets, construct the morphological features of the mouse targets. Perform target association on the mouse targets in continuous video images, and estimate and predict the motion state of the mouse targets based on Kalman filtering. At the same time, obtain the target distance of the mouse targets.

[0050] In one specific implementation, the morphological characteristics of the rodent target are represented as follows: F = [B, K, R, M]; Where B represents the bounding box geometric parameters of the mouse target, K represents the skeleton key points of the mouse target, R represents the re-identification visual appearance features of the mouse target, and M represents the motion features of the mouse target in continuous video images.

[0051] The re-identified visual appearance feature R can be obtained through a deep feature extraction network, and the motion feature M may include the position sequence, velocity sequence and acceleration sequence of the mouse target in the most recent video frames.

[0052] For the mouse-like targets detected in the current video frame and the existing tracked targets, calculate the comprehensive matching score: Score=α·IoU+β·Sim(ReID)+γ·Sim(Pose)+δ·Sim(Motion).

[0053] Wherein, IoU represents the intersection-union ratio between the bounding boxes of the current mouse target and the corresponding target bounding boxes of the existing tracked target; Sim(ReID) represents the similarity of the re-identified visual appearance features between the current mouse target and the existing tracked target; Sim(Pose) represents the similarity of the skeleton key points between the current mouse target and the existing tracked target; and Sim(Motion) represents the similarity of the motion features between the two.

[0054] The Sim(ReID) can be represented by the cosine similarity between visual appearance feature vectors, the Sim(Pose) can be determined based on the Euclidean distance between corresponding skeleton key points, and the Sim(Motion) can be determined based on the target's motion speed, motion direction, and the similarity between continuous frame trajectories.

[0055] α, β, γ, and δ represent the weight coefficients of the corresponding feature terms, and satisfy the following: α+β+γ+δ=1.

[0056] In one specific implementation, the value of α can be in the range of 0.2 to 0.4, the value of β can be in the range of 0.2 to 0.4, the value of γ can be in the range of 0.1 to 0.3, and the value of δ can be in the range of 0.1 to 0.3.

[0057] For example, α=0.3, β=0.3, γ=0.2, δ=0.2 can be set.

[0058] When the overall matching score reaches the preset matching threshold S0, the mouse target detected in the current video frame is associated with the corresponding existing tracked target as the same target; when the overall matching score does not reach the preset matching threshold S0, a new tracking ID can be created for the currently detected mouse target.

[0059] When multiple rodent targets appear simultaneously in a video image, a unique tracking ID can be assigned to each rodent target, and the morphological features, motion trajectory, and motion state of each rodent target can be maintained separately.

[0060] After target association is completed, Kalman filtering is used to estimate and predict the motion state of the mouse targets.

[0061] In one specific implementation, the target motion state vector used by Kalman filtering is: X=[x, y, vx, vy, ax, ay, w, h]; Where x and y represent the position coordinates of the mouse target, vx and vy represent the movement velocity of the mouse target in the corresponding direction, ax and ay represent the acceleration of the mouse target in the corresponding direction, and w and h represent the width and height of the bounding box of the mouse target, respectively.

[0062] By updating the Kalman filter using the mouse target detection results in consecutive video frames, the position fluctuations generated during the target detection process can be smoothed out, and the position of the mouse target in subsequent moments can be predicted based on the current position, velocity, and acceleration of the mouse target.

[0063] When a mouse target is briefly lost due to occlusion by furniture or other objects, a search area can be established near the last location where the mouse target appeared, and candidate targets can be re-matched in subsequent video frames based on the saved re-identification visual appearance features R.

[0064] For example, 30 frames can be set as the preset re-capture frame count. When a successful re-match occurs within the preset frame count, the original tracking ID and movement trajectory of the corresponding rodent target are restored; if a successful re-match does not occur after the preset frame count, it can be determined that the corresponding rodent target has left the current effective tracking area. Furthermore, the target distance is obtained by a time-of-flight sensor, or determined based on calibrated camera parameters and the target's spatial geometry.

[0065] S4. Determine the emission window based on the target confidence level of the rodent target, target distance, target motion state, the safety status of the directional sound wave beam path, the angular error between the sound wave emission direction and the target direction, the target obstruction status, and the working status of the directional sound wave emission module.

[0066] This implementation does not immediately perform directional acoustic wave emission after detecting a rodent target, but generates a emission permit when all emission conditions are met simultaneously.

[0067] Specifically, the target confidence level P for rodent targets should be greater than or equal to the emission confidence threshold P1, i.e.: P≥P1; The emission confidence threshold P1 is higher than the preset detection threshold P0 used to confirm the target in step S2. For example, P1 can be set to 0.85.

[0068] The target distance d for rodent targets should be within the preset effective range, that is: d∈[dmin, dmax].

[0069] The preset effective operating distance range can be determined based on the effective operating capability of the parametric array transducer array, the installation environment, and the test calibration results.

[0070] The motion state of a rodent target should satisfy: v≤vmax; or: e≤emax; Where v represents the movement speed of the mouse target, vmax represents the preset maximum speed, e represents the prediction hit error of the mouse target, and emax represents the preset maximum hit error.

[0071] For example, vmax can be set to 0.5 m / s.

[0072] The target detection and recognition module also identifies people and pets in video images and obtains the target regions corresponding to the people and pets.

[0073] The sound beam path is determined based on the emission direction of the directional sound wave emission module, the range of the sound beam, and the location of the rodent target. It is then determined whether there is any overlap between the sound beam path and the target area corresponding to the person or pet.

[0074] When the sound beam path overlaps with the area where people or pets are located, the sound beam path safety conditions are not met, and a launch permit is not generated.

[0075] Furthermore, it can also determine whether there are highly reflective or vulnerable objects in the sound beam path. If there are highly reflective or vulnerable objects in the sound beam path that are unsuitable for directional sound wave emission, a emission permit will not be generated.

[0076] The angular error between the current emission direction of the directional acoustic wave emission module and the predicted direction of the rodent target is determined. The angular error condition is satisfied when the angular error is less than or equal to a preset maximum angular error θmax. For example, θmax can be set to 0.5°.

[0077] It also determines whether the rodent target is obstructed by furniture, walls, or other obstacles. The target occlusion condition is met when the rodent target is not obstructed by obstacles and the sound beam propagation path between the directional sound wave emission module and the rodent target is unobstructed.

[0078] Simultaneously, the operating status of the directional acoustic wave emission module is detected. This operating status may include transducer temperature, power supply status, and sound pressure feedback status. When the transducer temperature is within a preset normal range, the power supply status is normal, and the sound pressure feedback status is normal, the directional acoustic wave emission module is determined to be in an allowable emission state.

[0079] Launch permission is generated only when all of the above launch conditions are met simultaneously; if any condition is not met, the tracking status of the rodent target is maintained, and the gimbal is adjusted according to the actual situation, waiting for the target's motion state to change or waiting for the sound beam path to meet the safety conditions again.

[0080] S5. Based on the predicted location of the rodent target and the time delay from video image processing to the arrival of the sound wave at the rodent target location, control the gimbal to drive the directional sound wave emission module to rotate, so that the directional sound wave emission module points to the predicted location of the rodent target at a future time.

[0081] In one specific embodiment, the gimbal is a two-degree-of-freedom gimbal with two rotational degrees of freedom in the pitch and yaw directions, and the directional acoustic wave emission module is mounted on the gimbal.

[0082] The pitch and yaw angles of the gimbal are determined based on parameters such as the predicted location of the rodent target, the camera installation location, the camera's field of view, and the target distance.

[0083] In one specific implementation: pitch = arctan(Δz / d); yaw = arctan(Δx / d); Where Δz represents the vertical positional difference between the rodent target and the directional acoustic wave emitting module, Δx represents the horizontal positional difference between the two, and d represents the distance between the rodent target and the directional acoustic wave emitting module.

[0084] It can also convert the coordinates of the rodent target in the video image into the target rotation angle of the pan-tilt unit by using a pinhole camera model or corresponding geometric transformation relationship, based on the camera installation height, camera field of view, and spatial dimensions of the monitored area.

[0085] The gimbal can use a stepper motor or servo motor as the drive mechanism, and use a PID control algorithm to control the rotation in the pitch and yaw directions respectively.

[0086] In one implementation, the PID control parameters can be dynamically adjusted based on the movement speed of the rodent target. When the rodent target is moving at high speed, the proportional control parameter P is increased to improve the gimbal's response speed to changes in the rodent target's position; when the rodent target is stationary or moving at low speed, the proportional control parameter P is decreased to reduce overshoot and oscillations generated during gimbal rotation; simultaneously, the derivative control parameter D can be used to suppress overshoot during gimbal movement.

[0087] Since there is a certain time delay between the acquisition of video images from the camera and the actual propagation of sound waves to the location of the rodent target, this embodiment calculates the predicted location at future times based on the movement state of the rodent target and compensates for the aforementioned time delay.

[0088] The predicted time Δt of the future time relative to the current time is determined according to the following formula: Δt=t_proc+t_gimbal+t_sound; Where t_proc represents the video image processing latency, t_gimbal represents the gimbal response latency, and t_sound represents the sound wave propagation latency.

[0089] in: t_sound=d / c; d represents the distance between the directional sound wave emitting module and the rodent target, and c represents the speed of sound.

[0090] In one specific implementation, the video image processing delay t_proc can be 20 to 50 ms, the gimbal response delay t_gimbal can be 30 to 100 ms, and the speed of sound in the air c can be approximately 340 m / s.

[0091] For example, when the distance d of the rodent target is 10m, the sound wave propagation delay is about 29ms.

[0092] Based on the prediction time Δt and the motion state of the mouse target obtained by Kalman filtering in step S3, the predicted position of the mouse target at time t+Δt is calculated, and the gimbal is controlled to make the directional sound wave emission module point to the predicted position, thereby jointly compensating for the video image processing delay, gimbal response delay and sound wave propagation delay.

[0093] Once the angular error between the actual emission direction of the directional acoustic wave emission module and the predicted direction of the rodent target reaches the preset requirement, it enters the ready-to-emit state.

[0094] In one embodiment, the present invention may also have a fully automatic working mode and a semi-automatic working mode.

[0095] In fully automatic mode, the system automatically completes target detection of rodents, shape locking, motion tracking, future position prediction, launch window determination, gimbal aiming, and directional sound wave emission.

[0096] In semi-automatic mode, the system automatically completes the detection, tracking, prediction, and gimbal aiming of rodent targets. When the launch window is met, it sends a confirmation message to the user terminal and executes directional sound wave transmission after receiving the user's confirmation command.

[0097] S6. Determine the sound wave emission parameters based on the target distance and movement speed of the rodent target, form a directional sound beam through a parametric array transducer array, and emit the sound wave in a directional manner to the predicted position of the rodent target at a future time according to the corresponding sound wave emission parameters.

[0098] In one specific implementation, the directional acoustic wave transmitting module employs parametric array acoustic wave technology to generate difference frequency acoustic waves through the nonlinear propagation of at least two ultrasonic carrier signals in the air.

[0099] For example, a first ultrasonic carrier signal f1 and a second ultrasonic carrier signal f2 can be generated: f1 = 40kHz; f2 = 41kHz; The corresponding difference frequency is: |f2-f1|=1kHz.

[0100] The frequencies mentioned above are merely one specific implementation method. For example, the ultrasonic carrier frequency can be set in the range of 40kHz to 60kHz, and the frequency of the difference frequency sound wave can be set in the range of 500Hz to 4kHz.

[0101] The directional acoustic wave emission module may include a DDS waveform generator, a power amplifier, a parametric array transducer array, a phase control circuit, and a sound pressure feedback sensor.

[0102] The DDS waveform generator uses direct digital frequency synthesis to generate at least two ultrasonic carrier signals and can support continuous wave, frequency sweep and pulse modulation modes.

[0103] The ultrasonic carrier signal generated by the DDS waveform generator is amplified by a power amplifier and then drives the parametric array transducer array.

[0104] The power amplifier can be a Class-D power amplifier, and its output power is determined based on the size of the parametric array transducer, the rated parameters of the transducer, and the test calibration results.

[0105] A parametric transducer array can include 16 to 64 piezoelectric ceramic transducer elements. Multiple piezoelectric transducer elements can be arranged in a circular array, a rectangular array, or a linear array.

[0106] The spacing between adjacent piezoelectric transducer units can be determined according to the wavelength λ of the ultrasonic carrier used, for example, it can be set to λ / 2. When using a 40kHz ultrasonic carrier as the design reference, λ / 2 is approximately 4.3mm.

[0107] The phase control circuit determines the phase delay φ_i of each piezoelectric transducer unit according to the direction and distance of the rodent target, and controls each piezoelectric transducer unit according to the determined phase delay, so that the sound waves generated by different transducer units are coherently superimposed in the direction of the rodent target, thereby electronically scanning and focusing the sound beam.

[0108] Therefore, the mechanical direction can be adjusted over a wide range of angles using a gimbal, and the direction and focus position of the sound beam can be further adjusted using the phase control of the parametric array transducer array.

[0109] In one specific implementation, the size of the acoustic beam action area formed by the parametric array transducer array is determined by testing and calibration based on the array size, operating frequency, phase control parameters, and target distance.

[0110] A reference microphone can also be placed near the parametric transducer array as a sound pressure feedback sensor. The sound pressure feedback sensor detects the sound pressure output of the parametric transducer array and feeds the detected sound pressure information back to the control processing module. The control processing module adjusts the output of the power amplifier according to the sound pressure feedback information to form a closed-loop control of the sound pressure output.

[0111] The acoustic emission parameters can be dynamically adjusted based on the target distance d, the movement speed v of the rodent target, and the historical acoustic emission results.

[0112] In one specific implementation, the target distance can be divided into a short distance range, a medium distance range, and a long distance range.

[0113] When rodent targets are in the close range, the first set of sound wave emission parameters can be used to match the sound pressure level, pulse width, frequency sweep range, and emission duration at the target location with the close-range action conditions.

[0114] When the target is a rodent at a medium distance, the sound pressure level at the target location can be appropriately increased based on the first set of sound wave emission parameters. A combination of pulse modulation and frequency sweep can be used, and the emission duration can be appropriately extended.

[0115] When the target is a rodent at a distance, the sound pressure level at the target location can be further increased and / or the emission duration can be increased, and the angular accuracy requirements of the sound wave emission direction can be increased accordingly.

[0116] The distance ranges and corresponding acoustic parameters mentioned above are only provided as specific implementation parameters to illustrate the linkage and adjustment relationship between the target distance for rodents and the acoustic parameters. In practical applications, the specific parameters can be determined based on the size of the parametric array transducer array, its output capability, the target distance, and the installation environment.

[0117] When the target mouse is moving at high speed, you can continue to track the target mouse and wait for it to enter a low-speed movement state or a short pause state before emitting sound waves.

[0118] When the future position prediction accuracy obtained in step S3 meets the preset prediction hit error requirement, the gimbal can be aimed in advance and directional sound wave emission can be performed based on the predicted position of the rodent target at a future time.

[0119] When the previous sound wave effect did not meet the preset effect conditions and sound wave emission is performed again, at least one of the sound wave frequency combination, pulse interval and frequency sweep mode can be adjusted to change the sound wave parameter combination used for re-emission.

[0120] S7. After completing one sound wave emission, continue to acquire video images of the rodent target, and evaluate the sound wave effect based on the motion and posture of the rodent target after emission.

[0121] Specifically, it can obtain information such as the velocity change Δv, posture change, lying down state, and static duration t_static of rodent targets.

[0122] In one implementation, the velocity change Δv of the rodent target before and after the sound wave emission is obtained, and the time during which the rodent target remains stationary or in a low-motion state after the sound wave emission is statistically analyzed to obtain the stationary duration t_static.

[0123] When the velocity change Δv of the rodent target reaches the preset velocity change threshold, and the static duration t_static reaches the preset static time threshold, the result of this sound wave action is determined to have met the preset action conditions, and the transmission is stopped.

[0124] If the mouse target is still moving, or if the corresponding state does not meet the preset action conditions, the next launch is not executed immediately, but the launch window determination in step S4 is re-executed.

[0125] The next directional sound wave transmission is only permitted after the target confidence level, target distance, target motion state, sound beam path safety state, angle error, obstruction state, and directional sound wave transmission module state meet the transmission conditions again.

[0126] When transmitting again, the frequency combination, pulse interval, or frequency sweep mode can be adjusted based on the results of the previous sound wave action.

[0127] In one specific implementation, a maximum limit can be set for the cumulative number of launches against the same rodent target, for example, a maximum of three launches are allowed. If the preset condition for action is not met after reaching the preset maximum number of launches, the sound wave emission against the rodent target is stopped, and a corresponding event prompt can be sent to the user terminal.

[0128] The result of the sound wave action can be mainly judged by the speed, movement state, posture and static duration of the rodent target obtained through video images, thereby forming a closed-loop control process of visual detection, directional sound wave emission, visual feedback and re-judgment.

[0129] S8. During the execution of steps S4 to S7, personnel and pets are continuously detected along the sound beam path.

[0130] The target detection and recognition module can detect not only rodent targets, but also human targets and pet targets such as cats and dogs, and obtain the corresponding human target area and pet target area.

[0131] The current sound beam path is determined based on the current direction of the gimbal, the actual emission direction of the directional sound wave emission module, the sound beam range, and the location of the rodent target. It is then determined whether the area where the person or pet target is located has entered the sound beam path.

[0132] If a person or pet is detected in the sound beam path before the sound wave is emitted, no emission permission is generated; for emission permission that has been generated but not yet executed, the emission permission can be revoked.

[0133] If a person or pet is detected entering the beam path during directional sound wave transmission, a safety interruption is immediately executed.

[0134] In one specific implementation, the control processing module shuts down the power amplifier used to drive the parametric array transducer array and locks the gimbal used to adjust the emission direction of the directional acoustic wave emission module, so as to stop the acoustic energy output and stop the gimbal from continuing to track and emit.

[0135] Simultaneously, the temperature of the transducer in the directional acoustic wave emission module can be continuously monitored. When the transducer temperature exceeds the preset temperature threshold, acoustic wave emission is paused; similarly, directional acoustic wave emission is prohibited or interrupted when a power supply abnormality or sound pressure feedback abnormality occurs.

[0136] To limit continuous sound wave output, upper limits can also be set for the duration of a single launch and the cumulative number of launches against the same rodent target.

[0137] After a monitoring event ends, event data corresponding to that event can also be recorded. This event data may include the event time, corresponding video clip, rodent target tracking information, the sound wave emission parameters used, the number of emission cycles, and the results of the sound wave action.

[0138] It can also establish a communication connection with the user terminal via WiFi or 4G communication to enable remote video viewing, event notification, switching between fully automatic and semi-automatic modes, and corresponding remote control.

[0139] This invention also provides an intelligent rodent control and monitoring system based on visual recognition and directional sound waves for performing the above-described methods. The system includes a visible light monitoring camera, a target detection and recognition module, a shape locking and motion tracking module, a pan-tilt servo control module, a directional sound wave emission module, a emission window determination module, and a control processing module.

[0140] Visible light surveillance cameras are used to acquire visible light video images of the monitored area and provide the acquired visible light video images to the target detection and recognition module.

[0141] The target detection and recognition module is used to perform target detection and category recognition on the visible light video image. When a rodent target is detected, it outputs the target confidence score, target bounding box, and skeleton key points of the rodent target. The target detection and recognition module is also used to obtain the target distance of the rodent target. The target distance is obtained by a time-of-flight sensor or determined based on calibrated camera parameters and the spatial geometry of the target.

[0142] The target detection and recognition module is also used to identify people and pets in the visible light video image and output the corresponding area where the people or pets are located, so that the emission window determination module and the control processing module can make a sound beam path safety judgment.

[0143] In one alternative implementation, the system may also be equipped with at least one of a pyroelectric infrared sensor and a time-of-flight sensor to assist in the verification of candidate rodent targets identified by the target detection and recognition module.

[0144] The morphology locking and motion tracking module is connected to the target detection and recognition module. It is used to construct the morphological features of the rodent target based on the target bounding box, skeleton key points, visual appearance features and motion features, and to associate the rodent targets in continuous video images based on the bounding box intersection-union ratio, the similarity of the re-identified visual appearance features, the similarity of the skeleton key points and the similarity of the motion features.

[0145] The morphology locking and motion tracking module is also used to estimate and predict the motion state of the mouse target, such as position, velocity and acceleration, using Kalman filtering to obtain the mouse target's motion trajectory and predicted position at future moments, and to re-match the mouse target using re-identification visual appearance features when the mouse target is briefly occluded.

[0146] The launch window determination module receives information on the target confidence level, target distance, target motion state, sound beam path safety status, angular error between the sound wave emission direction and the target direction, target obstruction status, and the operating status of the directional sound wave emission module for rodent targets. Based on this information, it determines whether to generate a launch permission. The launch window determination module generates a launch permission only when all launch conditions are simultaneously met.

[0147] The gimbal servo control module includes a gimbal with two rotational degrees of freedom in the pitch and yaw directions. The directional acoustic wave emission module is mounted on the gimbal and is driven by the gimbal to change the direction of acoustic wave emission.

[0148] The gimbal servo control module uses the shape locking and motion tracking module to predict the future position of the rodent target, and combines this with video image processing latency, gimbal response latency, and sound wave propagation latency to control the gimbal to perform advanced aiming, so that the directional sound wave emission module points to the predicted future position of the rodent target.

[0149] The gimbal can use a stepper motor or servo motor as the actuator, and achieve servo control of pitch and yaw directions through a PID control algorithm.

[0150] The directional acoustic wave emission module includes a waveform generator, a power amplifier, a parametric array transducer array, and a phase control circuit.

[0151] In one specific embodiment, the waveform generator is a DDS waveform generator used to generate at least two ultrasonic carrier signals. The signal output terminal of the DDS waveform generator is connected to the signal input terminal of a power amplifier, and the signal output terminal of the power amplifier is connected to a parametric transducer array, thereby providing the power-amplified ultrasonic carrier signals to the parametric transducer array.

[0152] The parametric array transducer array comprises multiple piezoelectric transducer units. A phase control circuit determines the phase delay of each piezoelectric transducer unit based on the direction and distance of the rodent target, and controls each piezoelectric transducer unit according to the determined phase delay, enabling electronic scanning and focusing of the sound beam formed by the parametric array transducer array.

[0153] In a further embodiment, the directional sound wave emission module also includes a sound pressure feedback sensor. The sound pressure feedback sensor may be a reference microphone positioned near the parametric transducer array, used to detect the sound pressure output corresponding to the parametric transducer array and send the detected sound pressure information to the control processing module.

[0154] The control processing module can adjust the output of the power amplifier according to the sound pressure information, thereby providing feedback control over the sound pressure output of the directional sound wave emission module.

[0155] The control processing module is connected to the target detection and recognition module, the shape locking and motion tracking module, the gimbal servo control module, the directional acoustic wave emission module, and the emission window determination module, respectively, to coordinate the data transmission and working sequence between the modules.

[0156] During system operation, the control processing module can control the system to switch between standby state, detection state, tracking state, aiming state, launch state, pause state, and fault state.

[0157] When the system is in standby mode, the visible light surveillance camera continuously acquires video images of the monitored area. When a rodent target that meets the target confirmation criteria is continuously detected, the system enters tracking mode; the morphology locking and motion tracking module performs morphology locking and motion tracking on the rodent target and obtains its predicted position at a future time; the transmission window determination module further judges various transmission conditions; when the transmission window is met, the pan-tilt servo control module controls the directional sound wave transmission module to point to the predicted position of the rodent target at a future time; then, the sound wave parameters are selected according to the distance and movement state of the rodent target, and a directional sound beam is formed through the parametric array transducer array.

[0158] After one sound wave transmission ends, the system continues to acquire video images of the rodent target and judges the sound wave effect based on the target's movement and posture. If the sound wave effect does not meet the preset conditions, the transmission window judgment is re-executed, and a second transmission is performed when the transmission window is met again; transmission stops when the preset conditions are met or the preset maximum number of transmissions is reached.

[0159] During the above process, the control processing module continuously acquires the detection results of personnel and pets, and determines whether a person or pet has entered the sound beam path based on the sound beam path. When a person or pet is detected entering the sound beam path, the sound wave emission is prohibited or immediately interrupted.

[0160] When a camera connection error, rodent target loss, gimbal malfunction, transducer temperature error, power supply error, or sound pressure feedback error occurs, the control processing module can control the system to enter a pause state or fault state and stop directional sound wave emission.

[0161] The aforementioned target detection threshold, number of consecutive confirmation frames, comprehensive matching weight, number of recapture frames, video image processing latency, PTZ response latency, ultrasonic carrier frequency, frequency difference range, number of transducer units, array element spacing, target distance range, acoustic parameters, and allowed number of transmissions can all be set and adjusted according to the actual application scenario, equipment performance, and monitoring area. As long as the corresponding functions can be implemented according to the above-mentioned technical process of target detection, shape locking, motion prediction, transmission window determination, advanced aiming, parametric array directional acoustic wave transmission, visual feedback, and safety interruption, the corresponding specific parameter configurations can be adopted.

[0162] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural transformations made using the contents of the present invention's specification and drawings under the inventive concept of the present invention, or direct / indirect applications in other related technical fields, are included within the patent protection scope of the present invention.

Claims

1. A smart rodent control and monitoring method based on visual recognition and directional sound waves, characterized in that, Includes the following steps: S1. Acquire video images of the monitored area using a visible light surveillance camera, and perform real-time target detection on the video images; S2. Perform category recognition on the target in the video image to obtain the target category, target confidence, target bounding box and skeleton key points. When the target confidence of the identified rodent target reaches the preset detection threshold and the rodent target is detected in N consecutive video images, confirm the rodent target and enter the tracking state. S3. Construct the morphological features of the rodent target based on the bounding box geometric parameters, skeleton key points, visual appearance features, and motion features of the rodent target; perform target association on the rodent targets in continuous video images; estimate and predict the motion state of the rodent targets based on Kalman filtering; obtain the motion trajectory and predicted position of the rodent targets at future times; and obtain the target distance of the rodent targets; the target distance is obtained by a time-of-flight sensor or determined based on calibrated camera parameters and target spatial geometric relationships. S4. Determine the launch window based on the target confidence level, target distance, target motion state, beam path safety status of the directional sound wave, angle error between the sound wave emission direction and the target direction, target obstruction status, and working status of the directional sound wave emission module. Generate a launch permit when all launch conditions are met. S5. Based on the predicted position of the rodent target and the time delay from video image processing to the arrival of the sound wave at the target position, control the gimbal to drive the directional sound wave emission module to rotate, so that the emission direction of the directional sound wave emission module is pointed to the predicted position of the rodent target at a future time. S6. Determine the sound wave emission parameters based on the target distance and movement speed of the rodent target, form a directional sound beam through a parametric array transducer array, and emit the sound wave in a directional manner towards the predicted position of the rodent target according to the sound wave emission parameters. S7. After the sound wave is emitted, continue to acquire video images of the rodent target, and evaluate the sound wave effect based on the motion and posture of the rodent target after the emission; when the sound wave effect reaches the preset effect condition, stop the emission; when the sound wave effect does not reach the preset effect condition and the emission window is satisfied again, perform directional sound wave emission again within the preset maximum number of emission times. S8. During the execution of steps S4 to S7, personnel and pets are detected in the sound beam path. When personnel or pets are detected entering the sound beam path, sound wave emission is prohibited or interrupted.

2. The intelligent rodent control and monitoring method based on visual recognition and directional sound waves according to claim 1, characterized in that, In step S2: When the target confidence P of the detected rodent target is greater than or equal to the preset detection threshold P0 and the rodent target is detected in N consecutive video images, the rodent target is confirmed; False alarms are filtered based on various features of the rodent target, including its size ratio, aspect ratio, movement speed, movement intermittency, and consistency of continuous frame detection. Auxiliary verification information of candidate rodent targets is obtained by using at least one of a pyroelectric infrared sensor and a time-of-flight sensor; when the pyroelectric infrared sensor is used, infrared response information corresponding to the candidate rodent target is obtained to help determine whether it has the infrared response characteristics corresponding to a live target. When the time-of-flight sensor is used, distance information corresponding to the candidate rodent target is obtained; when the pyroelectric infrared sensor is used and the obtained infrared response information does not characterize the infrared response features corresponding to the live target, or when the time-of-flight sensor is used and the obtained distance information is abnormal, the directional acoustic emission process for the candidate rodent target is not triggered.

3. The intelligent rodent control and monitoring method based on visual recognition and directional sound waves according to claim 1, characterized in that, In step S3: The morphological characteristics of the rodent target are represented as follows: F = [B, K, R, M]; Wherein, B represents the bounding box geometric parameters of the rodent target, K represents the skeleton key points of the rodent target, R represents the re-identification visual appearance features of the rodent target, and M represents the motion features of the rodent target in continuous video images; A comprehensive matching score is calculated based on the detected rodent targets in the current video image and the existing tracked targets: Score=α·IoU+β·Sim(ReID)+γ·Sim(Pose)+δ·Sim(Motion); Where IoU represents the intersection-union ratio of bounding boxes, Sim(ReID) represents the similarity of visual appearance features in re-identification, Sim(Pose) represents the similarity of skeleton key points, and Sim(Motion) represents the similarity of motion features. α, β, γ and δ are the corresponding weight coefficients, and α+β+γ+δ=1. When the overall matching score reaches the preset matching threshold, the mouse target detected in the current video image is associated with the corresponding existing tracked target as the same target.

4. The intelligent rodent control and monitoring method based on visual recognition and directional sound waves according to claim 3, characterized in that, In step S3: The motion state of the mouse target, including its position, velocity, and acceleration, is estimated and predicted based on Kalman filtering. When the mouse target is lost due to occlusion, a search area is established near the last location of the mouse target. Candidate targets in subsequent video images are re-matched based on the re-identified visual appearance features. When a re-match is successful within a preset number of frames, the original tracking identifier of the mouse target is restored.

5. The intelligent rodent control and monitoring method based on visual recognition and directional sound waves according to claim 1, characterized in that, In step S4, the transmission window simultaneously satisfies the following conditions: The target confidence P of the rodent target is greater than or equal to the emission confidence threshold P1, wherein the emission confidence threshold P1 is higher than the preset detection threshold P0 in step S2; The target distance d of the rodent target is within the preset effective range; The moving speed v of the rodent target is less than or equal to the preset maximum speed vmax, or the prediction hit error e of the rodent target is less than or equal to the preset maximum hit error emax; The sound beam path does not overlap with the areas where the detected people and pets are located; The angular error between the sound wave emission direction and the target direction is less than or equal to the preset maximum angular error θmax; The rodent target was not obstructed by any obstacle, and the sound beam propagation path was unobstructed; Furthermore, the directional acoustic wave transmitting module is in a working state where transmission is permitted.

6. The intelligent rodent control and monitoring method based on visual recognition and directional sound waves according to claim 1, characterized in that, In step S5: The predicted time Δt of the future time relative to the current time is determined according to the following formula: Δt=t_proc+t_gimbal+t_sound; Where t_proc represents video image processing latency, t_gimbal represents gimbal response latency, and t_sound represents sound wave propagation latency; The sound wave propagation delay satisfies: t_sound=d / c; Where d represents the distance from the directional sound wave emitting module to the rodent target, and c represents the speed of sound; The predicted position of the rodent target at time t+Δt is obtained based on the predicted time Δt, and forward aiming is performed based on the predicted position; The servo control parameters of the gimbal are dynamically adjusted according to the movement speed of the rodent target.

7. The intelligent rodent control and monitoring method based on visual recognition and directional sound waves according to claim 1, characterized in that, In step S6: At least two ultrasonic carrier signals are generated and the parametric array transducer array is driven. The phase delay of each transducer unit in the parametric array is determined based on the direction and distance of the rodent target, and the sound beam is electronically scanned and focused by controlling the phase delay of each transducer unit. The at least two ultrasonic carrier signals generate difference frequency sound waves during the nonlinear propagation of the air.

8. The intelligent rodent control and monitoring method based on visual recognition and directional sound waves according to claim 1, characterized in that, In step S6: Based on the distance range to which the target distance of the rodent target belongs, the sound pressure level, pulse width, frequency sweep range, and transmission duration at the target location are jointly determined. As the target distance of the rodent target increases, the sound pressure level at the target location is increased and / or the emission duration is increased, and the angular accuracy requirement of the sound wave emission direction is improved; When the rodent target is in a high-speed movement state, wait for the rodent target to enter a low-speed movement state before emitting sound waves, or emit sound waves in advance based on the predicted position of the rodent target in the future. When the historical sound wave effect does not meet the preset effect conditions and step S6 is executed again, at least one of the sound wave frequency combination, pulse interval, and sweep frequency mode is adjusted.

9. The intelligent rodent control and monitoring method based on visual recognition and directional sound waves according to claim 1, characterized in that, In steps S7 and S8: The sound wave effect is determined based on the speed change and static duration of the rodent target. When the speed change of the rodent target reaches a preset speed change threshold and the static duration reaches a preset static time threshold, the sound wave effect is determined to meet the preset effect condition. If the effect of the sound wave does not meet the preset effect condition, the directional sound wave will be emitted again when the emission window is met again, and the total number of sound wave emissions will not exceed the preset maximum number. When a person or pet enters the sound beam path, the power amplifier used to drive the parametric array transducer array is turned off, and the pan-tilt unit used to adjust the emission direction of the directional sound wave emission module is locked.

10. An intelligent rodent control and monitoring system based on visual recognition and directional sound waves, characterized in that, It includes a visible light surveillance camera, a target detection and recognition module, a shape locking and motion tracking module, a transmission window determination module, a PTZ servo control module, a directional sound wave transmission module, and a control processing module; The visible light surveillance camera is used to capture visible light video images of the monitored area; The target detection and recognition module is used to perform target detection and category recognition on the visible light video image. When a rodent target is detected, it outputs the target confidence score, target bounding box, and skeleton key points of the rodent target. It is also used to identify human and pet targets in the visible light video image and to obtain the target distance of the rodent target. The target distance is obtained by a time-of-flight sensor or determined based on calibrated camera parameters and target spatial geometry. The morphology locking and motion tracking module is used to perform target association and motion tracking of the rodent target based on the target bounding box, skeletal key points, visual appearance features and motion features of the rodent target, and to obtain the motion trajectory and predicted position of the rodent target at future time through Kalman filtering; The launch window determination module is used to determine the launch window based on the target confidence level, target distance, target motion state, sound beam path safety state, angular error between the sound wave emission direction and the target direction, target obstruction state, and the working state of the directional sound wave emission module, and to generate a launch permit when all launch conditions are met. The gimbal servo control module is used to drive the directional sound wave emission module to change the sound wave emission direction, and according to the predicted future position of the rodent target, as well as the video image processing delay, gimbal response delay and sound wave propagation delay, the directional sound wave emission module is made to point to the predicted future position of the rodent target. The directional acoustic wave emission module includes a waveform generator, a power amplifier, a parametric array transducer array, and a phase control circuit. The waveform generator is used to generate at least two ultrasonic carrier signals. The power amplifier is used to drive the parametric array transducer array. The phase control circuit is used to control the phase delay of each transducer unit in the parametric array transducer array according to the direction and distance of the rodent target, so as to electronically scan and focus the sound beam. The control processing module is connected to the target detection and recognition module, the shape locking and motion tracking module, the launch window determination module, the gimbal servo control module, and the directional sound wave emission module, respectively, and is used to coordinate target detection, target tracking, launch window determination, gimbal aiming, sound wave parameter determination, directional sound wave emission, sound wave effect result evaluation, and safety interruption.

Images