Self-adaptive acousto-optic fusion detection sensor and operation method thereof

Abstract

The invention discloses a self-adaptive acousto-optic fusion detection sensor and an operation method thereof, and the self-adaptive acousto-optic fusion detection sensor comprises the following modules: a multi-mode sensing module, an environment sensing module, an intelligent processing unit, a data transmission module, and a power management module. According to the invention, through the environment sensing module and the environment prediction model, the sensor can dynamically adjust the working mode according to the environment change so as to adapt to various complex scenes and have strong adaptability; the multi-modal data fusion and deep reinforcement learning algorithm significantly improves the precision and reliability of target detection and recognition; an intelligent power management system and an energy recovery technology ensure stable operation of the sensor in different environments, and the service life is prolonged; the remote monitoring and management are supported, the operation and maintenance cost is reduced, the flexibility and expandability of the system are improved, and the system has very high intellectualization.

Description

Technical Field

[0001] The present invention relates to the field of sensor technology, and in particular to an adaptive acoustic-optical fusion detection sensor and an operation method thereof. Background Art

[0002] In existing technologies, acoustic and optical sensors are widely used in various detection tasks. However, single acoustic or optical sensors often have limitations in complex environments. For example, acoustic sensors are easily interfered with by environmental noise, and optical sensors have limited performance in low-light or complex background conditions. Although existing technologies have attempted to simply combine acoustic and optical sensors, these combinations generally lack adaptive capabilities and are unable to dynamically adjust detection strategies based on environmental changes, resulting in low detection accuracy and efficiency. In addition, no existing sensor can simultaneously achieve deep fusion of multimodal data, dynamic adjustment driven by environmental perception, and intelligent decision-making capabilities.

[0003] Based on this, an adaptive acoustic-optical fusion detection sensor and its operation method are designed. Summary of the Invention

[0004] The purpose of the present invention is to provide an adaptive acoustic-optical fusion detection sensor and an operating method thereof to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] An adaptive acoustic-optical fusion detection sensor, comprising the following components:

[0007] Multimodal perception module: including a high-sensitivity acoustic sensor array and a wide-spectrum optical sensor array;

[0008] Environmental perception module: used to monitor environmental parameters in real time, including light intensity 0-100,000 lx, temperature -40°C-70°C, humidity 0%-100% RH, noise level 20dB-120dB, wind speed 0-30m / s, and wind direction 0-360°; this module provides a basis for adaptive adjustment of the sensor;

[0009] Intelligent processing unit: Based on deep learning algorithms and adaptive control strategies, it integrates and processes acoustic and optical data. This unit can dynamically adjust the sensor's operating mode and parameters based on data provided by the environmental perception module.

[0010] Data transmission module: supports wired and wireless communication to transmit the processed detection results to the monitoring center;

[0011] Power management module: Adopts efficient energy management system, supports solar charging and battery power supply, and ensures stable operation of the sensor in different environments.

[0012] Preferably, in the multimodal perception module, the acoustic sensor array adopts a distributed microphone array, which can capture wide-band sound signals, covering a frequency range of 20Hz-100kHz, and supports adaptive beamforming technology, which can dynamically adjust the beam direction according to the target direction, thereby improving the positioning accuracy of the acoustic signal;

[0013] The optical sensor array includes a multispectral camera and a thermal imaging camera, which can simultaneously collect visible light 400nm-700nm, near-infrared light 700nm-1100nm and thermal infrared images. It supports multispectral fusion technology and can provide richer target information in low light or complex background conditions.

[0014] Preferably, in the data transmission module, wired communication includes RS485 and Ethernet communication, and wireless communication includes Wi-Fi, 4G, and 5G communication.

[0015] A method for operating an adaptive acoustic-optical fusion detection sensor comprises the following steps:

[0016] Step S1: Environmental parameter collection and initialization

[0017] S1.1. The environmental perception module is activated to collect the current environment's light intensity, temperature, humidity, and noise level in real time;

[0018] S1.2. Based on the collected environmental parameters, the intelligent processing unit initializes the working mode of the sound and light sensor;

[0019] When the light intensity is lower than 100lx, the optical sensor switches to low light mode and the gain is increased by 30%;

[0020] When the noise level exceeds 70dB, the sampling frequency of the acoustic sensor is increased to 10kHz.

[0021] S1.3. The data transmission module establishes a communication link and prepares to receive and transmit data;

[0022] Step S2: Multimodal data acquisition and preprocessing

[0023] S2.1. The multimodal perception module activates the acoustic sensor array and the optical sensor array to synchronously collect acoustic signals and optical images.

[0024] S2.2. The acoustic signal is filtered to remove interference signals with frequencies below 20 Hz and above 20 kHz. The optical image is processed using a denoising algorithm to reduce the impact of ambient noise on image quality.

[0025] S2.3. The intelligent processing unit performs time alignment and spatial registration on the pre-processed acoustic and optical data to ensure data consistency.

[0026] Step S3: Adaptive fusion and target recognition

[0027] S3.1. The intelligent processing unit dynamically adjusts the weight distribution of the fusion algorithm based on the real-time data and environmental prediction results provided by the environmental perception module;

[0028] When the ambient noise is high, the weight of optical data is increased;

[0029] When lighting conditions are poor, increase the weight of acoustic data;

[0030] S3.2. Use a deep learning algorithm to identify and classify targets on the fused data. The algorithm can identify a variety of targets, including people, vehicles, animals, etc., and output corresponding recognition results based on the target type.

[0031] S3.3. The intelligent processing unit generates an alarm signal or control instruction based on the recognition result and sends it to the monitoring center through the data transmission module;

[0032] Step S4: Adaptive adjustment and optimization

[0033] S4.1. The intelligent processing unit dynamically adjusts the sensor's operating mode and parameters based on target recognition results and environmental parameter changes;

[0034] If the target moves faster, the optical sensor's frame rate increases to 30fps;

[0035] If the target is far away, the gain of the acoustic sensor is increased by 20%;

[0036] S4.2. The sensor conducts regular self-tests to check the working status of each module. If any abnormality is found, it automatically switches to the backup mode and sends a fault report to the monitoring center through the data transmission module.

[0037] S4.3, the power management module dynamically adjusts the solar charging power and battery discharge strategy according to the sensor's operating status and ambient light conditions to ensure continuous operation of the sensor;

[0038] Step S5: Data transmission and monitoring

[0039] S5.1. The data transmission module transmits the processed detection results and sensor status information to the monitoring center.

[0040] S5.2. After receiving the data, the monitoring center will further analyze and store it, and send control instructions to the sensor as needed to achieve remote monitoring and management.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] Through the environmental perception module and environmental prediction model, the present invention enables the sensor to dynamically adjust its working mode according to environmental changes, adapt to various complex scenarios, and has strong adaptability;

[0043] Multimodal data fusion and deep reinforcement learning algorithms significantly improve the accuracy and reliability of target detection and recognition;

[0044] Intelligent power management system and energy recovery technology ensure stable operation of the sensor in different environments and extend its service life;

[0045] It supports remote monitoring and management, reduces operation and maintenance costs, improves system flexibility and scalability, and is highly intelligent. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0047] In the attached figure:

[0048] Figure 1 A schematic flow chart of an operation method according to an exemplary embodiment of the present invention is shown;

[0049] Figure 2 shows a schematic diagram of the module structure of the sensor; DETAILED DESCRIPTION

[0050] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with specific embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0051] An adaptive acoustic-optical fusion detection sensor according to this embodiment includes the following components:

[0052] Multimodal perception module: including a high-sensitivity acoustic sensor array and a wide-spectrum optical sensor array;

[0053] Environmental perception module: used to monitor environmental parameters in real time, including light intensity 0-100,000 lx, temperature -40°C-70°C, humidity 0%-100% RH, noise level 20dB-120dB, wind speed 0-30m / s, and wind direction 0-360°; this module provides a basis for adaptive adjustment of the sensor;

[0054] Intelligent processing unit: Based on deep learning algorithms and adaptive control strategies, it integrates and processes acoustic and optical data. This unit can dynamically adjust the sensor's operating mode and parameters based on data provided by the environmental perception module.

[0055] Data transmission module: supports wired and wireless communication to transmit the processed detection results to the monitoring center;

[0056] Power management module: Adopts efficient energy management system, supports solar charging and battery power supply, and ensures stable operation of the sensor in different environments.

[0057] In the multimodal perception module of this embodiment, the acoustic sensor array uses a distributed microphone array, which can capture wide-band sound signals, covering the frequency range of 20Hz-100kHz. It supports adaptive beamforming technology, which can dynamically adjust the beam direction according to the target direction, thereby improving the positioning accuracy of the acoustic signal.

[0058] The optical sensor array includes a multispectral camera and a thermal imaging camera, which can simultaneously collect visible light 400nm-700nm, near-infrared light 700nm-1100nm and thermal infrared images. It supports multispectral fusion technology and can provide richer target information in low light or complex background conditions.

[0059] In the data transmission module of this embodiment, wired communication includes RS485 and Ethernet communication, and wireless communication includes Wi-Fi, 4G, and 5G communication.

[0060] A method for operating an adaptive acoustic-optical fusion detection sensor comprises the following steps:

[0061] Step S1: Environmental parameter collection and initialization

[0062] S1.1. The environmental perception module is activated to collect the current environment's light intensity, temperature, humidity, and noise level in real time;

[0063] S1.2. Based on the collected environmental parameters, the intelligent processing unit initializes the working mode of the sound and light sensor;

[0064] When the light intensity is lower than 100lx, the optical sensor switches to low light mode and the gain is increased by 30%;

[0065] When the noise level exceeds 70dB, the sampling frequency of the acoustic sensor is increased to 10kHz.

[0066] S1.3. The data transmission module establishes a communication link and prepares to receive and transmit data;

[0067] Step S2: Multimodal data acquisition and preprocessing

[0068] S2.1. The multimodal perception module activates the acoustic sensor array and the optical sensor array to synchronously collect acoustic signals and optical images.

[0069] S2.2. The acoustic signal is filtered to remove interference signals with frequencies below 20 Hz and above 20 kHz. The optical image is processed using a denoising algorithm to reduce the impact of ambient noise on image quality.

[0070] S2.3. The intelligent processing unit performs time alignment and spatial registration on the pre-processed acoustic and optical data to ensure data consistency.

[0071] Step S3: Adaptive fusion and target recognition

[0072] S3.1. The intelligent processing unit dynamically adjusts the weight distribution of the fusion algorithm based on the real-time data and environmental prediction results provided by the environmental perception module;

[0073] When the ambient noise is high, the weight of optical data is increased;

[0074] When lighting conditions are poor, increase the weight of acoustic data;

[0075] S3.2. Use a deep learning algorithm to identify and classify targets on the fused data. The algorithm can identify a variety of targets, including people, vehicles, animals, etc., and output corresponding recognition results based on the target type.

[0076] S3.3. The intelligent processing unit generates an alarm signal or control instruction based on the recognition result and sends it to the monitoring center through the data transmission module;

[0077] Step S4: Adaptive adjustment and optimization

[0078] S4.1. The intelligent processing unit dynamically adjusts the sensor's operating mode and parameters based on target recognition results and environmental parameter changes;

[0079] If the target moves faster, the optical sensor's frame rate increases to 30fps;

[0080] If the target is far away, the gain of the acoustic sensor is increased by 20%;

[0081] S4.2. The sensor conducts regular self-tests to check the working status of each module. If any abnormality is found, it automatically switches to the backup mode and sends a fault report to the monitoring center through the data transmission module.

[0082] S4.3, the power management module dynamically adjusts the solar charging power and battery discharge strategy according to the sensor's operating status and ambient light conditions to ensure continuous operation of the sensor;

[0083] Step S5: Data transmission and monitoring

[0084] S5.1. The data transmission module transmits the processed detection results and sensor status information to the monitoring center.

[0085] S5.2. After receiving the data, the monitoring center will further analyze and store it, and send control instructions to the sensor as needed to achieve remote monitoring and management.

[0086] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations that come within the meaning and range of equivalents of the claims be embraced therein.

[0087] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. An adaptive acoustic and optical fusion detection sensor, characterized in that: Includes the following building blocks: Multimodal perception module: including a high-sensitivity acoustic sensor array and a wide-spectrum optical sensor array; Environmental perception module: used to monitor environmental parameters in real time, including light intensity 0-100,000 lx, temperature -40°C-70°C, humidity 0%-100% RH, noise level 20dB-120dB, wind speed 0-30m / s, and wind direction 0-360°; this module provides a basis for adaptive adjustment of the sensor; Intelligent processing unit: Based on deep learning algorithms and adaptive control strategies, it integrates and processes acoustic and optical data; This unit can dynamically adjust the working mode and parameters of the sensor according to the data provided by the environmental perception module; Data transmission module: supports wired and wireless communication to transmit the processed detection results to the monitoring center; Power management module: Adopts efficient energy management system, supports solar charging and battery power supply, and ensures stable operation of the sensor in different environments.

2. The adaptive acoustic-optical fusion detection sensor according to claim 1, characterized in that: In the multimodal perception module, the acoustic sensor array uses a distributed microphone array, which can capture wide-band sound signals, covering the frequency range of 20Hz-100kHz. It supports adaptive beamforming technology and can dynamically adjust the beam direction according to the target direction to improve the positioning accuracy of the acoustic signal. The optical sensor array includes a multispectral camera and a thermal imaging camera, which can simultaneously collect visible light 400nm-700nm, near-infrared light 700nm-1100nm and thermal infrared images. It supports multispectral fusion technology and can provide richer target information in low light or complex background conditions.

3. The adaptive acoustic-optical fusion detection sensor according to claim 1, characterized in that: In the data transmission module, wired communication includes RS485 and Ethernet communication, and wireless communication includes Wi-Fi, 4G, and 5G communication.

4. A method for operating an adaptive acoustic-optical fusion detection sensor, characterized in that: The following steps are involved: Step S1: Environmental parameter collection and initialization S1.

1. The environmental perception module is activated to collect the current environment's light intensity, temperature, humidity, and noise level in real time; S1.

2. Based on the collected environmental parameters, the intelligent processing unit initializes the working mode of the sound and light sensor; When the light intensity is lower than 100lx, the optical sensor switches to low light mode and the gain is increased by 30%; When the noise level exceeds 70dB, the sampling frequency of the acoustic sensor is increased to 10kHz. S1.

3. The data transmission module establishes a communication link and prepares to receive and transmit data; Step S2: Multimodal data acquisition and preprocessing S2.

1. The multimodal perception module activates the acoustic sensor array and the optical sensor array to synchronously collect acoustic signals and optical images. S2.

2. The acoustic signal is filtered to remove interference signals with frequencies below 20 Hz and above 20 kHz. The optical image is processed using a denoising algorithm to reduce the impact of ambient noise on image quality. S2.

3. The intelligent processing unit performs time alignment and spatial registration on the pre-processed acoustic and optical data to ensure data consistency. Step S3: Adaptive fusion and target recognition S3.

1. The intelligent processing unit dynamically adjusts the weight distribution of the fusion algorithm based on the real-time data and environmental prediction results provided by the environmental perception module; When the ambient noise is high, the weight of optical data is increased; When lighting conditions are poor, increase the weight of acoustic data; S3.

2. Use a deep learning algorithm to identify and classify targets on the fused data. The algorithm can identify a variety of targets, including people, vehicles, animals, etc., and output corresponding recognition results based on the target type. S3.

3. The intelligent processing unit generates an alarm signal or control instruction based on the recognition result and sends it to the monitoring center through the data transmission module; Step S4: Adaptive adjustment and optimization S4.

1. The intelligent processing unit dynamically adjusts the sensor's operating mode and parameters based on target recognition results and environmental parameter changes; If the target moves faster, the optical sensor's frame rate increases to 30fps; If the target is far away, the gain of the acoustic sensor is increased by 20%; S4.

2. The sensor performs regular self-tests to check the working status of each module; If an abnormality is detected, it will automatically switch to the backup mode and send a fault report to the monitoring center through the data transmission module; S4.3, the power management module dynamically adjusts the solar charging power and battery discharge strategy according to the sensor's operating status and ambient light conditions to ensure continuous operation of the sensor; Step S5: Data transmission and monitoring S5.

1. The data transmission module transmits the processed detection results and sensor status information to the monitoring center. S5.

2. After receiving the data, the monitoring center will further analyze and store it, and send control instructions to the sensor as needed to achieve remote monitoring and management.

Images