Method and device for controlling self-powered camera of multispectral photon sieve and medium

Abstract

The invention discloses a control method and device for a multispectral photon sieve self-powered camera and a medium, and the method comprises the steps: carrying out the multiband spectrum separation of incident light through a nano-structure filtering layer, obtaining optical signals of different wavelengths, and transmitting the separated optical signals to a photon storage and calculation integrated chip for optical domain feature extraction; incident light is reflected to a wavefront sensor through a beam splitter prism for wavefront distortion detection, and micromirror adjustment parameters are calculated based on wavefront distortion data so as to drive a micromirror execution mechanism to dynamically compensate aberration; environment energy is collected through a flexible solar cell and a vibration energy collector, and the environment energy is converted into electric energy; the electric energy is stored in a storage battery, the corresponding electric quantity state of the storage battery is monitored, and the working mode of the camera is dynamically switched according to the electric quantity state; the working mode comprises a full-function mode and a low-power-consumption mode.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent camera equipment, in particular to a control method, device and medium of a multi-spectral photon sieve self-powered camera. BACKGROUND

[0002] The current camera widely used in the field of intelligent environment perception is mainly based on visible light imaging technology, which has a limited spectral response range and cannot obtain spectral feature information of environmental substances, resulting in single function in application scenarios such as smoke detection and mold identification. Although some high-end devices try to introduce multi-spectral imaging technology, they generally use time-sharing collection or filter wheel switching mechanism, which not only has complex structure and large volume, but also has slow response speed, high power consumption and other problems, making it difficult to meet the needs of real-time environmental monitoring.

[0003] The adaptability of the existing camera system under complex lighting conditions is poor. Image distortion problems caused by factors such as backlight and strong light have long plagued the industry. Although some solutions use software algorithms for post-correction, they have the defects of large processing delay and serious loss of details. Adaptive optical correction technology at the hardware level is mostly used in professional fields such as astronomical observation, and its large volume and complex control system make it difficult to integrate into small cameras, which cannot adapt to the space limitations and cost requirements of civilian scenarios such as smart homes.

[0004] In terms of energy supply, traditional cameras rely heavily on wired power supply or large-capacity batteries, which greatly limits the installation flexibility and deployment range of the device. Although some research attempts to introduce solar power, the single energy source has insufficient reliability in continuous rainy or low-light environments, and lacks effective energy management mechanisms, resulting in the device stopping working directly when the energy is insufficient, which cannot guarantee the continuity of basic monitoring functions. In addition, the existing camera system generally has the problem of unreasonable power consumption distribution of functional modules, and high-energy-consuming components continue to run during unnecessary periods, further exacerbating the energy shortage. SUMMARY

[0005] The embodiments of the present application provide a control method, device and medium of a multi-spectral photon sieve self-powered camera to solve the above technical problems.

[0006] In one aspect, the embodiments of the present application provide a control method of a multi-spectral photon sieve self-powered camera, comprising: performing multi-band spectral separation on incident light through a nano-structured filtering layer to obtain light signals of different wavelengths, and transmitting the separated light signals to a photon storage and computing integrated chip for light domain feature extraction; reflecting the incident light to a wavefront sensor through a beam splitter prism for wavefront distortion detection, and calculating micro-mirror adjustment parameters based on the wavefront distortion data to drive the micro-mirror actuator to dynamically compensate for aberration. Harvesting ambient energy by a flexible solar cell and a vibration energy harvester, and converting the ambient energy into electrical energy; Storing the electrical energy to a storage battery, monitoring a corresponding state of charge of the storage battery, and dynamically switching a working mode of a camera according to the state of charge; the working mode includes a full-function mode and a low-power-consumption mode.

[0007] In an implementation manner of the present application, multi-band spectral separation of incident light is performed by a nano-structured filter layer, and light signals of different wavelengths are obtained, specifically including: A periodic nano-grating structure is made inside a front-end optical glass of a lens, and the periodic nano-grating structure is coaxially aligned with an optical center of the lens, so that the incident light passes through the periodic nano-grating structure vertically, and wavelength-selective transmission is generated; Different wavelengths of light signals are filtered and selected to match the characteristic absorption spectrum of a target substance.

[0008] In an implementation manner of the present application, the separated light signals are transmitted to a photonic storage and computing integrated chip for light domain feature extraction, specifically including: The photonic storage and computing integrated chip and the multispectral imaging unit image sensor are pin-bonded by flip-chip technology, so that the light input port of the photonic storage and computing integrated chip is connected to the light output port of the nano-structured filter layer by using a fiber array; The separated light signals are transmitted to the photonic storage and computing integrated chip by the fiber array, and an interference pattern is generated by multi-beam interference; The spatial frequency distribution of the interference pattern is analyzed to extract the interference pattern features, and the interference pattern features are compared with a preset substance spectrum database to obtain corresponding light domain features.

[0009] In an implementation manner of the present application, before the incident light is reflected to a wavefront sensor by a beam splitter prism for wavefront distortion detection, the method further includes: The wavefront sensor is coaxially installed with a light reflection path of the beam splitter prism, and an output end of the wavefront sensor is connected to a high-speed input pin of an FPGA through an LVDS interface; The beam splitter prism is fixed between the lens and the wavefront sensor by a support; The MEMS micro-mirror is fixed between the lens and the beam splitter prism, and a driving circuit of the MEMS micro-mirror is connected to a DAC output end of the FPGA.

[0010] In an implementation manner of the present application, the incident light is reflected to the wavefront sensor by the beam splitter prism for wavefront distortion detection, and micro-mirror adjustment parameters are calculated based on wavefront distortion data to drive the micro-mirror actuator to dynamically compensate aberration, specifically including: reflecting the preset proportion of the incident light to the aberration sensing unit through a spectrometer prism, and forming a spot array using a microlens array to capture the wavefront distortion; transmitting the detected aberration data to a processing unit, calculating the micro-mirror adjustment parameters, and amplifying the digital-to-analog converter output signal to a driving voltage to control the MEMS micro-mirror to deflect in a two-dimensional plane to achieve dynamic compensation of aberration.

[0011] In an implementation manner of the present application, further comprising: waiting for a preset time interval after the adjustment of the MEMS micro-mirror, and performing secondary detection on the compensated wavefront distortion; judging whether the compensated wavefront distortion needs further adjustment according to the secondary detection result, and if so, repeatedly performing dynamic compensation of aberration on the compensated wavefront distortion until the wavefront distortion is less than a preset tolerance value.

[0012] In an implementation manner of the present application, environmental energy is collected through a flexible solar cell and a vibration energy collector, and the environmental energy is converted into electrical energy, specifically comprising: attaching the flexible solar cell on the surface of the camera shell to capture light energy, and connecting the flexible solar cell to a charge management chip through an anti-reverse connection circuit; installing the vibration energy collector on the shock-absorbing support at the bottom of the camera to capture vibration energy, and converting the vibration energy into direct current through a rectifier circuit; parallelly connecting the outputs of the two energy sources to the charge management chip to uniformly manage the outputs of the two energy sources through the charge management chip.

[0013] In an implementation manner of the present application, the corresponding power state of the storage battery is monitored, and the working mode of the camera is dynamically switched according to the power state, specifically comprising: real-time monitoring the voltage and current parameters of the storage battery, and keeping the photonic computing and storage integrated chip and the multi-spectral imaging unit working simultaneously in the case of detecting that the power is higher than a first threshold value; in the case of detecting that the power is lower than a second threshold value, sending a low-power consumption instruction to the main controller to turn off the photonic computing and storage integrated chip, and reducing the working frequency of the multi-spectral imaging unit according to a preset rule.

[0014] On the other hand, the embodiments of the present application also provide a control device of a multi-spectral photonic sieve self-powered camera, the device comprising: at least one processor; and a memory in communication connection with the at least one processor; The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the control method of the multi-spectrum photon sieve self-powered camera as described above.

[0015] In another aspect, the embodiments of the present application also provide a nonvolatile computer storage medium, which stores computer executable instructions, and the computer executable instructions, when executed, implement the control method of the multi-spectrum photon sieve self-powered camera as described above.

[0016] The embodiments of the present application provide a control method, device and medium of a multi-spectrum photon sieve self-powered camera, which at least have the following beneficial effects: Directly using light interference / diffraction to complete feature extraction in the optical domain, eliminating the electrical signal conversion link, reducing the spectral analysis delay by an order of magnitude, and ensuring real-time identification of environmental substances such as smoke and mold; Hardware fusion of photon computing and spectral filtering avoids the data bandwidth limitation between the optical module and the external processor in the traditional scheme, significantly improves the multi-spectrum information throughput efficiency; Splitting synchronously performs imaging and distortion detection, adjusts the micro-mirror deflection through real-time feedback, and overcomes image blur caused by backlight and scattering; Aberration compensation works with the imaging light path to ensure the clarity and reliability of multi-spectrum data acquisition under complex lighting; Complementary power supply of flexible solar cells and vibration energy harvesters adapts to curved surface installation and no-light scenes, and realizes full-area deployment freedom; Charge state monitoring links battery charging and discharging control, solves the failure risk of single solar power supply in continuous rainy environment; Synchronous operation of photon computing and high-frame-rate imaging in full-function mode meets the demand of precise environmental analysis; In low-power mode, the high-power module is turned off and the acquisition frequency is reduced, which maintains the basic spectral monitoring function when the power supply is interrupted, and avoids complete equipment downtime. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings described herein are used to provide further understanding of the present application, form a part of the present application, and the illustrative embodiments of the present application and their descriptions are used to explain the present application, and do not constitute improper limitations on the present application. In the drawings: Figure 1 A flowchart of a control method of a multi-spectrum photon sieve self-powered camera provided by the embodiments of the present application is shown; Figure 2 An overall structure schematic diagram provided by the embodiments of the present application is shown; Figure 3 A multi-spectrum imaging and photon computing light path schematic diagram provided by the embodiments of the present application is shown; Figure 4 An adaptive optical correction module hardware structure schematic diagram provided by the embodiments of the present application is shown; Figure 5This is a schematic diagram of the circuit topology of the self-powered and energy management system provided in the embodiments of this application; Figure 6 This is a schematic diagram of dynamic power management state switching provided in an embodiment of this application; Figure 7 This is a schematic diagram of the internal structure of a control device for a multispectral photon sieve self-powered camera, provided in an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0020] Figure 1 This is a flowchart illustrating a control method for a multispectral photon sieve self-powered camera provided in an embodiment of this application.

[0021] The analysis method involved in the embodiments of this application can be implemented by a terminal device or a server, and this application does not impose any special limitations on it. For ease of understanding and description, the following embodiments are all described in detail using a server as an example.

[0022] It should be noted that the server can be a single device or a system composed of multiple devices, i.e., a distributed server. This application does not make any specific limitations on this.

[0023] like Figure 1 As shown in the embodiment of this application, a control method for a multispectral photon sieve self-powered camera includes: Step 101: Multi-band spectral separation of incident light is performed through a nanostructured filter layer to obtain optical signals of different wavelengths, and the separated optical signals are transmitted to a photonic in-memory computing chip for optical domain feature extraction.

[0024] Figure 2 This is a schematic diagram of the overall structure provided for an embodiment of this application. Figure 2As shown, the overall structure of the present application includes a main control module, an optical correction module, a flexible solar cell, and a vibration energy harvester. The main control module includes a battery, a power supply port, a charge management chip, an energy management chip, a main controller, an FPGA, a pixel sensing unit, a multi-spectral imaging unit, and a MEMS driving circuit. Among them, the multi-spectral imaging unit includes a photon storage and calculation integrated chip and a filter layer. The optical correction module includes a light splitting prism, a MEMS micro-mirror, a lens, and a housing.

[0025] In the present embodiment, the implementation of the nano-structured filter layer adopts an advanced nano-processing technology. Specifically, the periodic nano-grating structure is precisely manufactured on the inside surface of the front-end optical glass of the lens through electron beam lithography technology. This direct integration design avoids the assembly errors and optical path deviation problems caused by traditional external filters. For example, the grating period parameters are designed according to the spectral characteristics of the target detection material, such as optimizing the transmission efficiency of specific infrared wavebands for smoke detection.

[0026] It can be understood that the strict coaxial alignment of the nano-grating structure with the optical center of the lens is the key to the present embodiment. It should be noted that this process is completed through a high-precision optical alignment system, ensuring that the incident light passes vertically through the nano-structure, thereby obtaining the best spectral selection performance. Specifically, in practical applications, this design enables the system to simultaneously maintain excellent optical imaging quality and precise spectral separation capability.

[0027] The wavelength selection mechanism of the nano-grating is based on the physical diffraction principle. When the incident light passes through the periodic nano-structure, the light of specific wavelengths is enhanced and transmitted due to the phase matching condition, while the rest of the wavelengths are suppressed. This physical filtering method has a narrower passband width and higher wavelength resolution compared to traditional absorption-type filters. In actual environmental monitoring scenarios, this feature enables the system to clearly distinguish the characteristic absorption peaks of different substances, such as accurately identifying the characteristic spectra of harmful gases such as formaldehyde and benzene series.

[0028] In the present embodiment, the multi-spectral imaging unit is configured with a nano-structured filter layer, which realizes continuous spectral acquisition in the 400-1700nm waveband through specific nano-size and pitch design, can selectively filter different wavelength light signals, and obtain the spectral characteristics of environmental substances such as smoke and mold.

[0029] In the present embodiment, the integration of the photon storage and calculation integrated chip adopts an innovative three-dimensional packaging technology. Specifically, the photon storage and calculation integrated chip is interconnected with the pins of the CMOS image sensor through flip-chip technology, and this face-down mounting method significantly shortens the signal transmission path. For example, the interconnection process is completed in a clean room environment, and gold bump bonding technology is used to ensure the reliability and long-term stability of the connection.

[0030] It can be understood that the optical coupling between the output end of the nano-structured filtering layer and the input end of the photonic storage and calculation integrated chip adopts a custom-designed fiber array. It should be noted that the array is composed of hundreds of single-mode optical fibers arranged regularly, and high-precision FC / PC connectors are used at both ends to fix them, ensuring that the optical signal transmission loss is minimized. Specifically, in the smart home application scenario, this design enables the system to process information from multiple spectral channels simultaneously, enabling real-time environmental material identification.

[0031] The core of photonic computing lies in the optical processing mechanism of multi-beam interference. When the separated multi-wavelength optical signals enter the photonic storage and calculation integrated chip, controllable interference is generated through the carefully designed waveguide network. For example, the spatial frequency distribution of the interference pattern directly reflects the spectral characteristics of the measured material. This light domain processing method completely avoids the information loss and delay caused by photoelectric conversion in traditional schemes. In actual operation, the system matches the real-time interference characteristics with the pre-stored spectral database, such as identifying the presence of PM2.5 particles in the air and their concentration level through specific fringe patterns.

[0032] It can be understood that the entire light domain feature extraction process realizes hardware-level parallel processing. It should be noted that compared to the traditional serial processing flow of first collection and then calculation, this method of directly completing feature extraction in the light domain greatly improves the system response speed. Specifically, in the security monitoring application, this feature enables the system to complete suspicious gas detection and alarm within milliseconds, gaining valuable time for emergency response.

[0033] Figure 3 The multi-spectral imaging and photonic computing optical path schematic diagram provided for the embodiments of the present application. As shown in Figure 3 The incident light is vertically incident to the lens, and the MEMS micromirror rotation control causes the incident light to be incident to the light splitting prism, so that 90% of the incident light is transmitted to the imaging light path by the light splitting prism, and 10% of the incident light is reflected to the aberration sensing unit for wavefront analysis via the pixel detection circuit. The filtering layer of the multi-spectral imaging unit is made of a grating structure with a period of 200-500 nm through photolithography process, and is directly nested inside the front optical glass of the lens, coaxially aligned with the optical center of the lens to ensure perpendicular incidence of light. The photonic storage and calculation integrated chip is integrated at the hardware level with the multi-spectral imaging unit, and utilizes the high-speed parallel characteristics of photons to directly complete spectral feature extraction in the light domain. Data calculation is achieved through optical signal interference and diffraction, with a processing speed 10 times faster than traditional electronic chips, significantly reducing transmission delay and ensuring real-time monitoring. Pin bonding of the photonic storage and calculation integrated chip and the multi-spectral imaging unit image sensor is achieved through flip-chip technology, and the chip light input port and the filtering layer light output port are connected by a fiber array, with FC / PC connectors fixed at both ends of the fiber to ensure that the optical signal transmission loss is less than 3 dB.

[0034] Step 102, reflect the incident light to the wavefront sensor through the beam splitter prism for wavefront distortion detection, and calculate the micro-mirror adjustment parameters based on the wavefront distortion data to drive the micro-mirror actuator to dynamically compensate for aberration.

[0035] In this embodiment, the wavefront sensor is rigidly connected to the beam splitter prism through a specially designed metal bracket, ensuring strict coaxiality of the reflected light path. For example, this mechanical design takes into account the difference in material expansion coefficient caused by temperature changes, avoiding the problem of light path deviation caused by environmental temperature fluctuations.

[0036] It can be understood that the selection of LVDS high-speed interface is based on the real-time requirement of aberration detection. It should be noted that this low-voltage differential signal transmission method can effectively suppress electromagnetic interference, ensuring that the microsecond-level dynamic aberration data collected by the wavefront sensor can be transmitted to the FPGA processing unit without loss. Specifically, in the complex electromagnetic environment of smart home, this design ensures the stable operation of the aberration compensation system.

[0037] The installation position of the MEMS micro-mirror is carefully designed to be in the parallel light path between the lens and the beam splitter prism. For example, the micro-mirror is fixed by a ceramic base, and its drive circuit is directly connected to the DAC output of the FPGA, forming a forward channel for closed-loop control. It can be understood that this layout not only ensures the effectiveness of optical compensation, but also minimizes the system size, adapting to the demand for miniaturization of the camera.

[0038] In this embodiment, wavefront distortion detection and compensation form a complete control closed loop. Specifically, the reflection ratio of the beam splitter prism is optimized to ensure the sensitivity of aberration detection without affecting the light flux of the main imaging light path. For example, when the camera is facing a strong light source, the micro-lens array will divide the reflected light into a regular spot array. By analyzing the deformation and displacement of these spots, the wavefront distortion can be accurately reconstructed.

[0039] It can be understood that the aberration processing algorithm is implemented in hardware acceleration in FPGA. It should be noted that this algorithm can decompose common aberration modes such as defocus, astigmatism, and coma in real time, and convert them into driving parameters for the MEMS micro-mirror. Specifically, in the backlight scene, the system can quickly identify the wavefront distortion characteristics caused by strong light and complete compensation within milliseconds.

[0040] The driving control of the MEMS micro-mirror uses a high-precision analog amplification circuit. For example, the digital signal output by the DAC is converted into a driving voltage by a professional operational amplifier circuit, accurately controlling the deflection angle of the micro-mirror in a two-dimensional plane. It can be understood that this electrostatic driving method has the advantages of fast response speed and low power consumption, and is particularly suitable for home monitoring scenarios that require frequent adjustments.

[0041] In this embodiment, the aberration compensation system employs an iteratively optimized control strategy. Specifically, after each micromirror adjustment, the system allows sufficient stabilization time before performing a second detection. For example, this design avoids interference from mechanical vibration and electrical noise on the detection results, thus improving compensation accuracy.

[0042] Understandably, the secondary detection mechanism constitutes a key component of the closed-loop control. It should be noted that by comparing the wavefront detection results of two consecutive measurements, the system can determine the compensation effect and decide whether further adjustments are needed. Specifically, under continuously changing lighting conditions, this adaptive mechanism can track environmental changes and maintain optimal imaging quality.

[0043] The compensation termination condition is based on a preset wavefront tolerance standard. For example, when residual aberrations are detected to be below a set threshold, the system will stop adjusting and enter a monitoring state. Understandably, this intelligent control strategy ensures effective compensation while avoiding unnecessary energy consumption and extending equipment lifespan.

[0044] Figure 4 This is a schematic diagram of the hardware structure of the adaptive optics correction module provided in an embodiment of this application. Figure 4 As shown, a beam splitter reflects 10% of the light to a Shack-Hartmann wavefront sensor for wavefront distortion detection, and the detected wavefront distortion data is transmitted via LVDS to an FPGA. The FPGA sends digital control signals to the MEMS drive circuit, causing the MEMS drive circuit to output an adjustable voltage to the MEMS micromirror. Real-time angle compensation, such as ±5° range / 0.0025° measurement, is performed through closed-loop feedback control. The Shack-Hartmann wavefront sensor is coaxially mounted with the beam splitter's reflected light path, and its output is connected to the FPGA's high-speed input pin via an LVDS interface. The electrostatically driven MEMS micromirror is fixed to the optical path between the lens and the beam splitter via a ceramic substrate, and its drive circuit is connected to the FPGA's DAC output, with a drive voltage adjustment accuracy of 0.1V. During debugging, aberration threshold parameters are pre-written into the FPGA. When the wavefront sensor detects that the wavefront distortion exceeds 0.5λ (λ=632.8nm), the MEMS micromirror adjustment is automatically triggered with an adjustment step size of 0.01. After each adjustment, a second detection is performed at an interval of 50μs until the distortion is less than 0.1λ.

[0045] Step 103: Collect ambient energy using flexible solar cells and a vibration energy harvester, and convert the ambient energy into electrical energy.

[0046] In this embodiment, the design of the flexible solar cell fully considers the shape adaptability of the device and the energy harvesting efficiency. Specifically, the solar cell made of thin-film photovoltaic technology has excellent flexibility and lightness, which can perfectly fit the curved structure of the camera shell. For example, this fitting installation not only maintains the aesthetics of the device, but also maximizes the use of the surface space of the shell to receive environmental light energy. It can be understood that compared with traditional rigid solar panels, this flexible design is more suitable for various non-planar installation positions in the home environment.

[0047] It should be noted that the output end of the solar cell is connected to the charge management chip through an anti-reverse connection circuit. Specifically, this circuit uses a Schottky diode as the core component, which has a low forward voltage drop feature that ensures energy transmission efficiency and effectively prevents reverse current loss at night or in low light conditions. In practical applications, this design significantly improves the overall energy utilization rate of the solar system, especially in indoor environments where light conditions are unstable.

[0048] The installation position and structural design of the vibration energy harvester have been carefully optimized. For example, the piezoelectric ceramic sheet is installed on the shock-absorbing bracket at the bottom of the camera through a silicone pad. This design not only ensures the effective transmission of vibration energy, but also avoids the influence of mechanical resonance on imaging quality. It can be understood that the special structure of the shock-absorbing bracket can filter high-frequency mechanical noise while allowing low-frequency vibrations common in home environments, such as appliance operation and personnel movement, thereby improving energy harvesting efficiency.

[0049] Specifically, the vibration energy conversion circuit adopts a high-efficiency full-bridge rectification design. It should be noted that this circuit can convert the alternating current signal generated by the piezoelectric ceramic into a stable direct current output, and its conversion efficiency is significantly improved compared to the half-wave rectification scheme. In actual home scenarios, this design enables the system to continuously harvest energy from various environmental vibrations, forming a perfect complementary relationship with the solar system.

[0050] The parallel connection of the two energy sources adopts an intelligent switching control strategy. For example, the charge management chip has a maximum power point tracking algorithm built-in, which can automatically optimize the output matching of the two energy sources according to environmental conditions. It can be understood that this design ensures that the system can obtain the maximum available energy in any environment, such as using solar energy during the day and automatically switching to vibration energy mode at night.

[0051] It should be noted that the parallel connection rather than the series connection allows the two energy sources to work independently without interfering with each other. In actual operation, when one energy source temporarily fails due to environmental changes, the other energy source can immediately take over the power supply task, ensuring the continuous and stable operation of the device. This redundant design greatly improves the reliability of the system in a variable environment.

[0052] In a typical home environment, the device obtains its primary energy source during the day from solar panels, supplemented by a vibration energy harvester. At night, when solar power fails, the vibration energy harvester automatically becomes the main power source. Understandably, this adaptive operating mechanism allows the device to cope with various complex environmental conditions, completely eliminating the dependence on a fixed power source found in traditional cameras.

[0053] It should be noted that low-loss designs are employed at every stage of the entire process, from energy harvesting and conversion to storage, maximizing energy utilization efficiency. In practical applications, this high-efficiency energy management chip enables the device to maintain basic functionality for several days, even under adverse conditions such as continuous rain.

[0054] Figure 5 This is a schematic diagram of the circuit topology of a self-powered and energy management system provided in an embodiment of this application. Figure 5 As shown, the flexible solar cell is designed to match the shape of the top of the camera housing and is attached to the housing surface with conductive adhesive. A Schottky diode is connected in series at its output terminal to prevent reverse leakage, and it is then connected to the input terminal of the charging management chip via wires. The vibration energy harvester uses a piezoelectric ceramic sheet, which is attached to the vibration-damping bracket at the bottom of the camera with a silicone pad. Its output terminal is converted to DC power by a full-bridge rectifier circuit and then connected in parallel with the output terminal of the solar cell to the charging management chip. The battery is connected to the output terminal of the charging management chip via a PH2.0 interface and a filter circuit, and is also connected to the input pin of the energy management chip. The energy management chip includes MEMS drivers, FPGAs or sensors, and a photonic computing chip core.

[0055] In addition, this application is also compatible with the design of an external adapter for power supply, so that power can be supplied through the adapter when neither the flexible solar cell nor the vibration energy harvester can provide power.

[0056] Step 104: Store electrical energy in the battery, monitor the corresponding battery power status, and dynamically switch the camera's working mode according to the power status; the working modes include full-function mode and low-power mode.

[0057] Figure 6 This is a schematic diagram illustrating the dynamic power management state switching provided in an embodiment of this application. Figure 6As shown, the application uses I2C communication to real-time return voltage and current data, and the host controller obtains real-time sampling values through the voltage / current detection circuit to control the working mode of the camera. For example, when the power is less than 20%, the camera is switched to a low-power mode, and the host controller is feedback controlled, and the host controller uses the GPIO pin to turn off the energy management chip such as DC / DC corresponding to the photonic computing integrated chip. When the power is greater than 80%, the camera is switched to a full-function mode, i.e., a normal working mode, and the charging management chip is turned off according to the feedback control. It should be noted that when the camera is powered by an adapter, the power value is not distinguished.

[0058] In this embodiment, the battery power monitoring uses an intelligent sensing network based on an I2C bus. Specifically, the energy management chip is connected with the host controller through a standard I2C interface to real-time collect multi-dimensional parameters such as voltage, current and temperature of the battery. For example, this digital bus communication method has stronger anti-interference ability than the traditional analog detection, and is particularly suitable for scenes where there are various electrical electromagnetic interferences in the home environment. It can be understood that the energy management chip is built-in with an advanced battery modeling algorithm, which can dynamically correct the power estimation result according to the historical charging and discharging data, avoiding the precision decline problem of the traditional voltage detection method after the battery aging.

[0059] It should be noted that the system sets multiple power threshold values to trigger different working modes. Specifically, these threshold values are not fixed values, but are dynamically adjusted according to the actual health status of the battery and the environmental temperature. In actual application, this adaptive threshold management strategy significantly prolongs the service life of the battery, while ensuring the continuous running ability of the device under extreme conditions.

[0060] When the system detects that the power is higher than the first threshold value, it will automatically enter a full-function working mode. For example, in this mode, the photonic computing integrated chip and the multi-spectral imaging unit work cooperatively to realize the maximum performance of the environmental sensing ability. It can be understood that at this time, the system can simultaneously process the information of multiple spectral channels, and analyze the characteristics of various substances in the environment in real time through the parallel computing ability of the photonic computing integrated chip.

[0061] Specifically, the energy distribution strategy in the full-function mode is carefully optimized. It should be noted that the system will dynamically adjust the power supply priority of each module according to the importance of the current task, for example, temporarily increasing the operation frequency of the photonic computing integrated chip when suspicious substances are detected. This intelligent resource management method ensures the key functions while minimizing the overall energy consumption. In the actual home security scene, the full-function mode can provide the most comprehensive environmental monitoring capability, including air quality analysis, hazardous substance detection and high-definition video monitoring, and other comprehensive functions.

[0062] When the power is lower than the second threshold, the system will start the hierarchical energy-saving mechanism. For example, first, the power supply of the photonic computing chip is turned off by the hardware-level control signal, and the shutdown of this high-energy-consuming module can immediately save a large amount of power. It can be understood that this hardware-level power-off has a more thorough energy-saving effect than the software standby mode, completely eliminating static power consumption.

[0063] It should be noted that the multispectral imaging unit still maintains basic working ability in the low-power mode. Specifically, the system gradually reduces the sampling frequency according to the preset rules, and selectively closes part of the non-critical spectral channels. In actual application, this progressive frequency reduction strategy not only ensures the continuity of the basic monitoring function, but also realizes the linear reduction of energy consumption. For example, in continuous rainy weather, the system may only maintain the lowest frequency sampling of a single visible light channel, reducing the energy consumption to a very small proportion of the full-function mode.

[0064] The working mode switching adopts a hardware-software collaborative design architecture. For example, the energy management chip triggers mode switching through a hardware interrupt signal, ensuring real-time response, while the host controller smoothly transitions the state of each functional module through a software algorithm, avoiding the impact of power fluctuations on system stability. It can be understood that this hybrid switching mechanism takes into account the dual requirements of fast response and stable operation.

[0065] Specifically, the system presets a complete set of configuration parameters for each working mode. It should be noted that these parameters include key operating parameters such as the power supply voltage, clock frequency, sampling rate, etc. of each functional module, ensuring that the system immediately enters the optimal working state after mode switching. In actual deployment, this design enables the device to intelligently adapt to various energy conditions, maximizing the battery life while ensuring basic functionality.

[0066] In this embodiment, a perfect abnormal situation handling process is also designed. For example, when an abnormal situation such as a sharp drop in power is detected, an emergency protection program is immediately started to preferentially save critical data and orderly close unnecessary functions. It can be understood that this safety mechanism effectively prevents system damage or data loss caused by sudden power failure.

[0067] It should be noted that the system will record relevant data of each mode switching and abnormal event. Specifically, these operation logs include switching time, power state, environmental temperature, etc. information, providing data support for subsequent energy management optimization. In actual maintenance, technicians can further optimize the energy consumption configuration and threshold settings of the system by analyzing these historical data.

[0068] In the embodiment, the battery is connected with the output end of the charge management chip through the PH2.0 interface, and is connected with the input pin of the energy management chip. The energy management chip communicates with the host controller through the I2C bus, and monitors the battery voltage and current in real time. When the voltage is greater than 4.2V (the power is greater than 80%), the control signal is output to turn off the charge management chip. When the voltage is less than 3.0V (the power is less than 20%), a low-power instruction is sent to the host controller, and the host controller turns off the power supply (supply voltage 1.8V) of the photon storage and calculation integrated chip through the GPIO pin, and only the low-power mode (working current <10mA) of the multi-spectral imaging unit is reserved, at this time the spectral acquisition frame rate is reduced to 1fps.

[0069] The above is the method embodiment of the present application. Based on the same inventive concept, the present application also provides a control device of a multi-spectral photon sieve self-powered camera, which has the structure as shown in Figure 7 .

[0070] Figure 7 The internal structure of a control device of a multi-spectral photon sieve self-powered camera provided by the present application is shown in Figure 7 . The device includes: at least one processor; and a memory in communication connection with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to: perform multi-band spectral separation on the incident light through the nano-structured filter layer, obtain light signals of different wavelengths, and transmit the separated light signals to the photon storage and calculation integrated chip for light domain feature extraction; reflect the incident light to the wavefront sensor through the light splitting prism for wavefront distortion detection, and calculate the micro-mirror adjustment parameter based on the wavefront distortion data to drive the micro-mirror actuator to dynamically compensate the aberration; collect environmental energy through the flexible solar cell and the vibration energy harvester, and convert the environmental energy into electrical energy; store the electrical energy to the battery, monitor the corresponding power state of the battery, and dynamically switch the working mode of the camera according to the power state; the working mode includes a full-function mode and a low-power mode.

[0071] The present application also provides a non-volatile computer storage medium, which stores computer executable instructions, and the computer executable instructions can be executed to: perform multi-band spectral separation on the incident light through the nano-structured filter layer, obtain light signals of different wavelengths, and transmit the separated light signals to the photon storage and calculation integrated chip for light domain feature extraction; The incident light is reflected to a wavefront sensor through a spectrometer prism to detect wavefront distortion, and micro-mirror adjustment parameters are calculated based on the wavefront distortion data to drive the micro-mirror actuator to dynamically compensate for aberration. The environmental energy is collected by the flexible solar cell and the vibration energy collector, and is converted into electric energy. The electric energy is stored in the battery, the corresponding state of charge of the battery is monitored, and the working mode of the camera is dynamically switched according to the state of charge; the working mode includes a full-function mode and a low-power mode.

[0072] The embodiments in the present application are described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other. Each embodiment mainly describes the difference from other embodiments. Especially, the device and medium embodiments are described simply because they are basically similar to the method embodiments, and the relevant parts can be referred to the description of the method embodiments.

[0073] The device and medium provided by the embodiments of the present application correspond to the method, and therefore, the device and medium also have the similar beneficial technical effects as the method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the device and medium will not be described here.

[0074] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0075] The present application is described with reference to flowcharts and / or block diagrams of the method, device (system), and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the computer or other programmable data processing devices produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one flow or multiple flows and / or blocks Figure 1 The device that implements the functions specified in one block or multiple blocks.

[0076] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0077] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0078] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0079] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) and / or cache memory, non-volatile memory, such as read-only memory (ROM), EPROM, and / or flash memory, etc. The memory is an example of computer readable media.

[0080] Computer readable media includes permanent and non-permanent, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD), or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer readable media does not include transitory media, such as modulated data signals and carrier waves.

[0081] It should also be noted that the terms "comprising", "comprises", "including", "includes" or any other variation thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a... " does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0082] The above embodiments of the present application are only used to illustrate the technical solutions of the present application, and not intended to limit the present application. Although the present application is described in detail, it should not be seen as limited to the technical solutions described above. It will be apparent that many modifications and changes can be made by one of ordinary skill in the art without departing from the spirit and scope of the application. Any modification, equivalent replacement, improvement, and the like made within the spirit and principle of the present application shall fall into the scope of claims of the present application.

Claims

1. A method of controlling a multispectral photonic sieve self-powered camera, characterized in that, The method comprises: Multi-band spectral separation of incident light by a nano-structured filtering layer to obtain light signals of different wavelengths, and transmitting the separated light signals to a photonic storage and calculation integrated chip for light domain feature extraction; Reflecting the incident light to a wavefront sensor through a light splitting prism for wavefront distortion detection, and calculating micro-mirror adjustment parameters based on the wavefront distortion data to drive the micro-mirror actuator to dynamically compensate for aberration; Collecting environmental energy through a flexible solar cell and a vibration energy harvester, and converting the environmental energy into electrical energy; Storing the electrical energy in a storage battery, monitoring the corresponding state of charge of the storage battery, and dynamically switching the working mode of the camera according to the state of charge; the working mode includes full function mode and low power consumption mode.

2. The method of claim 1, wherein the method further comprises: Multi-band spectral separation of incident light by a nano-structured filtering layer to obtain light signals of different wavelengths, specifically including: Periodic nanometer grating structure is made inside the front-end optical glass of the lens, and the periodic nanometer grating structure is coaxially aligned with the optical center of the lens, so that the incident light passes through the periodic nanometer grating structure vertically, and wavelength selective transmission is generated; Filtering light signals of different wavelengths to match the characteristic absorption spectrum of the target substance.

3. The method of claim 1, wherein the method further comprises: Transmitting the separated light signals to a photonic storage and calculation integrated chip for light domain feature extraction, specifically including: Pin bonding of the photonic storage and calculation integrated chip and the multispectral imaging unit image sensor is performed through flip chip technology, so that the light input port of the photonic storage and calculation integrated chip is connected with the light output port of the nano-structured filtering layer through a fiber array; The separated light signals are transmitted to the photonic storage and calculation integrated chip through the fiber array, and an interference pattern is generated through multi-beam interference; The spatial frequency distribution of the interference pattern is analyzed to extract the interference pattern features, and the interference pattern features are compared with a pre-set substance spectrum database to obtain corresponding light domain features.

4. The method of claim 1, wherein the method further comprises: Before reflecting the incident light to the wavefront sensor through the light splitting prism for wavefront distortion detection, the method further comprises: The wavefront sensor is coaxially installed with the light splitting prism reflection light path, and the output end of the wavefront sensor is connected with the FPGA high-speed input pin through the LVDS interface; The light splitting prism is fixed between the lens and the wavefront sensor through a support; The MEMS micro-mirror is fixed between the lens and the light splitting prism, and the driving circuit of the MEMS micro-mirror is connected with the DAC output end of the FPGA.

5. The method of claim 1, wherein the method further comprises: Reflecting the incident light to the wavefront sensor through the light splitting prism for wavefront distortion detection, and calculating micro-mirror adjustment parameters based on the wavefront distortion data to drive the micro-mirror actuator to dynamically compensate for aberration, specifically including: Reflecting a preset proportion of incident light to the aberration sensing unit through the light splitting prism, and using a microlens array to form a spot array to capture wavefront distortion; The detected aberration data is transmitted to the processing unit, the micro-mirror adjustment parameters are calculated, and the digital analog converter output signal is amplified to a driving voltage to control the MEMS micro-mirror to deflect in a two-dimensional plane to realize dynamic compensation of aberration.

6. The method of claim 5, wherein the method further comprises: The method further comprises: Wait for a preset time interval after the MEMS micro mirror is adjusted, and perform secondary detection on the compensated wavefront aberration; According to the secondary detection result, it is judged whether the compensated wavefront aberration needs further adjustment, if yes, the aberration dynamic compensation is repeatedly performed on the compensated wavefront aberration until the wavefront aberration is less than a preset tolerance value.

7. The method of claim 1, wherein the method further comprises: The environmental energy is collected by the flexible solar cell and the vibration energy collector, and the environmental energy is converted into electric energy, specifically including: The flexible solar cell is attached to the surface of the camera shell to capture light energy, and the flexible solar cell is connected to the charge management chip through the anti-reverse connection circuit; The vibration energy collector is installed on the shock-absorbing support at the bottom of the camera to capture vibration energy, and the vibration energy is converted into direct current through the rectifier circuit; The outputs of the two energy sources are connected in parallel to the charge management chip to uniformly manage the outputs of the two energy sources through the charge management chip.

8. The method of claim 1, wherein the method further comprises: The power state of the battery is monitored, and the working mode of the camera is dynamically switched according to the power state, specifically including: The voltage and current parameters of the battery are monitored in real time, and the photonic storage and calculation integrated chip and the multispectral imaging unit are kept working simultaneously when it is detected that the power is higher than the first threshold value; In the case that the power is lower than the second threshold value, a low-power instruction is sent to the main controller to close the photonic storage and calculation integrated chip, and the working frequency of the multispectral imaging unit is reduced according to a preset rule.

9. A control device for a multispectral photonic sieve self-powered camera, characterized in that, The device comprises: at least one processor; and a memory connected in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a control method of a multispectral photonic sieve self-powered camera according to any one of claims 1-8.

10. A non-transitory computer storage medium storing computer-executable instructions that, when executed, cause a computer to perform: The computer executable instructions are executed to implement a control method of a multispectral photonic sieve self-powered camera according to any one of claims 1-8.

Images