A monitoring method of a low-power-consumption radar camera and related equipment

Abstract

The application relates to the technical field of low-power-consumption radar cameras, and discloses a monitoring method of a low-power-consumption radar camera, wherein the method comprises the following steps: when a millimeter wave radar detects a moving target, radar target coordinate data is acquired, and an image sensor ready flag is generated according to the radar target coordinate data; a frame synchronization signal is sent according to the image sensor ready flag to collect an original image, image processing and video encoding are performed on the original image, a target image sequence and a target video code stream are obtained; effective bounding box data is calculated based on the target image sequence and the radar target coordinate data; a target crossing state is determined according to the effective bounding box data, a cloud cover is driven to track, and after the target video code stream and alarm metadata are uploaded, a deep sleep state is entered; the method filters shaking interference and small-size target false triggering, the image sensor and the encoder are only woken up when an effective target is detected, and then the average power consumption is reduced.

Description

Technical Field

[0001] This invention relates to the field of low-power radar camera technology, and in particular to a monitoring method and related equipment for a low-power radar camera. Background Technology

[0002] Existing security monitoring methods use passive infrared sensors to trigger recording based on motion detection. However, this approach relies on temperature differences between the target and the environment. In high-temperature environments or when the target is stationary and makes slight movements, the detection sensitivity decreases. Furthermore, it is susceptible to light fluctuations and interference from small animals, resulting in numerous false alarms. This leads to users receiving frequent invalid alarm messages, severely impacting the user experience.

[0003] Some smart cameras employ continuous visual analysis for target recognition, which reduces false alarm rates but requires the image sensor and processing chip to operate continuously, typically consuming several watts of power. When powered by batteries, the battery life is only a few hours to a few days, failing to meet the long-term unattended application requirements in wireless deployment scenarios. Frequent charging and maintenance increase operating costs. Existing PTZ tracking cameras generally suffer from frequent left-right shaking when the target approaches the center of the frame. Furthermore, the lack of dynamic compensation for PTZ angle after electronic zoom is enabled leads to decreased tracking accuracy or even target loss, affecting the capture of critical footage and recording quality. Summary of the Invention

[0004] This invention provides a monitoring method and related equipment for a low-power radar camera. This invention filters out jitter interference and false triggering by small targets, and only wakes up the image sensor and encoder when a valid target is detected, thereby reducing the average power consumption.

[0005] In a first aspect, embodiments of this application provide a monitoring method for a low-power radar camera, comprising: When the millimeter-wave radar detects a moving target, it acquires the radar target coordinate data and generates an image sensor ready flag based on the radar target coordinate data. The original image is acquired by sending a frame synchronization signal according to the image sensor readiness flag, and image processing and video encoding are performed on the original image to obtain the target image sequence and the target video stream; Calculate effective bounding box data based on the target image sequence and the radar target coordinate data; The target crossing status is determined based on the effective bounding box data, and the gimbal is driven to track the target. After uploading the target video stream and alarm metadata, the target enters a deep sleep state.

[0006] In conjunction with the first aspect, in a first implementation of the first aspect of the present invention, before the millimeter-wave radar detects a moving target, the method further includes: Read the configuration parameters, including the operating frequency, transmit power, scan period, and detection distance; Write the configuration parameters to the first register, with the first GPIO pin as the input direction. Call the configuration function to set the interrupt trigger type to high-level trigger and register the callback function, then call the interrupt enable function to activate the interrupt. Write sleep control bit combinations to the third, fourth, and fifth GPIO pins to sequentially disable the first, second, fifth, and sixth power domains. Write the rising edge trigger configuration of the sixth GPIO pin to the interrupt controller to enter a deep sleep state.

[0007] In conjunction with the first aspect, in a second implementation of the first aspect of the present invention, when the millimeter-wave radar detects a moving target, radar target coordinate data is acquired, and an image sensor ready flag is generated based on the radar target coordinate data, including: After the first GPIO pin goes high, triggering the callback function, the interrupt disable function is called and the semaphore is released to wake up the configuration thread. The first read function is called repeatedly to monitor the pin level. If a low level is detected, the loop is exited. Calculate the frequency domain spectrum of the radar echo signal, and calculate the radar target coordinate data based on the frequency domain spectrum; Based on the radar target coordinate data, a pulse width modulation waveform is output to the second GPIO pin to drive the bridge circuit to control the filter switching. The third and fourth power domains are powered on in sequence to generate an image sensor ready flag.

[0008] In conjunction with the first aspect, in a third implementation of the first aspect of the present invention, calculating the frequency domain spectral lines of the radar echo signal and calculating radar target coordinate data based on the frequency domain spectral lines includes: The radar echo signal is mixed and converted from analog to digital to obtain a time-domain sequence. Perform a fast Fourier transform on the time-domain sequence to obtain frequency-domain spectral lines, and traverse the frequency-domain spectral lines to extract distance gates whose reflection intensity exceeds an intensity threshold as candidate targets; The radial velocity of the candidate targets is calculated, and targets whose radial velocity exceeds the velocity threshold are filtered out. The azimuth and elevation angles are calculated using the phase difference of the multi-channel antenna array. The range, azimuth, elevation angles and radar cross section are combined to obtain the radar target coordinate data.

[0009] In conjunction with the first aspect, in a fourth implementation of the first aspect of the present invention, a pulse width modulation waveform is output to the second GPIO pin based on the radar target coordinate data to drive the bridge circuit to control the filter switching, and the third power domain and the fourth power domain are sequentially powered on to generate an image sensor ready flag, including: The illumination mode is determined based on the radar cross-section in the radar target coordinate data. According to the illumination mode, a pulse width modulation waveform is output to the second GPIO pin to control the bridge circuit to apply a pulse current to the electromagnetic coil to drive the filter switching; The third and fourth power domains are started sequentially, and resolution, frame rate and exposure parameters are sent to the image sensor to generate an image sensor ready flag.

[0010] In conjunction with the first aspect, in a fifth implementation of the first aspect of the present invention, acquiring an original image by sending a frame synchronization signal according to the image sensor readiness flag, and performing image processing and video encoding on the original image to obtain a target image sequence and a target video stream, includes: Send a frame synchronization signal according to the image sensor ready flag and acquire raw images in Bayer format; Black level correction is performed on the original image to obtain a three-channel color image, and lens shading correction, white balance processing, and noise reduction processing are performed on the three-channel color image to obtain the target image sequence; The target image sequence is divided into coding units, and video encoding is performed on each coding unit to obtain the target video stream.

[0011] In conjunction with the first aspect, in a sixth implementation of the first aspect of the present invention, calculating effective bounding box data based on the target image sequence and the radar target coordinate data includes: A background model is established based on the target image sequence, and a bounding box is calculated based on the background model; Based on the radar target coordinate data, the polar coordinates are converted to Cartesian coordinates, and the Cartesian coordinates are projected onto the pixel plane according to the focal length parameter and the principal point coordinates to obtain the projected coordinates; Calculate the Euclidean distance between the projected coordinates and the center coordinates of the bounding box, and calculate the diagonal length of the bounding box; Valid bounding box data are filtered based on the Euclidean distance and the diagonal length.

[0012] In conjunction with the first aspect, in the seventh implementation of the first aspect of the present invention, after determining the target crossing state based on the effective bounding box data and driving the gimbal to track it, and uploading the target video stream and alarm metadata, the system enters a deep sleep state, including: Based on the effective bounding box data, determine whether the target is located in the central region. If it is not in the central region, compare the sign bits by bitwise AND operation using the sign bit mask. If the sign bits are not equal, return the execution flag; if the sign bits are equal, return the filter flag. When the execution flag is returned, the second register is read to obtain the angle compensation coefficient, and the gimbal motor is driven according to the angle compensation coefficient; The target video stream and the alarm metadata are uploaded to the server. When the level of the first GPIO pin is continuously interrupted to a low level for a preset number of times, the wireless module and encoder are turned off, and the third power domain and the fourth power domain are turned off. A sleep control bit combination is written to the third GPIO pin, the fourth GPIO pin and the fifth GPIO pin to enter a deep sleep state.

[0013] In conjunction with the first aspect, in the eighth implementation of the first aspect of the present invention, when the execution flag is returned, the second register is read to obtain the angle compensation coefficient, and the gimbal motor is driven according to the angle compensation coefficient, including: When the execution flag is returned, the preset address of the second register is read to obtain the current cropping width, current cropping height, maximum available width and maximum available height, and the area zoom factor is calculated. The square root of the area zoom factor is used to obtain the angle compensation coefficient, and the original horizontal rotation angle and the original vertical rotation angle are calculated. The gimbal motor is driven according to the original horizontal rotation angle, the original vertical rotation angle, and the angle compensation coefficient.

[0014] In conjunction with the first aspect, in the ninth implementation of the first aspect of the present invention, the target video stream and the alarm metadata are uploaded to the server. When the level of the first GPIO pin is continuously interrupted to a low level for a preset number of times, the wireless module and encoder are turned off, and the third power domain and the fourth power domain are turned off. A sleep control bit combination is written to the third GPIO pin, the fourth GPIO pin, and the fifth GPIO pin to enter a deep sleep state, including: The target video stream and the alarm metadata are uploaded to the server, a timer is configured to generate an interrupt at a preset period, and the level of the first GPIO pin is read in the interrupt service routine; When the level of the first GPIO pin is low for a preset number of consecutive interrupts, a shutdown instruction is written to the wireless module control register, a stop instruction and a clear instruction are written to the video encoder control register to stop encoding and clear the buffer, a standby instruction is sent through the serial interface to make the image sensor enter a low-power mode, the third power domain and the fourth power domain are turned off in sequence, and a sleep control bit combination is written to the third GPIO pin, the fourth GPIO pin and the fifth GPIO pin to enter a deep sleep state.

[0015] Secondly, embodiments of this application provide a monitoring device for a low-power radar camera, comprising: The detection module is used to acquire radar target coordinate data when the millimeter-wave radar detects a moving target, and generate an image sensor ready flag based on the radar target coordinate data. The processing module is used to acquire raw images by sending a frame synchronization signal according to the image sensor readiness flag, and to perform image processing and video encoding on the raw images to obtain a target image sequence and a target video stream; The boundary calculation module is used to calculate effective bounding box data based on the target image sequence and the radar target coordinate data; The tracking module is used to determine the target crossing status based on the effective bounding box data and drive the gimbal to track the target. After uploading the target video stream and alarm metadata, it enters a deep sleep state.

[0016] Thirdly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the monitoring method of the low-power radar camera described above.

[0017] Fourthly, embodiments of this application provide a readable storage medium storing a computer program that, when executed by a processor, implements the steps of the monitoring method of the low-power radar camera described above.

[0018] Fifthly, embodiments of this application provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the monitoring method for the low-power radar camera described above.

[0019] In one solution provided by the aforementioned low-power radar camera monitoring method and related equipment, millimeter-wave radar active detection combined with a GPIO high-level interrupt triggering mechanism achieves accurate motion detection unaffected by ambient temperature and lighting conditions. Combined with 2000ms continuous high-level anti-shake verification and radar cross-section threshold judgment, it effectively filters out jitter interference and false triggering by small targets. The image sensor and encoder are only awakened when a valid target is detected, reducing the average power consumption of the method to the milliwatt level. By adaptively controlling the switching of infrared filters by reading the output value of the ambient light sensor, the filter is removed at night to improve infrared sensitivity and achieve low-light full-color imaging, while the filter is inserted during the day to ensure accurate color reproduction. This achieves concealed full-color night vision without the need for visible light supplementary lighting. The fusion verification is performed by projecting the radar target's three-dimensional coordinates onto the pixel plane and calculating the Euclidean distance with the center of the visual bounding box. Combined with continuous frame displacement stability judgment, a dual-core mechanism of radar and vision is implemented, significantly reducing the false alarm rate. By determining whether the target offset is located in the center region of one-sixth of the image width and height, and using a bitwise AND operation on the sign bit to determine the sign change of the offset, the gimbal avoids repeated swaying when the target is close to the center of the image. For the electronic zoom function, the angle compensation coefficient is obtained by reading the ISP crop configuration register, calculating the ratio of the cropped area to the maximum area, and then taking the square root. This dynamically compensates for the gimbal angle, solving the problem of decreased tracking accuracy caused by changes in the visible angle after zooming. After an alarm event ends, this invention shuts down each power domain in a preset order and writes the sleep control bit combination to achieve a rapid return to deep sleep state. Combined with battery power monitoring and low power protection mechanisms, this ensures an ultra-long battery life under wireless deployment conditions. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of the monitoring system of a low-power radar camera in one embodiment of the present invention; Figure 2 This is a flowchart illustrating a monitoring method using a low-power radar camera according to an embodiment of the present invention. Figure 3 This is a circuit diagram of an image sensor control module in one embodiment of the present invention; Figure 4 This is a circuit diagram of an infrared cutoff filter driving circuit in one embodiment of the present invention; Figure 5 This is a diagram of an analog power supply voltage regulator circuit in one embodiment of the present invention; Figure 6 This is a circuit diagram of an SDIO interface according to an embodiment of the present invention; Figure 7 This is a circuit diagram of a 30-pin main connector in one embodiment of the present invention; Figure 8 This is a schematic diagram of the structure of a monitoring device for a low-power radar camera according to an embodiment of the present invention; Figure 9 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof. It should also be understood that, as used in this specification and the appended claims, the term "and / or" refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0024] Furthermore, in the description of this invention and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0025] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0026] It should be understood that the sequence number of each step in the following embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0027] To illustrate the technical solution of the present invention, specific embodiments are described below.

[0028] To address the problems mentioned above in the background art, this application proposes a monitoring method and related equipment for a low-power radar camera. The monitoring method for a low-power radar camera provided by this invention can be applied to applications such as... Figure 1 The monitoring system for the low-power radar camera shown includes a client and a server.

[0029] In one embodiment, such as Figure 2 As shown, a monitoring method for a low-power radar camera is provided, which is then applied to... Figure 1 Taking a low-power radar camera monitoring system as an example, the following steps are included: 101. When the millimeter-wave radar detects a moving target, it acquires the radar target coordinate data and generates an image sensor ready flag based on the radar target coordinate data. 102. Acquire the original image by sending a frame synchronization signal according to the image sensor ready flag, and perform image processing and video encoding on the original image to obtain the target image sequence and the target video stream; 103. Calculate effective bounding box data based on target image sequences and radar target coordinate data; 104. Determine the target crossing status based on the valid bounding box data and drive the PTZ to track it. After uploading the target video stream and alarm metadata, enter deep sleep mode.

[0030] In this embodiment, millimeter-wave radar active detection combined with a GPIO high-level interrupt triggering mechanism achieves accurate motion detection unaffected by ambient temperature and lighting conditions. Combined with 2000ms continuous high-level anti-shake verification and radar cross-section threshold judgment, it effectively filters jitter interference and false triggering by small targets. The image sensor and encoder are only awakened when a valid target is detected, reducing the average power consumption of the method to the milliwatt level. By adaptively controlling the switching of infrared filters by reading the output value of the ambient light sensor, the filter is removed at night to improve infrared sensitivity and achieve low-light full-color imaging, while the filter is inserted during the day to ensure accurate color reproduction. A concealed full-color night vision effect can be obtained without visible light supplementary lighting. The fusion verification is performed by projecting the radar target's three-dimensional coordinates onto the pixel plane and calculating the Euclidean distance with the center of the visual bounding box. Combined with continuous frame displacement stability judgment, a dual-core mechanism of radar and vision is implemented, significantly reducing the false alarm rate. By determining whether the target offset is located in the center region of one-sixth of the image width and height, and using a bitwise AND operation to determine the sign of the offset, the gimbal avoids repeated oscillations when the target approaches the center of the image. For the electronic zoom function, the angle compensation coefficient is obtained by calculating the ratio of the clipped area to the maximum area by reading the ISP clipping configuration register and then taking the square root. This dynamically compensates for the gimbal angle, solving the problem of decreased tracking accuracy caused by changes in the visible angle after zooming. After an alarm event ends, this invention shuts down each power domain in a preset order and writes the sleep control bit combination, achieving a rapid return to deep sleep state. Combined with battery power monitoring and low battery protection mechanisms, this ensures an ultra-long battery life under wireless deployment conditions.

[0031] Figure 3This is the circuit schematic for the image sensor control module. SS2004 is the image sensor control chip. A1, A7, and C7 are the analog power input pins AVDD_1, AVDD_2, and AVDD_3; B5, C1, and E4 are the digital power input pins DVDD_1, DVDD_2, and DVDD_3; D4 and E2 are the digital I / O power pins DOVDD_1 and DOVDD_2; and D1, E1, F2, and D3 are the data output pins. <1> To D <5> E3, E4, E5, F4, F5, D5, and D8 are the MIPI differential output pins MIPI_D1_N, MIPI_D1_P, MIPI_CLK_N, MIPI_CLK_P, MIPI_D0_P, and MIPI_D0_N. A5 is the chip select (SID) pin. B1 and B7 are the analog ground pins AGND_1 and AGND_2. C3, D2, and E6 are the digital ground pins DOGND_1 to DOGND_4. C2 is the sensor master clock pin SENSOR_MCLK. A3... B3 is the sensor serial clock SENSOR_SCL pin, B4 is the sensor serial data SENSOR_SDA pin, C5 is the sensor power supply SENSOR_PWDN pin, C6 is the external clock EXICLK pin, E7 and F7 are the reference voltage VREFH and VREFN pins, B2 is the frame synchronization FSYNC pin, B6 is the line synchronization EFSYNC pin, C6, A6, D7, and A4 are unused NC pins, C1 to C4 are 100nF decoupling capacitors, and D9 and AC are the diode and resistor connection points.

[0032] In one specific embodiment, before the millimeter-wave radar detects a moving target, the method further includes: Read the configuration parameters, including the operating frequency, transmit power, scan period, and detection distance; Write configuration parameters to the first register, with the first GPIO pin as the input direction. Call the configuration function to set the interrupt trigger type to high-level trigger and register the callback function, then call the interrupt enable function to activate the interrupt. Write sleep control bit combinations to the third, fourth, and fifth GPIO pins to sequentially disable the first, second, fifth, and sixth power domains. Write the rising edge trigger configuration of the sixth GPIO pin to the interrupt controller to enter a deep sleep state.

[0033] Specifically, the main control chip completes the access operation of the configuration storage area during the system initialization process. After power-on, the main control chip loads the firmware and calls the SPI read function to extract system configuration parameters from the external memory, including the millimeter-wave radar's operating frequency (e.g., 77GHz), transmit power (e.g., -10dBm to +10dBm), radar scan cycle (e.g., 200ms), and target detection distance threshold (e.g., 0.5m to 20m). After parsing the data structure of this set of parameters, it is written into the configuration register, i.e., the first register, used for the radar module to initialize the working mode of the radar transmitter and signal processor. The main control chip configures the GPIO pin of the interrupt input channel (designated as the first GPIO pin) as the input direction, calls the pin function setting function to ensure the pin is in GPIO mode, and calls the input mode configuration function to specify the pin's direction attribute as input. After completing the direction configuration, it calls the interrupt trigger setting function to set the interrupt response type of the first GPIO pin to high-level trigger mode, registers the user-defined callback handling function to the interrupt service scheduler, and calls the interrupt enable function to activate the pin's interrupt capability. This allows the interrupt process to be triggered and the main control system to be woken up when an external device (such as a radar module) detects a moving target and pulls the level high to the pin. The main control chip outputs specific level combinations to the third, fourth, and fifth GPIO pins as control signals to the power management chip, driving it to sequentially shut down the corresponding power domains, including the first power domain powering the image sensor, the second power domain powering the video encoder, the fifth power domain powering the AI ​​processor, and the sixth power domain powering the MIPI CSI communication interface. The power management chip performs the power domain shutdown operation at 10-microsecond intervals according to the control instructions to ensure smooth power fluctuations and prevent electrical interference. The main control chip writes an interrupt configuration command to the interrupt controller, configuring the sixth GPIO pin to rising edge trigger mode. This sixth GPIO pin is connected to the target detection output port of the radar module, allowing the rising edge signal output by the radar after target recognition to be captured by the interrupt controller and trigger system wake-up. After all the above processes are completed, the main control chip sets the sleep flag and enters deep sleep mode via the WFI instruction. At this time, only the radar module, real-time clock circuit, interrupt controller, and power detection circuit remain active; all other modules are powered down.

[0034] In one specific embodiment, the process of performing step 101 may specifically include the following steps: After the first GPIO pin goes high, the callback function is called to disable the interrupt and release the semaphore to wake up the configuration thread. The first read function is called repeatedly to monitor the pin level. If a low level is detected, the loop is exited. Calculate the frequency domain spectrum of the radar echo signal, and calculate the radar target coordinate data based on the frequency domain spectrum; Based on the radar target coordinate data, a pulse width modulation waveform is output to the second GPIO pin to drive the bridge circuit to control the filter switching. The third and fourth power domains are powered on in sequence to generate the image sensor ready flag.

[0035] Specifically, when the millimeter-wave radar module detects a moving target that meets the conditions, its output terminal generates a high-level signal and applies it to the first GPIO pin. At this time, the change in the pin level is detected by the interrupt controller and triggers a callback function. Inside the callback function, the interrupt disable function is called to block subsequent interrupt responses from the first GPIO pin. At the same time, the semaphore release function is called to set the synchronization semaphore on which the configuration thread depends to an available state, so that the configuration thread is awakened from the waiting state and enters the execution flow. After the configuration thread is awakened, it enters a loop structure and continuously calls the first read function, namely the gpio_input_value function, to monitor the level state of the first GPIO pin in real time in a polling manner. If the first GPIO pin is detected to change from a high level to a low level within a preset delay period, it is determined that the previous high level was a valid pulse trigger signal and not level interference, and the loop is exited to enter the next stage of processing; otherwise, if the level remains low, it indicates that the trigger signal is noise or electrical glitches, and the system re-enters the waiting state to ensure that the system does not execute redundant processes due to false triggering. The main control chip reads the intermediate frequency (IF) sampling sequence from the radar module's cache. This IF sampling sequence is output from a mixer and digitized by an ADC, obtaining thousands of equally spaced time-domain echo samples within each sampling period. The main control chip then calls its internal digital signal processor module to perform a fast Fourier transform on the time-domain sampling sequence, converting the original time-domain signal into frequency-domain spectra. Each spectral line corresponds to the reflection intensity at different range gates. Doppler frequency shift information is extracted by combining the phase change between adjacent frames to determine the target's radial velocity. Based on the frequency-domain characteristics and the phase difference of the radar array antenna, angle estimation is performed to obtain the target's azimuth and elevation angles. These range, azimuth, and elevation angles are then used to construct a three-dimensional coordinate structure representing the target's position in space. The main control chip uses this target coordinate structure to determine if a low-light environment is required, i.e., whether image acquisition is in night mode. If so, it performs an infrared filter switching operation. The main control chip outputs a preset pulse width modulation waveform to the second GPIO pin. The PWM signal drives the optocoupler input through a current-limiting resistor, thereby controlling the bridge circuit composed of four MOSFETs to switch its conduction direction. This, in turn, controls the electromagnetic coil of the infrared cut-off filter connected to the front of the lens, applying magnetic force through a short reverse pulse to insert or remove the filter in the lens optical axis. Simultaneously, the Hall element in the bridge circuit detects the magnetic field response and feeds back the filter's position status to the main control chip. After the filter switching is complete, the main control chip sequentially outputs enable control signals to the third and fourth power domains. The power management unit controls the power chip to supply power to the analog power supply AVDD and digital power supply DVDD modules of the image sensor according to the power-on sequence. The power-on interval between each power domain is 100 microseconds to ensure power stability and interference-free power-on. After both power domains have been powered on and the power supply stability has been confirmed by the status detection pin, the main control chip sets the image sensor ready flag to be valid.

[0036] After repeatedly calling the first read function to monitor the pin level, the process also includes: obtaining the current system timestamp and reading the last alarm timestamp from the third register via the bus; calculating the time difference between the current system timestamp and the last alarm timestamp; comparing the time difference with a preset blocking time threshold; if the time difference is less than the preset blocking time threshold, marking this trigger as a blocking state; calling the interrupt enable function to reactivate the first GPIO pin interrupt; writing a sleep control bit combination to the third, fourth, and fifth GPIO pins to return to a deep sleep state; if the time difference is greater than or equal to the preset blocking time threshold, writing the current system timestamp to the third register to update the last alarm timestamp; and continuing to execute the step of calculating the frequency domain spectrum of the radar echo signal.

[0037] In one specific embodiment, the process of calculating the frequency domain spectral lines of the radar echo signal and calculating the radar target coordinate data based on the frequency domain spectral lines can specifically include the following steps: The radar echo signal is mixed and converted from analog to digital to obtain a time-domain sequence. Perform a fast Fourier transform on the time-domain sequence to obtain the frequency-domain spectrum, and traverse the frequency-domain spectrum to extract the distance gates whose reflection intensity exceeds the intensity threshold as candidate targets; The radial velocity of candidate targets is calculated and targets with radial velocities exceeding the velocity threshold are filtered out. The azimuth and elevation angles are calculated using the phase difference of a multi-channel antenna array. The range, azimuth, elevation angles, and radar cross section are combined to obtain radar target coordinate data.

[0038] Specifically, in the millimeter-wave radar module, a transmitter generates a linearly frequency-modulated continuous wave signal. This signal undergoes linear ramp modulation with a set center frequency (e.g., 77 GHz) and modulation bandwidth (e.g., 500 MHz), and is repeatedly transmitted at a preset scan period. When an object is encountered, the signal is reflected at different distances. The reflected echo signal is then received by a receiving antenna and input to a mixer along with a local reference signal. The mixer performs frequency difference calculations on the two signals to output an intermediate frequency (IF) signal. The frequency value of the IF signal is proportional to the distance to the target. The IF signal is amplified by a low-noise amplifier to improve the signal-to-noise ratio, and then input to an analog-to-digital converter (ADC) for high-speed sampling, for example, using a sampling rate of 100 kHz or higher to convert the continuous analog waveform into a discrete digital time-domain sequence. The time-domain data represents the amplitude variation of the target echo over time. The main control chip schedules the digital signal processing module to perform a fast Fourier transform on the time-domain sampled sequence, transforming the original time-domain signal into frequency-domain spectra. Each spectral line corresponds to a range cell, i.e., a range gate, and the amplitude reflects the reflection intensity at the range gate. The main control chip traverses all frequency domain spectral data, comparing the reflection intensity corresponding to each range gate with an intensity threshold. If the reflection intensity is higher than a preset intensity threshold, such as -60dBm, the range gate is marked as a candidate target. For a candidate target, the phase change of its echo signal is extracted over multiple consecutive scanning cycles. The Doppler frequency shift is calculated by combining the phase difference between adjacent cycles with the frequency modulation parameters, and the radial velocity of the target in the radar beam direction is obtained. If the absolute value of the radial velocity exceeds the velocity threshold, such as 0.1 m / s, it is determined to be a valid moving target. In the angle calculation stage, the radar module analyzes the phase difference of the reflected wave arriving at each channel through a multi-channel receiving antenna array. Combining the array structure and signal wavelength, the azimuth and elevation angles of the target are solved using methods such as direction cosine algorithm or MUSIC spectrum estimation, obtaining the target's direction information in the radar coordinate system. Simultaneously, the radar cross section (RCS) is calculated based on the target's echo intensity, received power, target range, and radar transmission parameters. This index reflects the target's ability to reflect radar waves and indirectly estimates the target's size or material. The main control chip assembles the target's range, azimuth, and elevation values ​​with the corresponding RCS values ​​to construct a target coordinate data structure, thereby characterizing the target's position and radar characteristics in three-dimensional space.

[0039] Figure 4This is the schematic diagram of the infrared cut-off filter driver circuit. U6 is the optocoupler isolation chip LK116C, IRCUT_IN is the infrared filter control input signal, IRCUT_P is the infrared filter positive control signal, R35 and R38 are input current limiting resistors with a resistance of 100K, MAIN_VCC is the main power input, VCC is the power input pins 3 and 5 of the optocoupler chip, GND is the ground connection pins 2 and 6, IN1 is the optocoupler input pin 3, OUT1 is the optocoupler output pin 1, IN2 is the optocoupler input pin 4, OUT2 is the optocoupler output pin 6, AOUT and BOUT are analog output connection points, J1 is the output connector pins 1 and 2, and C44 is the power supply decoupling capacitor with a capacitance of 100nF.

[0040] In one specific embodiment, the process of executing the steps of outputting a pulse width modulation waveform to the second GPIO pin based on radar target coordinate data to drive the bridge circuit to control the filter switching, and sequentially powering on the third and fourth power domains to generate an image sensor ready flag can specifically include the following steps: The illumination pattern is determined based on the radar cross-section in the radar target coordinate data. According to the illumination mode, the pulse width modulation waveform is output to the second GPIO pin to control the bridge circuit to apply a pulse current to the electromagnetic coil to drive the filter switching; The third and fourth power domains are started sequentially, and resolution, frame rate and exposure parameters are sent to the image sensor to generate an image sensor ready flag.

[0041] Specifically, after the main control chip receives the target coordinate data reported by the radar module, it analyzes the radar cross-section (RCS) field. The RCS field reflects the effective reflective cross-section size of the target, which is related to the target's volume and material. The main control chip compares the RCS value with a preset scattering area threshold. For example, when the RCS value is greater than 0.5 square meters, the target is considered to have sufficient identification value, the image acquisition link is initiated, and the lighting mode judgment logic is triggered. The main control chip reads the current illumination value from the ambient light intensity sensor via the I2C bus. The sensor outputs an illumination intensity value covering the visible light range of 380nm to 780nm and compares it with a preset day / night switching threshold (e.g., 10 Lux). If the illumination is lower than the threshold, it is determined to be night mode; otherwise, it is day mode. After completing the lighting mode judgment, the main control chip outputs the corresponding pulse width modulation (PWM) waveform to the second GPIO pin. The PWM signal drives the optocoupler after passing through a current-limiting resistor, turning on the optocoupler control terminal and controlling the bridge drive circuit composed of four MOSFETs through its output to apply control current to the electromagnetic coil of the infrared cut-off filter located in front of the camera. Based on the duty cycle and duration of the PWM signal, the bridge circuit applies a positive or reverse pulse current to the electromagnetic coil, causing a change in the coil's magnetic field to drive the filter to insert or remove. Simultaneously, a Hall sensor in the bridge circuit detects the current position of the filter and feeds it back to the main control chip, ensuring the operation is completed and preventing misalignment. Once the filter's state stabilizes, the main control chip sequentially activates the power domains required for image acquisition according to its power management strategy. First, it outputs a high-level enable signal to the third power domain control pin to activate the analog power supply unit (e.g., AVDD) of the image sensor. After the voltage stabilizes, it outputs an enable signal to the fourth power domain control pin to activate the digital power supply unit (e.g., DVDD). A power-on interval of at least 100 microseconds is maintained between each power domain, and a 10-microsecond voltage stabilization observation period is configured to prevent power interference or power-on glitches from causing false sensor activation. After power is supplied, the main control chip sends an initialization configuration command to the image sensor through the MIPI CSI control channel. This includes setting the resolution to 2560×1440 pixels, the frame rate to 30 frames per second, the exposure mode to automatic exposure, the white balance to automatic white balance, and the gain parameter range to 1 to 32 times. After the configuration command is transmitted, the image sensor's internal state machine completes the parameter loading and switches to the ready state. The main control chip reads the sensor's status register to confirm that the device is in working condition and generates an image sensor ready flag.

[0042] The illumination mode is determined based on the radar cross-section (RCS) in the radar target coordinate data. This includes: extracting the RCS from the radar target coordinate data and comparing it sequentially with a first area threshold and a second area threshold. If the RCS is less than the first area threshold, it is classified as a small target; the interrupt enable function is called to reactivate the first GPIO pin interrupt and write a sleep control bit combination to the third, fourth, and fifth GPIO pins, returning to deep sleep without activating the image sensor. If the RCS is greater than or equal to the first area threshold and less than the second area threshold, it is classified as a medium target; the digital output value of the ambient light sensor is read via the bus, and the digital output value is compared with the illumination threshold to determine the illumination mode. The sensor configuration parameters are set to half the standard resolution and the frame rate is the standard frame rate. The exposure time range and gain range are set to half of the standard exposure time limit and half of the standard gain limit, respectively, according to the lighting mode. If the radar cross-section is greater than or equal to the second area threshold, it is determined to be a large target. The digital output value of the ambient light sensor is read through the bus and compared with the illuminance threshold to determine the lighting mode. The resolution and frame rate in the sensor configuration parameters are set to the standard resolution and the frame rate to the standard frame rate. The exposure time range and gain range are set to the standard exposure time limit and the standard gain limit, respectively, according to the lighting mode. The pulse width modulation waveform is output to the second GPIO pin to control the filter switching according to the lighting mode. If the lighting mode is night mode, the filter is removed and the upper limit of the gain range is increased to twice the standard gain limit. If the lighting mode is day mode, the filter is inserted and the upper limit of the gain range is limited to half of the standard gain limit.

[0043] In one specific embodiment, the process of performing step 102 may specifically include the following steps: Send frame synchronization signals based on the image sensor readiness flag and acquire raw images in Bayer format; Black level correction is performed on the original image to obtain a three-channel color image. Lens shadow correction, white balance processing, and noise reduction processing are then performed on the three-channel color image to obtain the target image sequence. The target image sequence is divided into coding units, and video encoding is performed on each coding unit to obtain the target video bitstream.

[0044] Specifically, after the main control chip detects that the image sensor's ready flag has been set, it sends a frame synchronization signal to the image sensor via the MIPI CSI interface, instructing the image sensor to enter image acquisition mode and initiate the progressive readout process. Inside the sensor, the image unit array outputs data in Bayer mode, meaning each pixel location records the intensity value of a single color channel, typically arranged in RGGB format. The image sensor controls the integration time of each pixel according to the automatic exposure parameters. After exposure, the charge generated by the photodiode is amplified by a column amplifier and readout noise is suppressed by a related dual-sampling circuit before being sent to the on-chip ADC to convert it into a 12-bit digital signal, outputting a raw image frame in Bayer format. Upon receiving the raw image frame, the main control chip's image signal processor performs black level correction, subtracting the black level value measured under completely dark conditions from all pixel signals to remove dark current noise and fixed offset errors. It then uses a de-mosaic interpolation algorithm to restore the single-channel Bayer data into a complete three-channel RGB image, for example, using a 5×5 adaptive interpolation kernel to balance color sharpness and edge transition. After obtaining the RGB image, lens shadow correction is performed. This involves gain compensation for the brightness at the image edges based on a lens center-to-edge brightness attenuation characteristic model to correct non-uniform illumination caused by lens optical design. Next, the white balance processing flow begins. The main control chip analyzes the overall color temperature characteristics of the image, calculates the ratio difference between the average brightness of the red and blue channels and the green channel, and dynamically adjusts the gain coefficients of the RGB three channels based on the calculation results to make the image closer to natural colors and correct color casts under different light source conditions. After white balance is completed, the image noise reduction stage begins, employing a three-dimensional noise reduction strategy. This involves simultaneous modeling in the temporal and spatial domains, predicting the pixel change trend of the same target across multiple frames, using inter-frame averaging to reduce noise in static areas, and employing bilateral filters to maintain edge sharpness while suppressing abrupt noise in dynamic areas, thus reducing image graininess while maintaining sharpness. The resulting image is the target image sequence. Each frame is organized in YUV420 format, where Y represents the luminance component and U and V represent the chrominance components. The data is arranged chronologically in an internal buffer. The main control chip divides each frame of the target image into multiple coding tree units, such as 64×64 pixel block regions, and recursively subdivides them into coding units as small as 8×8 pixels based on content complexity. During the encoding process, intra-frame prediction or inter-frame prediction is performed for each coding unit. Intra-frame prediction estimates pixel structure using 35 directional modes, while inter-frame prediction uses motion estimation to find matching blocks in the reference frame and records motion vectors and residuals. The residual signal is quantized after integer discrete cosine transform processing and then compressed using CABAC arithmetic coding mode to form an efficient and compact video bitstream.The encoder controls the output rate according to a set constant bitrate strategy, for example, a target bitrate of 2 Mbps. It dynamically adjusts the quantization parameters to keep the output bitrate fluctuation between frames within ±10%. All encoded image frames are combined to form a standardized H.265 target video stream.

[0045] In one specific embodiment, the process of performing step 103 may specifically include the following steps: A background model is built based on the target image sequence, and bounding boxes are calculated based on the background model; Based on radar target coordinate data, polar coordinates are converted to Cartesian coordinates, and the Cartesian coordinates are projected onto the pixel plane according to the focal length parameter and principal point coordinates to obtain the projected coordinates; Calculate the Euclidean distance between the projected coordinates and the center coordinates of the bounding box, and calculate the diagonal length of the bounding box; Valid bounding box data are filtered based on Euclidean distance and diagonal length.

[0046] Specifically, the main control chip constructs a temporal image background model based on the brightness channel data from the target image sequence continuously acquired by the image sensor. This temporal image background model is constructed using cumulative frame averaging, Gaussian mixture modeling, or an adaptive update strategy to characterize the expected grayscale value distribution of static pixels in the scene. A foreground mask is calculated in each frame using pixel-level differences. The foreground mask reveals regions in the current frame that significantly deviate from the background model, indicating the pixel distribution of moving objects. The main control chip performs morphological operations on the foreground mask, such as opening and dilation operations, to remove isolated noise and fill holes within connected regions, forming a regularized target contour. In the processed foreground image, boundary analysis is performed on all connected components to extract the minimum bounding rectangle as the bounding box of the candidate target, and the center coordinates, width, and height of the bounding box are recorded. After the initial bounding box extraction at the image end, the main control chip performs cross-modal projection verification using 3D target coordinate data from the millimeter-wave radar module. The radar provides coordinates in polar coordinates, including target range, azimuth, and elevation. The main control chip converts these polar coordinates into Cartesian three-dimensional coordinates. The conversion process is as follows: x-coordinate equals range multiplied by cosine azimuth multiplied by cosine elevation; y-coordinate equals range multiplied by sine azimuth multiplied by cosine elevation; z-coordinate equals range multiplied by sine elevation. The resulting (x, y, z) three-dimensional point describes the target's spatial position in the radar coordinate system. The main control chip projects these three-dimensional Cartesian coordinates onto the image pixel plane based on the camera's intrinsic parameter matrix. The projection formula is: pixel horizontal coordinate u equals focal length fx multiplied by x divided by z plus principal point horizontal coordinate cx; pixel vertical coordinate v equals focal length fy multiplied by y divided by z plus principal point vertical coordinate cy. The projected coordinates (u, v) are the theoretical corresponding position of the radar target on the image plane. The main control chip extracts the center point coordinates from all candidate bounding boxes and calculates the Euclidean distance between the center point and the radar projected coordinates (u, v). Simultaneously, the diagonal length of each bounding box is calculated. The diagonal length is equal to the square root of the sum of the square of the width and the square of the height of the bounding box. A filtering logic is applied to all bounding boxes. If the Euclidean distance *d* is less than 30% of the diagonal length, and the area of ​​the bounding box meets a minimum pixel area threshold (e.g., greater than 2000 square pixels), then the bounding box is considered geometrically consistent with the radar target, is determined as a valid bounding box, and is marked as a confirmed target. If the above conditions are not met, the bounding box is excluded as a false alarm and discarded.

[0047] After filtering valid bounding box data based on Euclidean distance and diagonal length, the process further includes: reading the bounding box center coordinate sequence of N consecutive frames from the tracking buffer; calculating the horizontal and vertical displacement components of the bounding box center coordinates between adjacent frames; combining the horizontal and vertical displacement components to obtain an inter-frame displacement vector sequence; performing a weighted average calculation on the inter-frame displacement vector sequence to obtain a motion trend vector; adding the motion trend vector to the current frame bounding box center coordinates to obtain the predicted target position coordinates for the next frame; obtaining the actual bounding box center coordinates for the next frame; calculating the position deviation between the predicted target position coordinates and the actual bounding box center coordinates; if the position deviation exceeds a preset deviation threshold, it is determined to be a sudden change in the target motion state, the tracking buffer is cleared and the background model is re-established; if the position deviation is less than or equal to the preset deviation threshold, the actual bounding box center coordinates are appended to the tracking buffer to continue the subsequent steps.

[0048] In one specific embodiment, the process of performing step 104 may specifically include the following steps: Determine whether the target is located in the central region based on the valid bounding box data. If it is not in the central region, compare the sign bits by bitwise AND operation using the sign bit mask. If the sign bits are not equal, return the execution flag; if the sign bits are equal, return the filter flag. When the execution flag is returned, the second register is read to obtain the angle compensation coefficient, and the gimbal motor is driven according to the angle compensation coefficient; The target video stream and alarm metadata are uploaded to the server. When the level of the first GPIO pin is low for a preset number of consecutive interruptions, the wireless module and encoder are turned off, and the third and fourth power domains are turned off. The sleep control bit combination is written to the third, fourth, and fifth GPIO pins to enter a deep sleep state.

[0049] Specifically, after parsing the valid bounding box data, the main control chip extracts the center coordinates of the bounding box, i.e., the horizontal and vertical coordinates of the bounding box center. Combined with the width and height of the current image frame from the image sensor, it calculates the reference coordinates of the image center. Then, it calculates the target's offset relative to the image center, obtaining the horizontal and vertical offsets. It checks if the absolute value of the horizontal offset is less than one-sixth of the image width and if the absolute value of the vertical offset is less than one-sixth of the image height. If both conditions are met, the target is within the image center area, and a filtering flag is returned, without performing gimbal adjustment. If either condition is not met, the target deviates from the image center area, and the sign direction judgment logic continues. The main control chip compares the offset calculated in the current frame with the historical offset recorded in the previous frame. To avoid floating-point comparison errors, the sign of the offset value is extracted using a sign bitmask. A bitwise AND operation is performed between the current horizontal offset and the mask constant 0x80000000 to obtain the sign bit of the current offset value. Then, a bitwise AND operation is performed between the historical offset from the previous frame and the same mask to extract the historical sign bit. If the current sign bit is inconsistent with the historical sign bit, it indicates that the target has crossed the direction of the image center point. The historical offset needs to be reset and the execution flag returned, allowing the gimbal action to proceed. If the sign bits are consistent, it indicates that the target is still offset in the same direction. In this case, the filter flag is returned, and the system continues to observe whether the target moves closer to the center or maintains its direction. When the main control chip receives the execution flag, it enters the angle adjustment stage. It reads the angle compensation coefficient value from the second register through the register interface. The angle compensation coefficient reflects whether electronic scaling operations (image cropping and interpolation) are enabled for the current image. If enabled, subsequent angle calculations need to be proportionally amplified. Based on the proportional relationship between the horizontal offset and the total image width, combined with the field of view parameters of the camera module, the main control chip calculates the original horizontal rotation angle. If electronic scaling is detected, the angle compensation coefficient is used to amplify and correct the original rotation angle. This is then mapped to the corresponding motor speed level according to different angle ranges, generating gimbal rotation direction, rotation angle, and rotation speed control parameters. The driver program is then called to control the horizontal motor to perform rotation actions in the corresponding direction. After the PTZ rotation is completed, the main control chip extracts the target video stream segment centered on the alarm trigger time from the encoder data buffer. For example, it is a 15-second stream segment from 5 seconds before to 10 seconds after the trigger point. At the same time, it generates a structured alarm information data packet containing the target timestamp, bounding box position, identification confidence level and target type. The video stream and alarm data are then sent to the remote server through the network interface.The system enters the target status detection process, triggered by a timer with a 1-second cycle of interrupts. During each interrupt, the main control chip calls a read function to obtain the level status of the first GPIO pin and counts whether it remains low for a continuous number of interrupts. When the detection result satisfies multiple consecutive (e.g., 3 times) low levels, and no valid target exists in the current image frame, the current target event is determined to have ended, and the system enters a sleep rollback state. The main control chip controls the Wi-Fi module to shut down, writing a shutdown instruction to the control register to stop the wireless radio frequency. It then shuts down the video encoder, terminating the encoding process and clearing the buffer contents. Next, it shuts down the third and fourth power domains, stopping the analog and digital power supply to the image sensor. It sequentially writes the defined sleep control bit combination to the third, fourth, and fifth GPIO pins, guiding the power management unit to shut down the remaining operating circuits. Finally, the main control chip executes a low-power wait instruction, entering deep sleep mode, with only the real-time clock module, radar sensing circuit, and interrupt control logic remaining operational.

[0050] Figure 5 This is a schematic diagram of a simulated power supply voltage regulator circuit. U3 is an LDO linear regulator chip, HOST_3V3 is the input power supply (3.3V), AVDD is the output power supply, IN is the regulator input pin 4, OUT is the regulator output pin 1, EN is the enable control pin 3, GND is the ground pin 2, EP is the heat sink pin 5, R32 is the enable control resistor (10KΩ), C18 is the input filter capacitor (22μF / 6.3V), C20 is the input filter capacitor (100nF), C22 is the output filter capacitor (100nF), C24 is the output filter capacitor (10μF / 6.3V), and GND is the ground connection point.

[0051] Figure 6This is the schematic diagram of the SDIO interface circuit. SDIO_CMD is the SDIO command line signal connected via a 100K current-limiting resistor R24; SDIO_D0 is the SDIO data line 0 signal connected via a 100K current-limiting resistor R27; SDIO_D1_IRQ is the SDIO data line 1 interrupt signal connected via a 100K current-limiting resistor R29; SDIO_D2 is the SDIO data line 2 signal connected via a 100K current-limiting resistor R30; and SDIO_D3 is the SDIO... Data line 3 signal is connected via current-limiting resistor R31 (100KΩ). SDIO_DET is the SDIO card detection signal, connected via capacitor C18 (100nF). HOST_3V3 is the 3.3V power supply. VDDMSC0 is the memory card power supply. R25 is a pull-up resistor (0 ohms). C15 is a decoupling capacitor (100nF). C16 is a decoupling capacitor (10μF / 6.3V). C17 is a decoupling capacitor (100nF). TP6 and TP1 are test points. GND is the ground connection point.

[0052] Figure 7This is the schematic diagram of a 30-pin main connector circuit. J1 is the 30-pin connector; MAIN_VCC is the main power input pin (pin 2); HOST_3V3 is the 3.3V power input pin (pin 4); BAT+ is the battery positive terminal pin (pin 6); STA_LED2 is the status LED2 control signal pin (pin 8); STA_LED1 is the status LED1 control signal pin (pin 10); VIN_DETALED0 is the power detection LED0 signal pin (pin 12); BCUT_MAIN is the main cutoff control signal pin (pin 14); IRCUT_P is the infrared cutoff filter positive control signal pin (pin 16); SDIO_DET is the SDIO card detection signal pin (pin 18); SDIO_D1_IRQ is the SDIO data line 1 interrupt signal pin (pin 20); SDIO_D0 is the SDIO data line 0 signal pin (pin 22); SDIO_CMD is the SDIO command line signal pin (pin 24); SDIO_D2 is the SDIO data line 2 signal pin (pin 26); SDIO_D3... Pin 28 is the SDIO data line 3 signal connection pin; pin 30 is the SDIO clock signal connection pin; pin 3 is the digital power output connection pin; pin 5 is the sensor reset signal connection pin; pin 7 is the sensor I2C data line connection pin; pin 9 is the sensor I2C clock line connection pin; pin 11 is the sensor master clock connection pin; pin 15 is the MIPI differential data 0 negative signal connection pin; pin 17 is the MIPI differential data 0 positive signal connection pin; pin 21 is the MIPI differential data 1 negative signal connection pin; pin 23 is the MIPI differential data 1 positive signal connection pin; pin 27 is the MIPI differential clock negative signal connection pin; pin 29 is the MIPI differential clock positive signal connection pin; and pins 1, 13, 19, and 25 are the ground connection pins.

[0053] In one specific embodiment, the process of reading the second register to obtain the angle compensation coefficient and driving the gimbal motor according to the angle compensation coefficient when the execution flag is returned can specifically include the following steps: When the execution flag is returned, the preset address of the second register is read to obtain the current cropping width, current cropping height, maximum available width and maximum available height, and the area zoom factor is calculated. Take the square root of the area zoom factor to obtain the angle compensation coefficient, and calculate the original horizontal and vertical rotation angles. The gimbal motor is driven based on the original horizontal rotation angle, the original vertical rotation angle, and the angle compensation coefficient.

[0054] Specifically, after receiving the execution flag's judgment result, the main control chip accesses the preset address of the second register in the image signal processor via a register read instruction. The second register address range stores the cropping region parameters of the current image frame, including the current cropping width, current cropping height, and the corresponding maximum available width and maximum available height. The main control chip sequentially reads the current cropping width and height and performs multiplication and division calculations with the maximum available width and height to obtain the area zoom factor. The area zoom factor equals the maximum available width multiplied by the maximum available height, then divided by the product of the current cropping width and current cropping height. The area zoom factor reflects whether electronic scaling is enabled and the scaling ratio. The main control chip performs a square root operation on the area zoom factor to obtain the angle compensation coefficient, which represents the actual viewing angle expansion ratio corresponding to a unit pixel offset under the current scaling state. Based on the ratio between the horizontal offset of the target center point relative to the image center and the total image width, the original horizontal rotation angle is calculated using the camera's factory-set horizontal field of view parameters. This original horizontal rotation angle equals the horizontal offset divided by the image width, multiplied by the camera module's horizontal viewing angle value. Similarly, the main control chip calculates the original vertical rotation angle based on the ratio between the vertical offset and the image height, combined with the vertical viewing angle value. These two original angles represent the uncompensated viewing angle deviation in the target offset direction. The original horizontal and vertical rotation angles are multiplied by an angle compensation coefficient to obtain the compensated angles to be executed under zoom conditions. Corresponding gimbal control commands are generated according to a preset angle drive mapping table, setting the horizontal rotation direction, horizontal angle, horizontal speed parameters, and vertical control parameters. The main control chip calls the gimbal control drive function, controlling the horizontal and vertical motors to perform precise rotation operations based on the current direction settings and angle values. This ensures the image is aligned with the target position in real time and dynamically adjusts to follow the offset trend, adapting to changes in the field of view under zoom conditions.

[0055] Based on the valid bounding box data, determine whether the target is located in the center region. If not, compare the sign bit using a bitwise AND operation on the sign bit mask. This includes: extracting the center coordinates from the valid bounding box data, calculating the horizontal and vertical offsets from the image center point, and determining whether the absolute value of the horizontal offset is less than a preset center ratio of the image width and whether the absolute value of the vertical offset is less than a preset center ratio of the image height. If both conditions are met, the target is determined to be in the center region. Read the historical horizontal and vertical offsets from the fourth register, clear them, and write them back to the fourth register. Return the filtering flag to terminate subsequent gimbal rotation. If the target is not in the center region, read the historical horizontal offset from the fourth register, extract the current sign bit by performing a bitwise AND operation between the horizontal offset and the sign bit mask value, and compare the historical horizontal offset with the sign bit mask value. The sign bit mask value is bitwise ANDed to extract the historical sign bit. The current sign bit is compared with the historical sign bit. If they are not equal, it is determined that the target has crossed the center point of the image. The historical horizontal offset is cleared to zero, written back to the fourth register, and the execution flag is returned to allow the gimbal to rotate. If the current sign bit is equal to the historical sign bit, the absolute value of the horizontal offset is further compared with the absolute value of the historical horizontal offset. If the absolute value of the horizontal offset is greater than the absolute value of the historical horizontal offset, it is determined that the target is moving away from the center point. The horizontal offset is written to the fourth register to update the historical horizontal offset and the filtering flag is returned. If the absolute value of the horizontal offset is less than or equal to the absolute value of the historical horizontal offset, it is determined that the target is approaching the center point. The historical horizontal offset is kept unchanged and the filtering flag is returned. The same three-state filtering decision process is performed on the vertical offset.

[0056] In one specific embodiment, the process of uploading the target video stream and alarm metadata to the server, and when the level of the first GPIO pin is continuously low for a preset number of interrupts, shutting down the wireless module and encoder, and disabling the third and fourth power domains, and writing a sleep control bit combination to the third, fourth, and fifth GPIO pins to enter a deep sleep state, can specifically include the following steps: Upload the target video stream and alarm metadata to the server, configure a timer to generate an interrupt at a preset period, and read the level of the first GPIO pin in the interrupt service routine; When the level of the first GPIO pin is low for a preset number of consecutive interrupts, a shutdown instruction is written to the wireless module control register, a stop instruction and a clear instruction are written to the video encoder control register to stop encoding and clear the buffer, a standby instruction is sent through the serial interface to make the image sensor enter a low-power mode, the third power domain and the fourth power domain are turned off in sequence, and a sleep control bit combination is written to the third GPIO pin, the fourth GPIO pin and the fifth GPIO pin to enter a deep sleep state.

[0057] Specifically, the target video stream segment, based on the alarm event trigger time, is extracted from the data buffer within the video encoding module. This segment is then combined with elements such as timestamp, bounding box coordinates, recognition confidence level, and target type to construct structured alarm metadata. The target video stream and metadata are then uploaded to a remote server via a network interface call to the transmission module. The video stream is sent continuously in packets using data sockets, while the alarm metadata is reported via a standard application interface in a structured data format, such as key-value pair encoding. After data upload, the main control chip configures a timer module to set a fixed time period, e.g., 1 second, enabling periodic interrupt functionality and registering an interrupt service routine in the system so that the interrupt response function is automatically triggered each time the timer expires. Inside the interrupt service routine, the main control chip calls the pin read function to poll the level state of the first GPIO pin, checking if the current pin is in a low-level state and maintaining a continuous low-level count variable. This variable records the number of low-level occurrences in consecutive interrupts. If the number of interrupts during the detection process does not reach the set threshold, the system continues to operate, waiting for the next interrupt cycle to continue detection. When it is detected that the first GPIO pin has been in a low-level state for a preset number of consecutive interrupts, it indicates that the radar module has not detected any new target signals, and the current monitoring process is considered to be over, and the power consumption control stage should begin. When entering the low-power process, the main control chip writes a shutdown control command to the control register of the wireless module, completely shutting down the Wi-Fi radio frequency transmission and reception modules and disconnecting the network connection with the external server to reduce overall power consumption. Subsequently, it writes a stop encoding command to the control register of the video encoder to terminate the subsequent image compression process and writes a clear command to clear the compression buffer and reference frame buffer inside the encoder to prevent invalid data from occupying resources. The main control chip sends a standby command to the image sensor via the serial control interface, triggering the image sensor's internal state switching logic to switch it into a low-power standby mode. In this mode, the image array stops scanning, and the exposure and readout circuits are no longer powered. Only the clock and configuration registers are kept in their current state, thus reducing current consumption to below the milliampere level. The main control chip sequentially sends control commands to the power management module to shut down the third and fourth power domains, terminating the analog and digital power supply to the image sensor. After the power domains are completely shut down, a sleep control bit combination is written to the third, fourth, and fifth GPIO pins, forming a specific pin level combination signal. This signal is recognized by the power management unit as a deep sleep request, and the power supply to all non-critical modules such as image processing, data transmission, and motion control is shut down in a preset order, causing the main control chip to enter an interrupt-waiting state. The entire system enters a deep sleep mode, in which only the radar module, interrupt controller, and real-time clock module remain operational, and the overall power consumption is controlled within the milliampere range.

[0058] Before driving the gimbal motor based on the original horizontal rotation angle, original vertical rotation angle, and angle compensation coefficient, the process includes: reading the historical speed parameters and historical direction parameters of the previous gimbal rotation from the fourth register; comparing the historical direction parameters with the currently calculated direction parameters to determine if the rotation direction has changed; if the historical direction parameters and the direction parameters are not equal, it is determined that the direction has changed, the current speed parameter is forcibly set to the minimum speed value, and the gimbal control function is called through the minimum speed value to start the gimbal rotation, and the current speed parameter is increased by a preset acceleration until the target speed parameter determined based on the compensated rotation angle is reached; if the historical direction parameters and the direction parameters are equal, it is determined that the rotation is in the same direction, the angle change rate between the current frame compensated rotation angle and the previous compensated rotation angle is calculated, the acceleration parameter is determined based on the angle change rate, the target speed parameter is calculated based on the acceleration parameter and the historical speed parameters, the gimbal control function is called multiple times to gradually adjust the speed from the historical speed parameters to the target speed parameters to achieve smooth acceleration and deceleration, and the target speed parameters and direction parameters are written to the fourth register to update the historical record.

[0059] The angle compensation coefficient is obtained by taking the square root of the area zoom factor, and the original horizontal and vertical rotation angles are calculated. This includes: taking the square root of the area zoom factor to obtain a linear scaling factor as the angle compensation coefficient; reading the center position coordinates of the current cropping area, including the horizontal and vertical center positions, from the second register; reading the maximum available width and maximum available height from the second register; calculating the horizontal position coordinates of the theoretical optical center as half of the maximum available width; calculating the vertical position coordinates of the theoretical optical center as half of the maximum available height; and calculating the horizontal position relative to the theoretical optical center. The horizontal crop offset pixel value is set at the coordinates. The vertical crop offset pixel value relative to the theoretical optical center is calculated. The horizontal crop offset pixel value is divided by the current crop width and multiplied by the horizontal field of view to obtain the horizontal crop offset angle. The horizontal offset is divided by the image width and multiplied by the horizontal field of view to obtain the target offset horizontal angle. The target offset horizontal angle is calculated and subtracted from the horizontal crop offset angle to obtain the horizontal rotation angle after position correction. The horizontal rotation angle after position correction is multiplied by the angle compensation coefficient to obtain the original horizontal rotation angle. The same crop offset compensation and field of view scaling compensation process is performed in the vertical direction to obtain the original vertical rotation angle.

[0060] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0061] In one embodiment, a monitoring device for a low-power radar camera is provided, which corresponds one-to-one with the monitoring method for the low-power radar camera in the above embodiments. For example... Figure 8As shown, the monitoring device of this low-power radar camera includes: The detection module 801 is used to acquire radar target coordinate data when the millimeter-wave radar detects a moving target, and generate an image sensor ready flag based on the radar target coordinate data. The processing module 802 is used to acquire the original image by sending a frame synchronization signal according to the image sensor ready flag, and to perform image processing and video encoding on the original image to obtain the target image sequence and the target video stream; Boundary calculation module 803 is used to calculate effective bounding box data based on the target image sequence and the radar target coordinate data; The tracking module 804 is used to determine the target crossing status based on the effective bounding box data and drive the gimbal to track the target. After uploading the target video stream and alarm metadata, it enters a deep sleep state.

[0062] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0063] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0064] This application also provides a computer device, such as... Figure 9 As shown, the computer device includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor. When the processor executes the computer program, it implements the steps in any of the above method embodiments, or when the processor executes the computer program, it implements the functions of each module / unit in the above device embodiments.

[0065] For example, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the computer device.

[0066] Those skilled in the art will understand that Figure 9 The computer device described is merely an example and does not constitute a limitation on the computer device. It may include more or fewer components than shown, or combine certain components, or different components. For example, the computer device may also include input / output devices, network access devices, buses, etc.

[0067] The aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0068] The memory can be an internal storage unit of the computer device, such as a hard drive or RAM. The memory can also be an external storage device of the computer device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory can include both internal and external storage units of the computer device.

[0069] This application also provides a readable storage medium storing a computer program, which, when executed by a processor, can implement the steps in the above-described method embodiments.

[0070] This application provides a computer program product that, when run on an electronic device, enables the electronic device to implement the steps described in the various method embodiments above.

[0071] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0072] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0073] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0074] In the embodiments provided in this application, it should be understood that the disclosed apparatus / devices and methods can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0075] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0076] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A monitoring method for a low-power radar camera, characterized in that, include: When the millimeter-wave radar detects a moving target, it acquires the radar target coordinate data and outputs a pulse-width modulation waveform to the second GPIO pin based on the radar target coordinate data to drive the bridge circuit to control the filter switching. The third and fourth power domains are then powered on sequentially to generate an image sensor ready flag. Specifically, generating the image sensor ready flag includes: determining the illumination mode based on the radar cross-section in the radar target coordinate data; outputting a pulse-width modulation waveform to the second GPIO pin based on the illumination mode to control the bridge circuit to apply a pulse current to the electromagnetic coil to drive the filter switching; and sequentially activating the third and fourth power domains to send resolution, frame rate, and exposure parameters to the image sensor, thereby generating the image sensor ready flag. The original image is acquired by sending a frame synchronization signal according to the image sensor readiness flag, and image processing and video encoding are performed on the original image to obtain the target image sequence and the target video stream; Calculating valid bounding box data based on the target image sequence and the radar target coordinate data; specifically, calculating valid bounding box data includes: establishing a background model based on the target image sequence, and calculating bounding boxes based on the background model; converting polar coordinates to Cartesian coordinates based on the radar target coordinate data, and projecting the Cartesian coordinates onto the pixel plane according to the focal length parameter and principal point coordinates to obtain projected coordinates; calculating the Euclidean distance between the projected coordinates and the center coordinates of the bounding box, and calculating the diagonal length of the bounding box; filtering valid bounding box data based on the Euclidean distance and the diagonal length. The target crossing status is determined based on the effective bounding box data, and the gimbal is driven to track the target. After uploading the target video stream and alarm metadata, the target enters a deep sleep state.

2. The monitoring method of the low-power radar camera according to claim 1, characterized in that, Before the millimeter-wave radar detects a moving target, it also includes: Read the configuration parameters, including the operating frequency, transmit power, scan period, and detection distance; Write the configuration parameters to the first register, with the first GPIO pin as the input direction. Call the configuration function to set the interrupt trigger type to high-level trigger and register the callback function, then call the interrupt enable function to activate the interrupt. Write sleep control bit combinations to the third, fourth, and fifth GPIO pins to sequentially disable the first, second, fifth, and sixth power domains. Write the rising edge trigger configuration of the sixth GPIO pin to the interrupt controller to enter a deep sleep state.

3. The monitoring method of the low-power radar camera according to claim 2, characterized in that, When the millimeter-wave radar detects a moving target, it acquires the target's coordinate data, including: After the first GPIO pin goes high, triggering the callback function, the interrupt disable function is called and the semaphore is released to wake up the configuration thread. The first read function is called repeatedly to monitor the pin level. If a low level is detected, the loop is exited. Calculate the frequency domain spectrum of the radar echo signal, and calculate the radar target coordinate data based on the frequency domain spectrum.

4. The monitoring method of the low-power radar camera according to claim 3, characterized in that, Calculating the frequency domain spectral lines of the radar echo signal, and calculating radar target coordinate data based on the frequency domain spectral lines, including: The radar echo signal is mixed and converted from analog to digital to obtain a time-domain sequence. Perform a fast Fourier transform on the time-domain sequence to obtain frequency-domain spectral lines, and traverse the frequency-domain spectral lines to extract distance gates whose reflection intensity exceeds an intensity threshold as candidate targets; The radial velocity of the candidate targets is calculated, and targets whose radial velocity exceeds the velocity threshold are filtered out. The azimuth and elevation angles are calculated using the phase difference of the multi-channel antenna array. The range, azimuth, elevation angles and radar cross section are combined to obtain the radar target coordinate data.

5. The monitoring method of the low-power radar camera according to claim 4, characterized in that, Based on the image sensor readiness flag, a frame synchronization signal is sent to acquire the original image, and image processing and video encoding are performed on the original image to obtain the target image sequence and the target video stream, including: Send a frame synchronization signal according to the image sensor ready flag and acquire raw images in Bayer format; Black level correction is performed on the original image to obtain a three-channel color image, and lens shading correction, white balance processing, and noise reduction processing are performed on the three-channel color image to obtain the target image sequence; The target image sequence is divided into coding units, and video encoding is performed on each coding unit to obtain the target video stream.

6. The monitoring method of the low-power radar camera according to claim 5, characterized in that, Based on the effective bounding box data, the target crossing status is determined and the gimbal is driven to track it. After uploading the target video stream and alarm metadata, it enters a deep sleep state, including: Based on the effective bounding box data, determine whether the target is located in the central region. If it is not in the central region, compare the sign bits by bitwise AND operation using the sign bit mask. If the sign bits are not equal, return the execution flag; if the sign bits are equal, return the filter flag. When the execution flag is returned, the second register is read to obtain the angle compensation coefficient, and the gimbal motor is driven according to the angle compensation coefficient; The target video stream and the alarm metadata are uploaded to the server. When the level of the first GPIO pin is continuously interrupted to a low level for a preset number of times, the wireless module and encoder are turned off, and the third power domain and the fourth power domain are turned off. A sleep control bit combination is written to the third GPIO pin, the fourth GPIO pin and the fifth GPIO pin to enter a deep sleep state.

7. The monitoring method of the low-power radar camera according to claim 6, characterized in that, When the execution flag is returned, the second register is read to obtain the angle compensation coefficient, and the gimbal motor is driven according to the angle compensation coefficient, including: When the execution flag is returned, the preset address of the second register is read to obtain the current cropping width, current cropping height, maximum available width and maximum available height, and the area zoom factor is calculated. The square root of the area zoom factor is used to obtain the angle compensation coefficient, and the original horizontal rotation angle and the original vertical rotation angle are calculated. The gimbal motor is driven according to the original horizontal rotation angle, the original vertical rotation angle, and the angle compensation coefficient.

8. The monitoring method of the low-power radar camera according to claim 7, characterized in that, The target video stream and the alarm metadata are uploaded to the server. When the level of the first GPIO pin is continuously low for a preset number of interruptions, the wireless module and encoder are turned off, and the third and fourth power domains are turned off. A sleep control bit combination is written to the third, fourth, and fifth GPIO pins to enter a deep sleep state, including: The target video stream and the alarm metadata are uploaded to the server, a timer is configured to generate an interrupt at a preset period, and the level of the first GPIO pin is read in the interrupt service routine; When the level of the first GPIO pin is low for a preset number of consecutive interrupts, a shutdown instruction is written to the wireless module control register, a stop instruction and a clear instruction are written to the video encoder control register to stop encoding and clear the buffer, a standby instruction is sent through the serial interface to make the image sensor enter a low-power mode, the third power domain and the fourth power domain are turned off in sequence, and a sleep control bit combination is written to the third GPIO pin, the fourth GPIO pin and the fifth GPIO pin to enter a deep sleep state.

9. A monitoring device for a low-power radar camera, characterized in that, The steps for implementing the monitoring method of the low-power radar camera as described in any one of claims 1 to 8 include: The detection module is used to acquire radar target coordinate data when the millimeter-wave radar detects a moving target, and generate an image sensor ready flag based on the radar target coordinate data. The processing module is used to acquire raw images by sending a frame synchronization signal according to the image sensor readiness flag, and to perform image processing and video encoding on the raw images to obtain a target image sequence and a target video stream; The boundary calculation module is used to calculate effective bounding box data based on the target image sequence and the radar target coordinate data; The tracking module is used to determine the target crossing status based on the effective bounding box data and drive the gimbal to track the target. After uploading the target video stream and alarm metadata, it enters a deep sleep state.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the monitoring method of the low-power radar camera as described in any one of claims 1 to 8.

11. A readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the monitoring method of the low-power radar camera as described in any one of claims 1 to 8.

12. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the monitoring method of the low-power radar camera as described in any one of claims 1 to 8.

Images