AI recognition mobile detection and video recording control method

Abstract

The application relates to an AI recognition movement detection and video recording control method, and relates to the technical field of artificial intelligence, and comprises the following steps: step one, photon event collection and interframe difference sequence generation: a photon counting sensor array is used to continuously collect frame images of a monitoring scene, and interframe difference images of multiple continuous frames are arranged in time sequence to form an interframe difference sequence; step two, variation mode decomposition is used to separate light components and motion components; step three, a Poisson pulse neural network is used for processing to generate a motion transient intensity graph; and step four, block-level connected aggregation judgment and video recording control: the motion transient intensity graph obtained in step three is divided into image blocks, and block energy values are calculated, and when global motion intensity in a sliding detection window continuously falls below a preset static threshold, a video recording device is controlled to stop recording; the application realizes high accuracy and low false trigger rate of movement detection under complex light scenes such as evening dim light and night vision switching.

Description

Technical Field

[0001] This disclosure relates to, but is not limited to, the field of artificial intelligence technology, and in particular to an AI-based method for motion detection and video recording control. Background Technology

[0002] Traditional motion detection methods mainly include inter-frame differencing, background subtraction, and optical flow. Inter-frame differencing detects moving regions by calculating pixel differences between adjacent video frames. It has the advantages of simple computation and good real-time performance, but it is extremely sensitive to changes in lighting. When ambient lighting changes gradually or the automatic exposure function of the monitoring equipment adjusts parameters, it is prone to generating large areas of false motion responses. Background subtraction establishes and maintains a background model of the scene, comparing the current frame with the background model to extract foreground moving targets. Commonly used background modeling methods include Gaussian mixture models, codebook models, and visual background extractors. These methods perform well in stable indoor environments, but in outdoor scenes, due to continuous changes in natural light, light fluctuations caused by cloud cover, and the rapid decrease in light intensity at dusk, the background model is difficult to update accurately and in a timely manner, leading to a significant decrease in detection performance. Optical flow detects moving regions by estimating the motion vectors of pixels between adjacent frames, providing motion direction and velocity information. However, it has high computational complexity, making it difficult to meet real-time processing requirements, and it is sensitive to image noise.

[0003] In recent years, the rapid development of artificial intelligence technology, especially deep learning, has brought new technological paths to motion detection. Object detection algorithms based on convolutional neural networks can directly identify and locate moving targets from video frames, achieving significant improvements in detection accuracy. However, traditional deep convolutional neural networks use continuous numerical values ​​for information representation and transmission, and the network inference process involves a large number of floating-point multiplication and addition operations, resulting in high demands on computing resources and energy consumption. In edge computing scenarios such as surveillance equipment, it is difficult to deploy complex deep learning models due to limitations in hardware computing power and power consumption budgets. Furthermore, frame-based deep learning methods still face challenges posed by changes in lighting conditions; when the lighting conditions of the training data differ from those of the actual deployment scenario, the generalization performance of the model will be affected. Summary of the Invention

[0004] This disclosure provides an AI-based motion detection and recording control method that combines variational mode decomposition signal processing technology with Poisson spiking neural network artificial intelligence technology. This effectively separates the interference from gradual changes in illumination from the actual motion signal. The event-driven and sparse computing characteristics of the spiking neural network significantly reduce computational power consumption, making it suitable for deployment in edge monitoring devices. This method achieves high accuracy and low false trigger rate for motion detection in complex lighting scenarios such as dim lighting at dusk and night vision switching.

[0005] To solve the above problems, the technical solution of the present invention is implemented as follows: An AI-based motion detection and recording control method includes the following steps: Step 1: Photon event acquisition and inter-frame differential sequence generation: A photon counting sensor array is used to continuously acquire frame images of the monitored scene. Differential operations are performed on adjacent frame images to obtain inter-frame differential images. The inter-frame differential images of multiple consecutive frames are arranged in chronological order to form an inter-frame differential sequence. Step 2: Variational mode decomposition to separate illumination and motion components: Perform variational mode decomposition on the inter-frame difference sequence obtained in Step 1 to separate the low-frequency illumination slowly varying component and the mid-to-high frequency real motion component. Reconstruct the mid-to-high frequency real motion component according to pixel position to form a motion difference image sequence. Step 3: Processing the motion transient intensity map using a Poisson spiking neural network: Input the motion difference image sequence obtained in Step 2 into a Poisson spiking neural network for processing. After passing through a Poisson pulse encoding layer, a leakage current integral firing neuron layer, and a spatiotemporal convolutional spiking neuron layer, the output pulse firing rate is processed to generate a motion transient intensity map by performing transient energy calculation. Step 4, Block-level Connectivity Aggregation Decision and Recording Control: Divide the motion transient intensity map obtained in Step 3 into image blocks and calculate the block energy value. Aggregate the connected components of motion blocks whose block energy values ​​exceed the preset motion threshold to form motion regions. When the number of motion blocks contained in any motion region exceeds the preset number threshold, trigger the recording device to start recording. When the global motion intensity in the sliding detection window is continuously lower than the preset static threshold, control the recording device to stop recording.

[0006] Furthermore, in step one, the photon counting sensor array is a photon counting single-photon avalanche diode array sensor. Each pixel unit of the photon counting single-photon avalanche diode array sensor independently detects incident photons and outputs a photon count value. The photon count values ​​of each pixel unit within the continuous acquisition period are combined to form a frame image. The differential operation is an operation that subtracts the absolute value of each pixel from the images of two adjacent frames.

[0007] Furthermore, in step two, the specific process of variational mode decomposition is as follows: variational mode decomposition is performed on the time series of each pixel position in the inter-frame difference sequence, and the time series of each pixel position is decomposed into multiple intrinsic mode components. Based on the center frequency of each intrinsic mode component, the intrinsic mode components with a center frequency lower than the preset frequency limit are classified as low-frequency illumination slowly varying components, and the intrinsic mode components with a center frequency higher than or equal to the preset frequency limit are classified as mid-to-high frequency real motion components.

[0008] In step three, the specific process of Poisson pulse coding is as follows: For each pixel of each frame of motion difference image in the motion difference image sequence, the gray value of the current pixel is used as the firing rate parameter of the Poisson process. The preset coding time window is evenly divided into multiple time steps. In each time step, a random number uniformly distributed between zero and one is generated. When the random number is less than the product of the current pixel gray value and the time step length, a pulse is generated in the current time step. When the random number is greater than or equal to the product of the current pixel gray value and the time step length, no pulse is generated in the current time step. In this way, the gray value of each pixel is converted into a binary pulse sequence composed of multiple time steps.

[0009] Furthermore, the specific process of the leakage integral firing neuron layer is as follows: An array of leakage integral firing neurons with the same number of pixels as the motion difference image is constructed. Each leakage integral firing neuron receives a binary pulse sequence at the corresponding pixel position as input and maintains a membrane potential state variable. At the beginning of each time step, the current membrane potential state variable is first multiplied by a fixed attenuation coefficient less than one to achieve membrane potential leakage. Then, it is determined whether an input pulse is received at the current time step. If an input pulse is received, a fixed synaptic input increment is added to the membrane potential state variable. If no input pulse is received, the membrane potential state variable retains the leaked value. Subsequently, the updated membrane potential state variable is compared with a preset firing threshold. When the membrane potential state variable is greater than or equal to the firing threshold, the current leakage integral firing neuron generates an output pulse at the current time step and resets the membrane potential state variable to zero. When the membrane potential state variable is less than the firing threshold, the current leakage integral firing neuron does not generate an output pulse at the current time step. After traversing all time steps within the encoding time window, each leakage integral firing neuron outputs an output pulse sequence of the same length as the input binary pulse sequence.

[0010] Furthermore, the specific process of the spatiotemporal convolutional spiking neuron layer is as follows: a spatiotemporal convolutional spiking neuron layer is set after the leakage integral firing neuron array. For each neuron position in the leakage integral firing neuron array, all adjacent neurons within the coverage area of ​​the spatial convolution kernel of a preset size centered on the current neuron are determined. In each time step, the total number of output pulses generated by all neurons within the coverage area of ​​the spatial convolution kernel in the current time step is counted. The total number of output pulses is used as the input of the spatiotemporal convolutional spiking neuron corresponding to the current position in the current time step. The spatiotemporal convolutional spiking neuron is processed using the same membrane potential leakage, input accumulation, threshold comparison and reset process as the leakage integral firing neuron. The output of the spatiotemporal convolutional spiking neuron layer is a spiking neuron array of the same size as the input.

[0011] Furthermore, the calculation process of the pulse firing rate is as follows: for each neuron output by the spatiotemporal convolutional spiking neuron layer, the total number of output pulses generated by the current neuron within the encoding time window is counted. The total number of output pulses is divided by the total number of time steps contained in the encoding time window to obtain the pulse firing rate of the current neuron. The pulse firing rates of all neurons are arranged according to their spatial positions to form a pulse firing rate image.

[0012] Furthermore, the specific process of transient energy calculation is as follows: For each pixel position in the pulse firing rate image, obtain the three pulse firing rate values ​​corresponding to the current position in the current frame, the previous frame, and the next frame. Multiply the pulse firing rate value of the current frame by itself to obtain the first intermediate value. Multiply the pulse firing rate value of the previous frame by the pulse firing rate value of the next frame to obtain the second intermediate value. Subtract the second intermediate value from the first intermediate value to obtain the transient energy value of the current pixel position. When the first intermediate value is less than the second intermediate value, set the transient energy value of the current pixel position to zero. Combine the transient energy values ​​of all pixel positions to form a motion transient intensity map.

[0013] Furthermore, in step four, the specific process of connected component aggregation is as follows: the motion transient intensity map is divided into multiple image blocks of fixed size, the arithmetic mean of the transient energy values ​​of all pixels in each image block is calculated as the block energy value, the image blocks with block energy values ​​greater than a preset motion threshold are marked as motion blocks, all motion blocks are marked with eight-neighbor connected components, and adjacent motion blocks are merged into the same motion region.

[0014] Furthermore, in step four, while the recording device is in recording mode, steps one to three are continuously executed to obtain the motion transient intensity map, the arithmetic mean of the transient energy values ​​of all pixels in the motion transient intensity map is calculated as the global motion intensity, a sliding detection window containing a preset number of frames is established, and when the global motion intensity of all consecutive frames in the sliding detection window is less than the preset static threshold, the recording device is controlled to stop recording.

[0015] The AI-based motion detection and recording control method of the present invention has the following beneficial effects: The present invention uses a photon-counting single-photon avalanche diode array sensor as the image acquisition front end. This sensor can detect incident light at the single-photon level and can still acquire effective image information in extremely low-light environments. Compared with traditional complementary metal-oxide-semiconductor image sensors, it has higher sensitivity and a larger dynamic range, providing a hardware foundation for motion detection in low-light scenes at dusk and night vision switching scenarios, and significantly extending the effective working time of the video surveillance system.

[0016] This invention performs variational mode decomposition (VMD) on inter-frame difference sequences to effectively separate low-frequency interference, such as gradually changing illumination, from the mid-to-high-frequency signals corresponding to real motion. VMD has a solid mathematical foundation, produces stable decomposition results that are less prone to mode aliasing, and can adaptively determine the center frequency of each intrinsic mode component based on signal characteristics. By eliminating low-frequency gradually changing illumination components and retaining mid-to-high-frequency real motion components, this invention effectively solves the technical problem of traditional motion detection methods easily generating false motion responses in scenarios with gradually changing illumination and automatic exposure adjustment, significantly reducing the false trigger rate.

[0017] This invention introduces a Poisson spiking neural network as the core artificial intelligence processing unit. Poisson pulse coding converts pixel grayscale values ​​into a pulse sequence conforming to a Poisson distribution. This encoding method is highly similar to the information representation mechanism of biological nervous systems, naturally mapping motion intensity to pulse firing rate. The leakage integral firing neuron layer performs temporal filtering on the input pulses through the leakage and integration mechanism of membrane potential, enhancing the response to continuous motion signals and suppressing random noise pulses. The spatiotemporal convolutional spiking neuron layer introduces local spatial correlation constraints through spatial neighborhood aggregation, effectively suppressing isolated noise responses due to the lack of neighborhood support, thus enhancing the response of the real motion region. Compared to traditional deep convolutional neural networks, spiking neural networks use discrete pulses for information transmission, exhibiting event-driven and sparse computation characteristics. While achieving artificial intelligence motion recognition functions, it significantly reduces computational power consumption, making it particularly suitable for deployment in edge computing scenarios such as monitoring equipment. Attached Figure Description

[0018] Figure 1 A schematic diagram illustrating the temporal domain variation characteristics of the original inter-frame difference sequence acquired by the photon counting sensor array provided in an embodiment of the present invention; Figure 2 A schematic diagram illustrating the dynamic evolution of membrane potential in the leaky integral firing neuron model provided in this embodiment of the invention; Figure 3 A schematic diagram illustrating the time-domain characteristics of pulse firing rate under three typical motion modes provided in this embodiment of the invention; Figure 4 This is a schematic diagram of the transient energy response characteristics corresponding to the three motion modes provided in the embodiments of the present invention. Detailed Implementation

[0019] An AI-based motion detection and recording control method includes the following steps: Step 1: Photon event acquisition and inter-frame differential sequence generation: A photon counting sensor array is used to continuously acquire frame images of the monitored scene. Differential operations are performed on adjacent frame images to obtain inter-frame differential images. The inter-frame differential images of multiple consecutive frames are arranged in chronological order to form an inter-frame differential sequence. Step 2: Variational mode decomposition to separate illumination and motion components: Perform variational mode decomposition on the inter-frame difference sequence obtained in Step 1 to separate the low-frequency illumination slowly varying component and the mid-to-high frequency real motion component. Reconstruct the mid-to-high frequency real motion component according to pixel position to form a motion difference image sequence. Step 3: Processing the motion transient intensity map using a Poisson spiking neural network: Input the motion difference image sequence obtained in Step 2 into a Poisson spiking neural network for processing. After passing through a Poisson pulse encoding layer, a leakage current integral firing neuron layer, and a spatiotemporal convolutional spiking neuron layer, the output pulse firing rate is processed to generate a motion transient intensity map by performing transient energy calculation. Step 4, Block-level Connectivity Aggregation Decision and Recording Control: Divide the motion transient intensity map obtained in Step 3 into image blocks and calculate the block energy value. Aggregate the connected components of motion blocks whose block energy values ​​exceed the preset motion threshold to form motion regions. When the number of motion blocks contained in any motion region exceeds the preset number threshold, trigger the recording device to start recording. When the global motion intensity in the sliding detection window is continuously lower than the preset static threshold, control the recording device to stop recording.

[0020] This invention employs a photon-counting sensor array to continuously acquire data from monitored scenes. The photon-counting sensor array is preferably a photon-counting single-photon avalanche diode array sensor. This type of sensor operates based on Geiger mode; when a single photon is incident on a pixel unit, it triggers an avalanche multiplication effect, generating a detectable current pulse, thus achieving single-photon-level sensitivity. Compared to traditional complementary metal-oxide-semiconductor (CMOS) image sensors or charge-coupled device (CCD) sensors, the photon-counting single-photon avalanche diode array sensor can still acquire effective image information in extremely low-light environments, making it particularly suitable for motion detection tasks in low-light scenes at dusk and night vision switching scenarios.

[0021] Each pixel unit of a photon-counting single-photon avalanche diode array sensor operates independently, counting incident photons within a preset acquisition period. In a typical configuration, the sensor array resolution is set to 640 x 480 pixels or 1280 x 720 pixels, and the acquisition frame rate is set to 25 to 30 frames per second. The number of photons detected by each pixel unit within one acquisition period forms the photon count value for that pixel location. Arranging the photon count values ​​of all pixel units within the same acquisition period according to their spatial positions constitutes a complete frame image. The dynamic range of the photon count value is typically 0 to 65535, and it is stored using a 16-bit unsigned integer.

[0022] In alternative implementations, the photon counting sensor array can also employ other types of high-sensitivity sensors, such as electron multiplication charge-coupled device sensors or scientific-grade complementary metal-oxide-semiconductor sensors. These sensors also have the ability to perform effective imaging in low-light environments and can be selected based on the cost and performance requirements of the actual application scenario.

[0023] After acquiring frame images, a difference operation is performed on adjacent frame images to extract change information in the scene. The specific process of the difference operation is as follows: Let the frame image acquired at the current time be the [number]th frame. Frame, the frame image captured at the previous moment is the first frame. Frame, in which Represents the frame number and is a positive integer greater than 1. For any pixel position in the image... ,in Represents the row coordinates of a pixel. Representing the column coordinates of a pixel, the pixel value at that location in the inter-frame difference image is calculated as follows: [The text abruptly ends here, likely due to an incomplete sentence or a formatting error.] Frame in position Photon count value minus the first The photon counts at the same position in each frame are calculated, and then the absolute value of the difference is taken. This pixel-by-pixel subtraction and absolute value taking operation method ensures that the difference result is always non-negative, and at the same time can detect both increases and decreases in brightness in the scene.

[0024] The mathematical expression for inter-frame difference operation is as follows: ,in Indicates the first Frame and the Inter-frame difference image at pixel location The difference at that point, Indicates the first Frame image at pixel position The photon count at that location, Indicates the first The photon count at the same pixel location in a frame image. This represents the absolute value operation. Through the above difference operation, the difference result of static background areas in the scene is close to zero because the pixel values ​​are basically the same between adjacent frames; while the difference result of moving areas in the scene is larger because the pixel values ​​are significantly different due to the change of the target position. This characteristic allows inter-frame difference images to effectively highlight motion information and suppress static backgrounds.

[0025] The inter-frame difference images of multiple consecutive frames are arranged sequentially in chronological order to form an inter-frame difference sequence. Assume that a total of [number] frames were acquired within the time period to be processed. A frame image can then be generated. Inter-frame difference images, arranged in the order they were generated, constitute an inter-frame difference sequence. In practical applications, to ensure the effectiveness of subsequent variational mode decomposition (VMD) processing, the inter-frame difference sequence typically contains 32 to 128 inter-frame difference images, corresponding to a video duration of approximately 1 to 5 seconds. Longer sequence lengths help VMD more accurately separate components of different frequencies, but also increase computational latency and memory usage; shorter sequence lengths offer faster response times but may decrease decomposition accuracy. In actual deployment, a suitable sequence length within the above range can be selected based on processor performance and real-time requirements. While inter-frame difference sequences can extract scene change information, they include both effective changes caused by the movement of real objects and interference changes caused by factors such as gradual changes in lighting, automatic exposure adjustment, and sensor noise. Especially during the evening hours, when natural light intensity continuously decreases, the automatic exposure function of surveillance cameras constantly adjusts exposure parameters to adapt to changes in lighting, resulting in global low-frequency fluctuations in the inter-frame difference images. Furthermore, the activation of the infrared fill light when switching to night vision mode can cause sudden changes in image brightness. Failure to effectively separate these interferences can lead to false recordings or missed detection of real moving targets.

[0026] refer to Figure 1 In this embodiment, the horizontal axis represents time in seconds, covering a continuous sampling period of 0 to 10 seconds. The vertical axis represents the pixel difference value, which is calculated by the difference in photon counts at corresponding pixel positions in two adjacent frames. The solid blue curve in the figure depicts the time-domain waveform of the original inter-frame difference signal, and the light blue area below the curve is used to enhance the visualization of the signal change trend. From the waveform characteristics, it can be observed that the original inter-frame difference sequence exhibits complex mixed signal characteristics, containing the superposition of multiple frequency components. At approximately 2 seconds, 4.5 seconds, and 7 seconds on the time axis, the signal shows obvious local high-frequency oscillations. These oscillations correspond to real motion events in the actual scene, manifested as rapid fluctuations in the pixel difference value. At the same time, the entire signal sequence is also superimposed with a slowly changing low-frequency envelope. This envelope component mainly originates from the gradual influence of ambient light, such as changes in natural light intensity caused by cloud cover, automatic dimming of indoor lighting equipment, and global brightness adjustment introduced by the camera's automatic exposure compensation mechanism. The low-frequency, slowly varying illumination components are superimposed with mid-to-high-frequency motion components, making it difficult to accurately extract real motion information directly from the original inter-frame difference sequence. The original signal also contains a certain amount of random noise, manifested as subtle, irregular waveform jitter. This invention subsequently employs variational mode decomposition (VMD) to perform frequency domain separation processing on the original inter-frame difference sequence, using 0.8 Hz as the boundary threshold to effectively separate the slowly varying illumination components below this frequency from the real motion components above it, thereby providing a clean motion feature input for subsequent spiking neural network processing.

[0027] This invention employs variational mode decomposition (VMD) to analyze inter-frame difference sequences, separating low-frequency interference such as slowly varying illumination from the actual motion signal. VMD is an adaptive signal decomposition method that can decompose complex signals into several intrinsic mode components (EMCs) with different center frequencies. Each EMC is compactly distributed around its center frequency in the frequency domain. Compared with methods such as empirical mode decomposition (EMD), VMD has a more robust mathematical foundation, produces more stable decomposition results, and is less prone to mode aliasing. VMD processing is performed independently for the time series of each pixel position in the inter-frame difference sequence. Specifically, for any pixel position in the inter-frame difference sequence... The pixel values ​​at that location are extracted from all inter-frame difference images and arranged in chronological order to form a one-dimensional time series. Let the inter-frame difference sequence contain... For frame-to-frame difference images, the time series length corresponding to each pixel position is... By pixel position For example, its corresponding time series can be represented as ,in Indicates the first Inter-frame difference image at pixel position The difference at that point, The frame number is a number ranging from 1 to 1. .

[0028] The core idea of ​​variational mode decomposition is to decompose the input signal into... The goal is to obtain 10 intrinsic mode components (IMCs) such that the sum of all IMCs equals the original signal, while minimizing the bandwidth of each IMC. The decomposition process is achieved by solving a constrained variational problem. The objective function is the sum of the estimated bandwidths of all IMCs, and the constraint is to reconstruct the original signal from the sum of the IMCs. This variational problem is solved iteratively using the alternating direction multiplier method, updating each IMC and its corresponding center frequency in each iteration until convergence. The number of IMCs is... These are important parameters in variational mode decomposition and need to be preset according to signal characteristics. For inter-frame difference time series in motion detection scenarios, they are typically... Set to 3 to 5. Smaller. Values ​​that are too high may cause different frequency components to be mixed in the same intrinsic mode component, making effective separation impossible. A value that is too high may lead to over-decomposition, causing the effective signal to be scattered across multiple intrinsic mode components. In a preferred configuration, When set to 4, variational mode decomposition will decompose the time series of each pixel location into 4 intrinsic mode components.

[0029] After variational mode decomposition is completed, we obtain Each intrinsic mode component (IMC) corresponds to a center frequency. Interference such as gradual changes in illumination and automatic exposure adjustment typically manifests as slowly changing low-frequency signals, with their frequency components mainly concentrated in the 0.1 Hz to 0.5 Hz range. In contrast, pixel value changes caused by the movement of real objects typically manifest as faster mid-to-high-frequency signals, with their frequency components mainly distributed in the 0.5 Hz to 10 Hz range. Based on this difference in frequency characteristics, the IMC components are divided into two categories by setting preset frequency limits.

[0030] A preset frequency limit is used to distinguish between low-frequency, slowly varying light components and mid-to-high-frequency real motion components. Its value is determined based on the light variation characteristics and target motion characteristics of the monitored scene. A typical setting is to set the preset frequency limit to 2% to 5% of the acquisition frame rate. For example, when the acquisition frame rate is 25 frames per second, the preset frequency limit can be set to 0.5 Hz to 1.25 Hz; when the acquisition frame rate is 30 frames per second, the preset frequency limit can be set to 0.6 Hz to 1.5 Hz. In a preferred embodiment, the preset frequency limit is set to 0.8 Hz.

[0031] Based on the comparison results between the center frequency of each intrinsic mode component and a preset frequency limit, the intrinsic mode components are classified as follows: those with a center frequency lower than the preset frequency limit are classified as low-frequency slowly varying illumination components, reflecting slowly changing interference information such as illumination gradients and automatic exposure adjustments; those with a center frequency higher than or equal to the preset frequency limit are classified as mid-to-high-frequency true motion components, reflecting effective change information caused by object motion in the scene. Low-frequency slowly varying illumination components are identified and discarded, retaining only the mid-to-high-frequency true motion components for subsequent processing. For each pixel location, all mid-to-high-frequency true motion components corresponding to that location are superimposed to obtain the time series of that pixel location after filtering out illumination interference. All pixel locations are reorganized according to their spatial location, forming a frame image at each moment. These images are arranged in chronological order to form a motion difference image sequence. Each frame of the motion difference image sequence contains only the pixel values ​​reconstructed from the mid-to-high-frequency true motion components, effectively eliminating the influence of illumination gradients and automatic exposure jitter.

[0032] The motion difference image sequence obtained after variational mode decomposition has eliminated low-frequency interference such as slowly varying illumination, but it still exists in the traditional frame image format. To further extract motion features and enhance the detection capability for transient motion, this invention inputs the motion difference image sequence into a Poisson spiking neural network for processing. A Poisson spiking neural network is an artificial intelligence network that simulates the information processing mechanism of a biological nervous system. Its neurons communicate through discrete pulse signals, rather than the continuous numerical values ​​used in traditional artificial neural networks. This pulse-based information representation method has a natural advantage in time-series signal processing, effectively capturing the transient characteristics of signals, while also having lower power consumption in hardware implementation, making it particularly suitable for deployment in edge computing scenarios such as monitoring equipment.

[0033] The first step in Poisson spiking neural network processing is to convert the pixel grayscale values ​​in the motion difference image sequence into a pulse sequence; this process is called Poisson pulse coding. The reason for choosing the Poisson process as the pulse coding method is that it is a classic mathematical model describing the frequency of random events, and its pulse firing exhibits statistical memorylessness, meaning that whether a pulse is generated at each moment depends only on the current firing rate parameter and is independent of previous pulse firing history. This characteristic is highly similar to how sensory neurons in biological nervous systems encode stimulus intensity: stronger stimuli correspond to a higher pulse firing rate, and weaker stimuli correspond to a lower pulse firing rate.

[0034] Poisson pulse coding is performed independently for each pixel of each frame of a motion difference image sequence. For any pixel position... First, the grayscale value of the current pixel is read and used as the firing rate parameter for the Poisson process. Since the pixel values ​​in motion difference images typically range from 0 to 255 or 0 to 65535, and the firing rate parameter of the Poisson process needs to be normalized to a suitable range, the grayscale value is normalized by dividing it by the maximum grayscale value before encoding. The normalized value represents the expected probability that the pixel will generate a pulse per unit time.

[0035] The encoding time window is a crucial parameter in Poisson pulse coding, determining the length and temporal resolution of the pulse sequence. The preset encoding time window is evenly divided into multiple time steps, each corresponding to a sampling point in the pulse sequence. The length of the encoding time window is typically set to 10 to 50 milliseconds, and the number of time steps is typically set to 16 to 64. In a preferred configuration, the encoding time window is set to 20 milliseconds, and the number of time steps is set to 32, resulting in each time step being 0.625 milliseconds long. A higher number of time steps increases the accuracy of the pulse sequence's representation of the original grayscale values, but also correspondingly increases the computational load.

[0036] Within each time step, the following steps are performed to determine whether a pulse is generated: First, a random number uniformly distributed between zero and one is generated, which can be obtained using pseudo-random number generation methods such as a linear congruential generator or the Mason tween algorithm. Then, this random number is compared with the product of the current pixel's grayscale value and the time step length. Let the normalized grayscale value of the current pixel be... The time step length is The generated random number is ,in This represents the normalized pixel grayscale value, with a value range from 0 to 1. Indicates the duration of a single time step. This represents a random number generated at the current time step that follows a uniform distribution from zero to one. When the random number... Less than the product When a random number is generated, a pulse is generated at the current time step, and the value at the corresponding position in the pulse sequence is set to 1; when a random number is generated... Greater than or equal to the product If a pixel's grayscale value is not specified, no pulse is generated at the current time step, and the value at the corresponding position in the pulse sequence is set to 0. This pulse generation mechanism based on probability comparison ensures that pixels with larger grayscale values ​​generate more pulses within the encoding time window, pixels with smaller grayscale values ​​generate fewer pulses, and pixels with a grayscale value of zero generate no pulses. By traversing all time steps within the encoding time window, the grayscale value of each pixel is converted into a binary pulse sequence composed of multiple time steps. Each element in the sequence takes only 0 or 1 values, where 0 represents no pulse and 1 represents a pulse.

[0037] In optional implementations, the length of the encoding time window and the number of time steps can be adjusted according to the actual application scenario. For real-time monitoring scenarios requiring fast response, the encoding time window can be shortened to 10 milliseconds and the number of time steps reduced to 16 to reduce processing latency; for scenarios with high detection accuracy requirements, the encoding time window can be extended to 50 milliseconds and the number of time steps increased to 64 to improve the statistical stability of the pulse sequence. Another optional implementation is to use rate coding instead of Poisson coding, that is, to directly determine the total number of pulses generated within the encoding time window based on the pixel grayscale value, and then distribute these pulses evenly or randomly in each time step. This method reduces the computational overhead of random number generation.

[0038] After Poisson pulse encoding, each pixel position corresponds to a binary pulse sequence. These pulse sequences are then input into a leaky integral-fired neuron layer for processing. The leaky integral-fired neuron is the most commonly used neuron model in spiking neural networks. Its working mechanism simulates the dynamic changes in membrane potential of biological neurons: after receiving an input pulse, the membrane potential rises, and simultaneously decays and leaks over time. When the accumulated membrane potential exceeds a threshold, the neuron fires an output pulse and resets. This mechanism allows the leaky integral-fired neuron to integrate the input pulse over time, transforming scattered input pulses into a time-smooth output pulse sequence, thereby enhancing the response to continuous motion signals and suppressing random noise.

[0039] A leakage integral firing neuron array is constructed, the size of which is the same as the number of pixels in the motion difference image, i.e., one leakage integral firing neuron corresponds to each pixel location. Let the resolution of the motion difference image be... ,in This represents the image width, which is the number of pixels in the horizontal direction. The image height, i.e., the number of pixels in the vertical direction, is represented by the leakage integral firing neuron array. There are 10 neurons. Each leakage integral firing neuron receives a binary pulse sequence at the corresponding pixel location as input and maintains a membrane potential state variable to record the current membrane potential value. The initial value of the membrane potential state variable is set to zero.

[0040] The leakage integral firing neuron processes data at each time step according to the following procedure. At the beginning of each time step, a membrane potential leakage operation is first performed: the current membrane potential state variable is multiplied by a fixed decay coefficient less than one to obtain the leaked membrane potential value. The fixed decay coefficient typically ranges from 0.8 to 0.99, and this coefficient determines the leakage rate of the membrane potential. A smaller fixed decay coefficient means faster membrane potential leakage, reducing the neuron's sensitivity to input pulses with longer time intervals, and making it more inclined to respond to dense, continuous pulse inputs; a larger fixed decay coefficient means slower membrane potential leakage, allowing the neuron to accumulate the effects of input pulses over a longer time range. In a preferred configuration, the fixed decay coefficient is set to 0.9, a value that maintains adequate time integration capability while avoiding response lag caused by long-term accumulation of membrane potential.

[0041] The mathematical expression for membrane potential leakage is: ,in This represents the value of the membrane potential state variable at the start of the current time step. This represents a constant with a fixed attenuation coefficient, whose value ranges from 0 to 1. This represents the membrane potential value after the leakage operation. After the membrane potential leakage is completed, it is determined whether an input pulse has been received at the current time step. The pulse value at the corresponding position of the current time step is read from the input binary pulse sequence. If the value is 1, it indicates that an input pulse has been received; if the value is 0, it indicates that no input pulse has been received. When an input pulse is received, a fixed synaptic input increment is added to the membrane potential state variable; when no input pulse is received, the membrane potential state variable remains unchanged after the leakage. The fixed synaptic input increment represents the contribution of a single input pulse to the membrane potential, and its value is usually set to 0.1 to 0.5. The larger the synaptic input increment, the more significant the effect of a single input pulse on the membrane potential, and the easier it is for the neuron to be activated; the smaller the synaptic input increment, the more input pulses are needed to trigger neuronal firing. In a preferred configuration, the fixed synaptic input increment is set to 0.2.

[0042] The membrane potential update after receiving the input can be expressed as: ,in This represents the membrane potential value after leakage. This represents a fixed increment of synaptic input. This represents the input pulse value at the current time step, and its value can be either 0 or 1. This represents the membrane potential value after receiving input. The updated membrane potential state variable is then compared with a preset firing threshold to determine whether to generate an output pulse. The firing threshold is a key parameter controlling the neuron's output characteristics, and its value is typically set between 0.5 and 2.0. When the membrane potential state variable is greater than or equal to the firing threshold, the current leakage current integrating neuron generates an output pulse at the current time step, while simultaneously resetting the membrane potential state variable to zero, preparing for the next round of integration. When the membrane potential state variable is less than the firing threshold, the current leakage current integrating neuron does not generate an output pulse at the current time step, and the membrane potential state variable retains its updated value and continues to participate in the integration process of subsequent time steps. In a preferred configuration, the preset firing threshold is set to 1.0.

[0043] refer to Figure 2 In this embodiment, the horizontal axis represents discrete time steps, ranging from 0 to 100 time units. The vertical axis represents the membrane potential of the neuron. The values ​​range from 0 to approximately 1.3. The figure contains three curves of different colors, corresponding to three leaky integral firing neurons driven by Poisson encoding of pixels with different input grayscale values. The red curve corresponds to high grayscale input pixels with a normalized grayscale value of 0.8; the blue curve corresponds to medium grayscale input pixels with a normalized grayscale value of 0.5; and the green curve corresponds to low grayscale input pixels with a normalized grayscale value of 0.2. The black horizontal dashed line marks the firing threshold. When the membrane potential reaches or exceeds this threshold, the neuron generates an output pulse and resets the membrane potential to 0. The membrane potential update of the leaky integral firing neuron follows a recursive formula. ,in The attenuation coefficient characterizes the natural leakage characteristics of the film potential when there is no input. For synaptic weights, This represents the input pulse value at the current time step. Observing the curve shape, the red curve corresponding to high grayscale pixels shows a rapid accumulation of membrane potential due to the high input pulse firing rate, frequently reaching the threshold and triggering firing events, followed by membrane potential reset to form a sawtooth waveform. The blue curve corresponding to medium grayscale pixels has a moderate membrane potential accumulation rate, with a firing frequency between high and low grayscale. The green curve corresponding to low grayscale pixels has a slow membrane potential accumulation due to sparse input pulses, remaining at a low level most of the time, with relatively few firing events. This mechanism, which converts input grayscale values ​​into pulse sequences through Poisson encoding and then processes the output firing rate through leaky integral firing neurons, achieves an effective conversion from static pixel intensity to dynamic temporal pulse features.

[0044] Following the above process, all time steps within the encoding time window are traversed. Each leaky integral-fire neuron outputs a sequence of output pulses of the same length as the input binary pulse sequence. The output pulse sequence is also a binary sequence, where 1 indicates that an output pulse was generated at that time step, and 0 indicates that no output pulse was generated. After processing by the leaky integral-fire neuron layer, random noise pulses in the original input are effectively suppressed because they cannot continuously accumulate membrane potential, while continuous pulse inputs corresponding to real motion signals can successfully trigger output pulses, achieving preliminary signal filtering and denoising.

[0045] In optional implementations, the fixed attenuation coefficient, fixed synaptic input increment, and preset firing threshold can be jointly adjusted according to specific application scenarios. For scenarios requiring the detection of rapid motion, the fixed attenuation coefficient can be appropriately increased and the preset firing threshold decreased, enabling neurons to respond quickly to brief motion signals. For scenarios requiring the detection of continuous motion and the suppression of transient interference, the fixed attenuation coefficient can be appropriately decreased and the preset firing threshold increased, causing neurons to respond only to continuous motion signals.

[0046] The output pulse sequence of the leakage integral firing neuron layer reflects the independent motion response of each pixel location, but the spatial correlation between adjacent pixels has not been considered. Actual moving targets typically occupy a continuous region in an image, where multiple pixels simultaneously generate motion responses. To leverage this spatial correlation to enhance the robustness of motion detection and suppress isolated noise responses, this invention sets a spatiotemporal convolutional spiking neuron layer after the leakage integral firing neuron array to perform spatial neighborhood aggregation on the output pulses. The spatiotemporal convolutional spiking neuron layer processes the output of the leakage integral firing neuron array using a spatial convolution kernel of a preset size. The spatial convolution kernel is a two-dimensional window, and its size determines the range of spatial neighborhood aggregation. The size of the spatial convolution kernel is typically set to 3 x 3, 5 x 5, or 7 x 7 pixels. Smaller spatial convolution kernels retain more spatial detail information, suitable for detecting small moving targets; larger spatial convolution kernels aggregate a wider range of neighborhood information, providing stronger noise suppression but potentially blurring the boundaries of small targets. In a preferred configuration, the size of the spatial convolution kernel is set to 3 x 3 pixels.

[0047] Spatial neighborhood aggregation is performed for each neuron location in the leakage current integral firing neuron array. For neurons located at position... For each neuron, the spatial convolution kernel centered on the current neuron determines all neighboring neurons within its coverage area. For example, with a 3x3 spatial convolution kernel, the coverage area includes the location... , , , , , , , , There are a total of 9 neurons. For neurons at the image boundary, the spatial convolution kernel may exceed the image range. In this case, a boundary padding strategy can be used. Common strategies include zero padding, which treats the position outside the range as having no pulse output, or mirror padding, which mirrors the pulse value at the boundary to the position outside the range.

[0048] Within each time step, the total number of output pulses generated by all neurons within the coverage area of ​​the spatial convolution kernel at the current time step is counted. Continuing with the example of a 3x3 spatial convolution kernel, the output pulse values ​​of the 9 neurons within the coverage area are checked at each time step, and all pulses with a value of 1 are counted. The total number obtained is the pulse aggregation result for the current time step. The value of this aggregation result ranges from 0 to 9, reflecting the motion activity level of the current position and its surrounding neighborhood. The total number of output pulses is used as the input to the spatiotemporal convolution spiking neuron corresponding to the current position at the current time step. The spatiotemporal convolution spiking neuron uses the same membrane potential leakage, input accumulation, threshold comparison, and reset process as the leakage integral firing neuron. Specifically, the spatiotemporal convolution spiking neuron also maintains a membrane potential state variable. At the beginning of each time step, the membrane potential is multiplied by a fixed attenuation coefficient to achieve leakage. Then, the total number of neighborhood pulses is multiplied by a fixed synaptic input increment and accumulated onto the membrane potential. Subsequently, the membrane potential is compared with the firing threshold to determine whether an output pulse is generated. The difference is that the input to a spatiotemporal convolutional spiking neuron is no longer a single pulse of 0 or 1, but a pulse count value within the neighborhood. Therefore, the accumulation rate of its membrane potential is proportional to the activity level of the neighborhood.

[0049] The fixed attenuation coefficient, fixed synaptic input increment, and firing threshold used in the spatiotemporal convolutional spiking neuron layer can be the same as those used in the leakage integral firing neuron layer, or they can be set independently according to the characteristics of spatial aggregation. In a preferred configuration, the firing threshold of the spatiotemporal convolutional spiking neuron layer is set to 3.0, which is higher than the firing threshold of the leakage integral firing neuron layer. This ensures that the output pulse is triggered only when multiple neurons in the neighborhood respond simultaneously, thereby effectively suppressing isolated noise points. The output of the spatiotemporal convolutional spiking neuron layer is a spiking neuron array of the same size as the input; that is, the size of the output array remains the same. There are 10 neurons, each outputting a pulse sequence. After spatial neighborhood aggregation, the effective pulses in the output pulse sequence are mainly concentrated in the real motion region, and isolated noise responses are effectively suppressed due to the lack of neighborhood support.

[0050] The pulse sequence output by the spatiotemporal convolutional spiking neuron layer needs to be converted into numerical form for subsequent transient energy calculations. The pulse firing rate is a standard indicator of neuron activity, defined as the ratio of the number of pulses generated by the neuron within a given time period to the length of that time period. For each neuron output by the spatiotemporal convolutional spiking neuron layer, the total number of output pulses generated by the current neuron within the encoding time window is counted. The output pulse sequence of this neuron is iterated through, and all elements with a value of 1 are counted to obtain the total number of output pulses. Dividing the total number of output pulses by the total number of time steps contained in the encoding time window yields the pulse firing rate of the current neuron. The calculation of the pulse firing rate can be expressed as follows: ,in This represents the current spiking rate of the neuron. This represents the total number of output pulses generated by the current neuron within the encoding time window. This represents the total number of time steps contained within the encoding time window. The pulse firing rate ranges from 0 to 1; a larger value indicates a more active neuron and a stronger motor response at the corresponding location.

[0051] The firing rates of all neurons are arranged according to their spatial location to form a firing rate image of the same size as the motion difference image. Each pixel value in the firing rate image represents the motion response intensity at the corresponding location after processing by the Poisson spiking neural network; a larger value indicates a higher probability of motion at that location. While the firing rate image reflects the motion response intensity at each pixel location, its ability to detect rapidly changing transient motion signals is limited. To enhance the detection capability for transient events such as motion initiation and cessation, this invention performs transient energy calculation on the firing rate. Transient energy calculation originates from the concept of energy operators in signal processing, which can highlight the instantaneous change characteristics of signals and has good response characteristics for non-stationary signals where amplitude and frequency change simultaneously.

[0052] Transient energy calculation requires the pulse firing rate values ​​of the current frame and its adjacent frames, therefore at least three consecutive frames of pulse firing rate images are needed. For each pixel position in the pulse firing rate image... This retrieves the three pulse firing rate values ​​corresponding to the current position in the current frame, the previous frame, and the next frame. Let the current frame be the [frame number]. Frame, then it is necessary to obtain the first frame. Frame, First Frame and the Frame in position The pulse firing rate values ​​are denoted as follows: , and ,in This indicates the pulse firing rate at the current pixel position in the previous frame. This represents the pulse firing rate at the current pixel position in the current frame. This represents the pulse firing rate at the current pixel position in the next frame. Transient energy calculation is performed in the following steps. First, the pulse firing rate value of the current frame is multiplied by itself, i.e., the transient energy is calculated. The product of the value and itself yields the first intermediate value. This first intermediate value reflects the square of the motion response intensity at that position in the current frame, amplifying stronger motion responses. Then, the pulse firing rate value of the previous frame is multiplied by the pulse firing rate value of the next frame, thus calculating... and The product of the first and second intermediate values ​​is used to obtain the second intermediate value. The second intermediate value reflects the product of the motion responses of the two frames before and after the current frame. When the motion signal is stable and continuous in time, the second intermediate value is close to the first intermediate value; when the motion signal undergoes transient changes, there will be a significant difference between the second intermediate value and the first intermediate value.

[0053] Subtracting the second intermediate value from the first intermediate value yields the transient energy value at the current pixel location. This operation can be expressed as follows: ,in Indicates the first Frame at pixel position The transient energy value at a given location. When the motion signal is stable and continuous, the pulse firing rates of adjacent frames are close, and the transient energy value approaches zero. When the motion signal suddenly increases or decreases, the pulse firing rates of the current frame differ significantly from those of the preceding and following frames, resulting in a positive transient energy value. In some cases, due to noise fluctuations or signal attenuation, the first intermediate value may be less than the second intermediate value, leading to a negative transient energy value. Negative transient energy values ​​are physically unreasonable; therefore, when the first intermediate value is less than the second intermediate value, the transient energy value at the current pixel location is set to zero to ensure that the transient energy value is always non-negative.

[0054] refer to Figure 3 and Figure 4 ; Figure 3This diagram illustrates the temporal characteristics of the pulse firing rate under three typical motion modes. The first subplot depicts a scene where motion suddenly begins. The horizontal axis represents frame number, ranging from 0 to 100, and the vertical axis represents the pulse firing rate, ranging from 0 to 1. Before frame number 30, the pulse firing rate remains at a low level of approximately 0.1, corresponding to a static state with no significant motion in the scene; the curve exhibits smooth, small random fluctuations. Starting from frame number 30, the pulse firing rate suddenly jumps to a high level of approximately 0.8, corresponding to the moment the target object in the scene begins to move. The red vertical dashed line precisely marks this starting point of motion. Thereafter, until frame number 100, the pulse firing rate remains high, indicating the continuation of the motion state. The second subplot depicts a scene where motion suddenly stops. Between frames 0 and 60, the pulse firing rate remains at a high level of approximately 0.8, corresponding to the target object being in continuous motion. Starting from frame number 60, the pulse firing rate drops sharply to a low level of approximately 0.1, corresponding to the moment the motion stops; the red vertical dashed line marks this ending point of motion. The pulse rate then remained low until frame number 100. The third sub-figure depicts a scene with smooth motion. Throughout the time range of 0 to 100 frames, the pulse rate fluctuated around a mean of approximately 0.5, marked by an orange dashed horizontal line. The small fluctuations and lack of significant abrupt changes indicate that the motion remained stable and continuous, with no start or end events. These three modes cover the most common types of motion changes in real-world video surveillance scenarios.

[0055] Figure 4 Showing with Figure 3 The transient energy response characteristics corresponding to the three motion modes are shown. Transient energy is determined by the formula... Calculation, where Indicates pixel position In the The pulse firing rate of the frame. The first subplot shows the transient energy response of the sudden start mode, with the horizontal axis representing the frame number and the vertical axis representing the transient energy value. Near frame number 30, a significant positive peak appears in the transient energy, due to the square term in the formula when the firing rate jumps sharply from a low to a high value. Significantly increased product terms between consecutive frames The presence of low-value frames results in a small, positive difference with a large amplitude. A green vertical dashed line marks the peak detection location, and a box indicates that this peak corresponds to the motion start event. The second sub-figure shows the transient energy response of the sudden motion stop mode. A significant negative peak appears near frame number 60. This is because when the firing rate drops sharply from a high value to a low value, the square term of the current frame decreases, while the product term of the preceding and following frames is larger due to the inclusion of high-value frames, resulting in a negative difference. Detecting this negative peak can be used to determine the motion termination event. The third sub-figure shows the transient energy response of the steady motion mode. Because the firing rate remains relatively stable throughout the time series, and the values ​​of preceding and following frames are close, the two terms in the transient energy formula cancel each other out, causing the transient energy value to fluctuate slightly near zero, with no significant peak. Another box indicates that this peakless feature represents a steady motion state. By detecting the peak value and its polarity of the transient energy, the start and end times of motion can be effectively identified, providing crucial information for subsequent video recording control decisions.

[0056] The transient energy values ​​of all pixel locations are arranged spatially to form a motion transient intensity map. The motion transient intensity map has the same size as the original motion difference image, and each pixel value represents the intensity of the motion transient response at the corresponding location. Regions with larger values ​​in the motion transient intensity map correspond to the location of the moving target or the motion boundary region, while regions with smaller values ​​or zero values ​​correspond to a stationary background or a stable, continuous internal region. In an optional implementation, transient energy calculation can also be performed using an extended time window, considering not only the previous and next frames but also the two previous and two next frames, resulting in a more stable transient energy estimate through weighted combination. Another optional implementation is to add spatial smoothing processing after transient energy calculation, using Gaussian filtering or mean filtering to smooth the motion transient intensity map, thereby reducing the impact of isolated noise points on subsequent block-level connectivity aggregation decisions.

[0057] The motion transient intensity map obtained after processing by the Poisson pulse neural network has effectively extracted the motion transient features in the scene. However, directly making recording control decisions based on pixel-level transient energy values ​​is easily interfered with by sporadic noise points, leading to frequent false triggers or missed detections. Real moving targets usually have a certain spatial size, which is represented as continuous high-energy regions in the motion transient intensity map, while noise points often exhibit isolated and scattered distribution characteristics. This invention adopts a block-level connected component aggregation decision strategy. First, the motion transient intensity map is divided into image blocks for coarse-grained analysis. Then, spatially continuous motion regions are identified through connected component aggregation. Finally, recording control decisions are made based on the size of the motion region, thereby effectively suppressing false triggers caused by noise while maintaining detection sensitivity. Dividing the motion transient intensity map into multiple fixed-size image blocks is the basic step for implementing block-level analysis. The size selection of image blocks needs to strike a balance between detection accuracy and noise resistance. Smaller image blocks can retain more spatial details and have a stronger detection capability for small-sized moving targets, but they are also more susceptible to local noise. Larger image blocks can effectively suppress random noise through spatial averaging, but may lead to a decrease in the detection capability of small targets or inaccurate target boundary localization.

[0058] The fixed size of image patches is typically set to 8 x 8 pixels, 16 x 16 pixels, or 32 x 32 pixels. In a preferred configuration, the fixed size of the image patches is set to 16 x 16 pixels. Let the resolution of the motion transient intensity map be... ,in This represents the image width, which is the number of pixels in the horizontal direction. This represents the image height, i.e., the number of pixels in the vertical direction. The fixed size of the image patch is... pixels, of which Let represent the side length of the image patch. Then the number of image patches that can be divided horizontally is: The number of image blocks that can be divided vertically is ,in This indicates the rounding up operation. Taking a motion transient intensity map with a resolution of 640 x 480 pixels and an image block size of 16 x 16 pixels as an example, the image is divided into 40 blocks horizontally and 30 blocks vertically, for a total of 1200 image blocks.

[0059] For regions that cannot be completely divided at image boundaries, two processing strategies can be adopted. One strategy is to treat regions at the boundary that are smaller than the size of a complete image patch as independent image patches, but normalize them according to the actual number of pixels they contain when calculating the block energy value; the other strategy is to fill the boundaries of the motion transient intensity map so that its size is exactly an integer multiple of the image patch size, and the fill value is usually set to zero. In a preferred embodiment, the first strategy is adopted to avoid the influence of the filled region on the detection results. After the image patch division is completed, the block energy value of each image patch is calculated as a measure of the motion intensity of that region. The block energy value is calculated as follows: for each image patch, all pixels within its coverage area are traversed, the transient energy value of each pixel is read, all transient energy values ​​are accumulated and divided by the total number of pixels to obtain the arithmetic mean, which is the block energy value of the current image patch. The calculation of the block energy value can be expressed as follows: ,in This represents the block energy value of the current image patch. This represents the set of pixel locations covered by the current image patch. This indicates the total number of pixels contained in the current image patch. Indicates pixel position The transient energy value at a given point is summed, and the operation iterates through all pixel positions within the current image patch's coverage area. For a complete 16x16 pixel image patch, The value is 256; for incomplete image patches at the boundaries, The value represents the actual number of pixels contained. The reason for using the arithmetic mean instead of a sum as the block energy value is that image blocks at different locations may contain different numbers of pixels, especially those at boundaries. Using the arithmetic mean eliminates the influence of pixel count differences on the block energy value, making the block energy values ​​of different image blocks comparable. After calculating the block energy values ​​of all image blocks, it is necessary to determine which image blocks contain valid motion information. This determination is achieved by comparing the block energy values ​​with a preset motion threshold. The preset motion threshold is a key parameter for distinguishing moving areas from static backgrounds, and its value needs to be set according to the characteristics of the monitoring scene and the required detection sensitivity.

[0060] Setting a preset motion threshold requires consideration of several factors. In typical applications of photon-counting single-photon avalanche diode array sensors, due to the sensor's extremely high sensitivity, a certain degree of noise response may exist even in static background areas. The preset motion threshold needs to be set above the background noise level to avoid misinterpreting noise as motion. However, the preset motion threshold should not be set too high, otherwise weak motion signals may be missed. In a preferred configuration, the preset motion threshold is set to a value between 0.05 and 0.2, and the specific value can be determined through calibration in a target monitoring scenario. A commonly used calibration method is to acquire several frames of images in a static scene without moving targets, calculate the block energy value of each image block, and take the mean plus three times the standard deviation as the preset motion threshold. This setting method can eliminate the influence of background noise with high confidence.

[0061] Image blocks with block energy values ​​greater than a preset motion threshold are marked as moving blocks, while those with block energy values ​​less than or equal to the preset motion threshold are marked as stationary blocks. After this step, the motion transient intensity map is converted into a binary image block label map, where the label value for moving blocks is 1 and the label value for stationary blocks is 0. Individual moving block labels are insufficient to determine the existence of a valid moving target, as isolated noise points may also generate a small number of scattered moving blocks. Real moving targets, due to their spatial dimensions, typically have moving blocks that are spatially adjacent, forming continuous regions. This invention uses eight-neighbor connected component labeling to aggregate adjacent moving blocks into moving regions, thereby distinguishing between continuous moving target responses and isolated noise responses.

[0062] Eight-neighbor connectivity refers to the adjacency relationship between an image patch and its eight directly adjacent image patches in eight directions, including the four positive directions (top, bottom, left, and right) and the four diagonal directions (top left, top right, bottom left, and bottom right). Let the position of the current image patch in the image patch array be... ,in Represents the row index of the image patch. The column index representing the image patch then includes its eight neighbors, which are the location. , , , , , , , There are a total of 8 adjacent image blocks. If two moving blocks are adjacent to each other under the eight-neighbor relation, they are considered to belong to the same connected region.

[0063] The connected component labeling is implemented using a two-pass scanning algorithm. The first scan starts from the top left corner of the image block array and checks each image block sequentially from left to right and top to bottom. For an image block labeled as a moving block, it checks if any of its previously scanned neighbors (left, top, top left, and top right) contain any already labeled moving blocks. If no such moving block exists in the neighborhood, a new connected component label is assigned to the current moving block; if a labeled moving block exists, its connected component label is assigned to the current moving block; if multiple moving blocks with different connected component labels exist in the neighborhood, the current moving block is labeled with the smallest connected component label, and the equivalence relationships between these different labels are recorded. The second scan, based on the equivalence relationships recorded in the first scan, merges all equivalent connected component labels into a single label, ensuring that each connected moving region has a unique identifier.

[0064] After completing the eight-neighbor connectivity labeling, all adjacent motion blocks are merged into the same motion region, each with a unique region identifier. The number of motion blocks contained in each motion region is counted as a measure of its size. The size of the motion region is a key criterion for determining whether to trigger recording. Larger motion regions usually correspond to real moving targets, while smaller motion regions may be caused by noise or minor disturbances. By setting a preset threshold, small-scale pseudo-motion regions caused by noise can be effectively filtered out, triggering recording only for moving targets of a certain size. The preset threshold represents the minimum size of the motion region required to trigger recording, expressed in units of motion blocks. Setting the preset threshold requires comprehensive consideration of the characteristics of the monitoring scene, the minimum target size to be detected, and the acceptable false trigger rate. In a preferred configuration, the preset threshold is set to 3 to 10 motion blocks. Taking a 16x16 pixel image block size and a preset threshold of 5 as an example, the minimum motion region area required to trigger recording is 5 x 16 x 16, which equals 1280 square pixels. For an image with a resolution of 640 by 480 pixels, this area accounts for approximately 0.42 percent of the total image area, enabling the detection of small moving targets while suppressing false triggering by point noise.

[0065] The recording trigger decision process is as follows: All detected motion regions are traversed, and the number of motion blocks contained in each motion region is checked to see if it exceeds a preset threshold. When the number of motion blocks in any motion region exceeds the preset threshold, a valid moving target is determined to exist in the scene, triggering the recording device to start recording. Upon receiving the trigger signal, the recording device immediately initiates the video encoding and storage process, writing the current and subsequent video frames to the storage medium.

[0066] After recording begins, the video recording device needs to continuously monitor the motion state of the scene to determine when to stop recording. Continuing to record after the scene returns to stillness will waste storage space and cause inconvenience for subsequent video retrieval. This invention achieves automatic recording stop determination by establishing a sliding detection window and calculating global motion intensity. During recording, the device continuously executes three steps: photon event acquisition and inter-frame difference sequence generation, variational mode decomposition to separate illumination and motion components, and Poisson pulse neural network processing to generate a motion transient intensity map, thus obtaining the latest motion transient intensity map. For each frame of the motion transient intensity map, its global motion intensity is calculated as a measure of the overall motion level. The global motion intensity is calculated as follows: traverse all pixels in the motion transient intensity map, read the transient energy value of each pixel, sum all transient energy values, divide by the total number of pixels, and obtain the arithmetic mean. This arithmetic mean is the global motion intensity of the current frame. The global motion intensity can be expressed as... ,in This represents the global motion intensity of the current frame. This represents the width of the transient intensity map. Indicates the height of the transient intensity map of motion. Indicates pixel position The transient energy value at a given point is calculated by performing a double summation operation across all pixel positions in the image.

[0067] Global motion intensity reflects the overall motion activity of the current frame; a higher value indicates more intense motion or a larger motion area in the scene, while a lower value indicates a more static scene. However, the global motion intensity of a single frame may fluctuate due to transient noise and should not be directly used as a criterion for stopping recording. This invention establishes a sliding detection window to perform temporal smoothing of the global motion intensity to obtain more stable static determination results.

[0068] The sliding detection window is a time window containing a preset number of frames, used to store the global motion intensity values ​​of the most recent frames. The preset number of frames determines the length of the sliding detection window, typically set to 10 to 30 frames. In a preferred configuration, the preset number of frames is set to 15 frames. At a capture frame rate of 25 frames per second, 15 frames correspond to a time span of 0.6 seconds. The sliding detection window uses a first-in, first-out (FIFO) update strategy. Each new global motion intensity value is added to the end of the window, while the value at the front of the window is moved out, ensuring that the window always contains the global motion intensity values ​​of the most recent preset number of frames. The recording stop decision is based on the temporal evolution characteristics of the global motion intensity within the sliding detection window. The preset stillness threshold is the boundary for distinguishing between motion and stillness. Its value is typically set to one-half to three-quarters of the preset motion threshold to ensure that stillness is only determined after motion has significantly decreased, avoiding premature recording stoppage due to misjudging a short pause in motion. In a preferred configuration, the preset stillness threshold is set to a value between 0.02 and 0.1.

[0069] The recording stop decision process is as follows: After each update of the sliding detection window, it is checked whether the global motion intensity of all consecutive frames within the window is less than a preset static threshold. This decision uses a strict all-condition requirement, meaning that the global motion intensity of every frame within the sliding detection window must be less than the preset static threshold. If the global motion intensity of any frame is greater than or equal to the preset static threshold, the scene is considered not completely still, and recording continues. Only when the global motion intensity of all consecutive frames within the sliding detection window is less than the preset static threshold is the monitored scene determined to be in a static state, and the recording device is controlled to stop recording.

[0070] The strict judgment method, which requires all consecutive frames to meet the condition, rather than the lenient method that relies on the average within a window or a majority of frames, is used to ensure that recording is not prematurely stopped due to misjudging a moving target's brief stillness or slowing motion as a static scene. In actual monitoring scenarios, moving targets may pause briefly before resuming movement. If a lenient judgment method is used, recording might stop during the target's pause, missing the subsequent motion. While the strict judgment method may result in a delay of several frames after the scene truly comes to a standstill, it ensures the complete recording of the moving target's entire activity process.

[0071] After the recording device stops recording, the system returns to standby monitoring mode and continues the motion detection process, waiting for the next motion event to be triggered. To avoid storing too many fragmented video clips, a minimum recording duration can be set in the optional implementation. Even if the scene quickly returns to stillness after recording is triggered, recording is guaranteed to continue for at least the preset minimum duration before being allowed to stop. The minimum recording duration is typically set to 3 to 10 seconds.

[0072] The preferred embodiments of this disclosure have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of this disclosure. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of this disclosure shall be within the scope of the claims of this disclosure.

Claims

1. An AI-based method for motion detection and recording control, characterized in that, Includes the following steps: Step 1: Photon event acquisition and inter-frame differential sequence generation: A photon counting sensor array is used to continuously acquire frame images of the monitored scene. Differential operations are performed on adjacent frame images to obtain inter-frame differential images. The inter-frame differential images of multiple consecutive frames are arranged in chronological order to form an inter-frame differential sequence. Step 2: Variational mode decomposition to separate illumination and motion components: Perform variational mode decomposition on the inter-frame difference sequence obtained in Step 1 to separate the low-frequency illumination slowly varying component and the mid-to-high frequency real motion component. Reconstruct the mid-to-high frequency real motion component according to pixel position to form a motion difference image sequence. Step 3: Processing the motion transient intensity map using a Poisson spiking neural network: Input the motion difference image sequence obtained in Step 2 into a Poisson spiking neural network for processing. After passing through a Poisson pulse encoding layer, a leakage current integral firing neuron layer, and a spatiotemporal convolutional spiking neuron layer, the output pulse firing rate is processed to generate a motion transient intensity map by performing transient energy calculation. Step 4, Block-level Connectivity Aggregation Decision and Recording Control: Divide the motion transient intensity map obtained in Step 3 into image blocks and calculate the block energy value. Aggregate the connected components of motion blocks whose block energy values ​​exceed the preset motion threshold to form motion regions. When the number of motion blocks contained in any motion region exceeds the preset number threshold, trigger the recording device to start recording. When the global motion intensity in the sliding detection window is continuously lower than the preset static threshold, control the recording device to stop recording.

2. The AI-based motion detection and recording control method according to claim 1, characterized in that, In step one, the photon counting sensor array is a photon counting single-photon avalanche diode array sensor. Each pixel unit of the photon counting single-photon avalanche diode array sensor independently detects incident photons and outputs a photon count value. The photon count values ​​of each pixel unit within the continuous acquisition period are combined to form a frame image. The difference operation is an operation that subtracts the absolute value of each pixel from the images of two adjacent frames.

3. The AI-based motion detection and recording control method according to claim 1 or 2, characterized in that, In step two, the specific process of variational mode decomposition is as follows: variational mode decomposition is performed on the time series of each pixel position in the inter-frame difference sequence, and the time series of each pixel position is decomposed into multiple intrinsic mode components. Based on the center frequency of each intrinsic mode component, the intrinsic mode components with a center frequency lower than the preset frequency limit are classified as low-frequency illumination slowly varying components, and the intrinsic mode components with a center frequency higher than or equal to the preset frequency limit are classified as mid-to-high frequency real motion components.

4. The AI-based motion detection and recording control method according to claim 1, characterized in that, In step three, the specific process of Poisson pulse coding is as follows: For each pixel of each frame of motion difference image in the motion difference image sequence, the gray value of the current pixel is used as the firing rate parameter of the Poisson process. The preset coding time window is evenly divided into multiple time steps. In each time step, a random number uniformly distributed between zero and one is generated. When the random number is less than the product of the current pixel gray value and the time step length, a pulse is generated in the current time step. When the random number is greater than or equal to the product of the current pixel gray value and the time step length, no pulse is generated in the current time step. In this way, the gray value of each pixel is converted into a binary pulse sequence composed of multiple time steps.

5. The AI-based motion detection and recording control method according to claim 4, characterized in that, The specific process of the leakage integral firing neuron layer is as follows: An array of leakage integral firing neurons with the same number of pixels as the motion difference image is constructed. Each leakage integral firing neuron receives a binary pulse sequence at the corresponding pixel position as input and maintains a membrane potential state variable. At the beginning of each time step, the current membrane potential state variable is first multiplied by a fixed attenuation coefficient less than one to achieve membrane potential leakage. Then, it is determined whether an input pulse is received at the current time step. If an input pulse is received, a fixed synaptic input increment is added to the membrane potential state variable; if no input pulse is received, the membrane potential state variable retains the leaked value. Subsequently, the updated membrane potential state variable is compared with a preset firing threshold. When the membrane potential state variable is greater than or equal to the firing threshold, the current leakage integral firing neuron generates an output pulse at the current time step and resets the membrane potential state variable to zero. When the membrane potential state variable is less than the firing threshold, the current leakage integral firing neuron does not generate an output pulse at the current time step. After traversing all time steps within the encoding time window, each leakage integral firing neuron outputs an output pulse sequence of the same length as the input binary pulse sequence.

6. The AI-based motion detection and recording control method according to claim 5, characterized in that, The specific process of the spatiotemporal convolutional spiking neuron layer is as follows: a spatiotemporal convolutional spiking neuron layer is set after the leakage integral firing neuron array. For each neuron position in the leakage integral firing neuron array, all adjacent neurons within the coverage area of ​​the spatial convolution kernel of a preset size centered on the current neuron are determined. In each time step, the total number of output pulses generated by all neurons within the coverage area of ​​the spatial convolution kernel in the current time step is counted. The total number of output pulses is used as the input of the spatiotemporal convolutional spiking neuron corresponding to the current position in the current time step. The spatiotemporal convolutional spiking neuron is processed using the same membrane potential leakage, input accumulation, threshold comparison and reset process as the leakage integral firing neuron described in claim 5. The output of the spatiotemporal convolutional spiking neuron layer is a spiking neuron array of the same size as the input.

7. The AI-based motion detection and recording control method according to claim 6, characterized in that, The calculation process of the pulse firing rate is as follows: For each neuron output by the spatiotemporal convolutional spiking neuron layer, the total number of output pulses generated by the current neuron within the encoding time window is counted. The total number of output pulses is divided by the total number of time steps contained in the encoding time window to obtain the pulse firing rate of the current neuron. The pulse firing rates of all neurons are arranged according to their spatial positions to form a pulse firing rate image.

8. The AI-based motion detection and recording control method according to claim 7, characterized in that, The specific process of transient energy calculation is as follows: For each pixel position in the pulse firing rate image, obtain the three pulse firing rate values ​​corresponding to the current position in the current frame, the previous frame, and the next frame. Multiply the pulse firing rate value of the current frame by itself to obtain the first intermediate value. Multiply the pulse firing rate value of the previous frame by the pulse firing rate value of the next frame to obtain the second intermediate value. Subtract the second intermediate value from the first intermediate value to obtain the transient energy value of the current pixel position. When the first intermediate value is less than the second intermediate value, set the transient energy value of the current pixel position to zero. Combine the transient energy values ​​of all pixel positions to form a motion transient intensity map.

9. The AI-based motion detection and recording control method according to claim 1, characterized in that, In step four, the specific process of connected component aggregation is as follows: the motion transient intensity map is divided into multiple image blocks of fixed size, the arithmetic mean of the transient energy values ​​of all pixels in each image block is calculated as the block energy value, the image blocks with block energy values ​​greater than a preset motion threshold are marked as motion blocks, all motion blocks are marked with eight-neighbor connected components, and adjacent motion blocks are merged into the same motion region.

10. The AI-based motion detection and recording control method according to claim 9, characterized in that, In step four, while the recording device is in recording mode, steps one to three are continuously executed to obtain the motion transient intensity map. The arithmetic mean of the transient energy values ​​of all pixels in the motion transient intensity map is calculated as the global motion intensity. A sliding detection window containing a preset number of frames is established. When the global motion intensity of all consecutive frames in the sliding detection window is less than the preset static threshold, the recording device is controlled to stop recording.

Images