Motion detection early warning system and method based on avian optic tectum motion encoding mechanism

Abstract

The application discloses a kind of motion detection early warning system and method based on bird optic tectum motion encoding mechanism, including video acquisition device, data processing device and display device;Video acquisition device is used to collect video and is converted into digital signal and sent to data processing device;Data processing device is used to calculate the luminance information of each pixel point and obtain the motion response information of each pixel point in each frame video image, obtain the motion response information and motion direction information of enhanced motion target;Data processing device obtains the final response coefficient and integrates after enhanced motion response information, and outputs the position information of motion target in video;Display device is used to identify and display the position information of motion target.The application can more accurately and reliably detect and warn the motion target in natural image background and real natural video.

Description

Technical Field

[0001] This invention relates to the field of motion detection and early warning, and in particular to a motion detection and early warning system and method based on the motion coding mechanism of bird optic canopy. Background Technology

[0002] Moving target detection is an innate instinct for birds, determining various aspects of their lives, including predator avoidance and foraging. Neuroanatomical studies have shown that moving target detection in birds against complex backgrounds is primarily accomplished by the optic tectum visual pathway. The sensitivity to target movement and background-independent robustness benefit from processing by the optic tectum nucleus. Inspired by this, several motion detection models based on Fourier transforms and winner-takes-all competitive networks have been proposed, capable of achieving moving target detection in specific environments. However, their performance in moving target detection against complex or moving backgrounds is often unsatisfactory. Summary of the Invention

[0003] The purpose of this invention is to provide a motion detection and early warning system and method based on the motion coding mechanism of bird visual canopy, which can more accurately and reliably detect and warn of moving targets in natural image backgrounds and real natural videos.

[0004] The present invention adopts the following technical solution:

[0005] A motion detection and early warning system based on the motion coding mechanism of bird visual canopy includes a video acquisition device, a data processing device, and a display device;

[0006] A video capture device is used to capture video and convert the captured video into digital signals, which are then sent to a data processing device.

[0007] The data processing device is used to calculate the brightness information of each pixel in each frame of video image through a brightness perception module, and then obtain the motion response information of each pixel in each frame of video image through a motion detection module. The motion response information includes motion response information for both brightness increase and brightness decrease. Subsequently, the spatiotemporal accumulation module is used to obtain the enhanced motion response information and motion direction information of the moving target by using the motion response information of pixels with spatiotemporal correlation in different frames of video image. The data processing device also obtains the final response coefficient for adjusting the enhanced motion response information through a peripheral modulation module. The final response coefficient includes a motion direction contrast coefficient and a brightness contrast coefficient. After the information fusion module integrates the enhanced motion response information and the final response coefficient, it outputs the position information of the moving target in the video.

[0008] The display device is used to identify and display the position information of the moving target output by the data processing device.

[0009] The data processing device includes a brightness sensing module, a motion detection module, a spatiotemporal accumulation module, a peripheral modulation module, and an information fusion module;

[0010] The brightness perception module is used to calculate the brightness information of each pixel in each frame of the input video and to smooth the brightness information.

[0011] The motion detection module is used to calculate the ON-OFF signal of the brightness change of each pixel in each frame of video image, and perform delay processing on the ON signal and OFF signal after separating them; then, the delay of the obtained ON signal and the delay of the OFF signal are convolved with the OFF signal and ON signal of the same pixel in the same frame of video image, and the convolution results are added together to finally obtain the motion response information at the corresponding pixel position.

[0012] The spatiotemporal accumulation module is used to accumulate the motion information of spatiotemporally correlated pixels in different frames of video images in four motion directions: up, down, left, and right. This yields motion information values ​​in each of the four directions, which are the enhanced motion response information. Finally, the largest motion information value is selected as the motion direction information at that pixel location, and the corresponding direction is the motion direction of the moving target at that pixel location. If the motion information values ​​in each direction are all below a set motion information threshold E... e If the value is zero, it is considered that there is no moving target at the location of that pixel.

[0013] The peripheral modulation module is used to obtain the final response coefficient of the enhanced motion response information through a division normalization operation; the final response coefficient is composed of motion direction contrast coefficient and brightness contrast coefficient with different weights;

[0014] The information fusion module integrates the enhanced motion response information output by the spatiotemporal accumulation module and the final response coefficients output by the peripheral modulation module. It convolves the enhanced motion response information and the final response coefficients from the same pixel location to obtain a fusion value. The maximum value in the fusion value is used as the fusion comparison value. Then, the fusion comparison value is compared with the set motion detection threshold. If the fusion comparison value is greater than the motion detection threshold, it is considered that there is a moving target in the video image of that frame.

[0015] When a moving target is present in the video frame, the information fusion module also uses the pixel position corresponding to the corresponding fusion comparison value as the reference pixel, calculates the fusion comparison value corresponding to each pixel around the reference pixel, and determines whether it is greater than the set comparison threshold of the reference pixel's fusion comparison value. Then, the positions of all pixels corresponding to the fusion comparison values ​​greater than the set comparison threshold and the reference pixel are used as the location of the moving target.

[0016] When the spatiotemporal accumulation module acquires motion information values ​​in four motion directions, the pixel position (x) m ,y q ,t k The motion information values ​​in each direction of motion are shown below:

[0017] The motion information value in the rightward direction is:

[0018]

[0019] The motion information value in the leftward direction is:

[0020]

[0021] The motion information value in the downward direction is:

[0022]

[0023] The motion information value in the upward direction is:

[0024]

[0025] Where, x m ,y q ,t k ,x m±i ,y q±i ,t k-i The subscript values ​​represent the position coordinates of the pixel, and M and Q represent the size of each video frame; W i D(x,y,t) represents the cumulative weight value of the current video image frame; S ON (x,y,t)·S D-OFF (x,y,t)+S OFF (x,y,t)·S D-ON (x,y,t), S ON (x,y,t) and S OFF (x,y,t) represent the positive and negative numbers of L(x,y,t) respectively, where L(x,y,t) is the response output of the brightness change in the motion detection module, and S D-ON (x,y,t) represents the delay of the ON signal, S D-OFF (x,y,t) represents the delay of the OFF signal.

[0026] In the aforementioned spatiotemporal accumulation module, the largest motion information value is selected as the motion direction information at that pixel location, and the motion direction corresponding to the largest motion information value is recorded as the motion direction of the moving target at that pixel location; if the motion information values ​​in all four directions are lower than the set motion information threshold E...e If the value is zero, it is considered that there is no moving target at the location of that pixel.

[0027] Pixel (x) m ,y q ,t k The enhanced motion response information at the location is:

[0028]

[0029] When dir = 0, it indicates that there is no moving target at the pixel location; dir = 1, 2, 3, 4 indicate that the movement direction is right, left, down, and up, respectively; d represents the pixel (x m ,y q ,t k The direction of movement at the location.

[0030] The peripheral modulation module includes a motion direction modulation module;

[0031] The motion direction modulation module is used to obtain the pixel position (x) m ,y q ,t k The motion direction contrast coefficient R dir (x m ,y q ,t k ):

[0032]

[0033] Where, N dir (x m ,y q ,t k ) represents a pixel (x) m ,y q ,t k The frequency of the direction of motion at a given location in the current video image. This represents the sum of the frequencies of all directions of motion. There are 5 groups of directions, where d = 0 indicates no motion, and d = 1, 2, 3, and 4 indicate the directions of motion as right, left, down, and up, respectively.

[0034] The peripheral modulation module further includes a brightness modulation module;

[0035] The brightness modulation module is used to obtain the pixel position (x) m ,y q ,t k The brightness contrast factor R lum (x m ,y q ,t k ):

[0036]

[0037] Where, N lum (x m ,y q ,t k ) represents a pixel (x) m ,y q ,t k The frequency of the brightness value at a given location in the current video image. This represents the sum of the frequencies of all brightness values.

[0038] The final output of the peripheral modulation module includes the motion direction contrast coefficient R. dir (x m ,y q ,t k ) and brightness contrast factor R lum (x m ,y q ,t k The final response coefficient is:

[0039] R(x m ,y q ,t k ) = w dir *R dir (x m ,y q ,t k )+(1-w dir )*R lum (x m ,y q ,t k );

[0040] Where R(x) m ,y q ,t k ) represents the pixel (x) in the peripheral modulation module. m ,y q ,t k The final response coefficient at location w dir The weighting coefficients representing the comparison of motion directions, 1-w dir This represents the weighting coefficient for brightness contrast.

[0041] The fused value output by the information fusion module is represented as follows:

[0042] S(x m ,y q ,t k )=F(x m ,y q ,t k )·R(x m,y q ,t k );

[0043] Where, F(x) m ,y q ,t k R(x) represents the enhanced motion response information output by the spatiotemporal accumulation module. m ,y q ,t k ) represents the final response coefficient output by the peripheral modulation module.

[0044] The early warning method of a motion detection and early warning system based on the motion coding mechanism of bird visual tectum includes the following steps:

[0045] A: Use a video capture device to acquire real-time video;

[0046] B: The brightness perception module is used to calculate the brightness information of each pixel in each frame of video image, and the brightness information is smoothed.

[0047] C: The motion detection module calculates the ON-OFF signal of brightness change for each pixel in each frame of video image, and performs delay processing on the ON and OFF signals after separation; then, the delays of the obtained ON and OFF signals are convolved with the OFF and ON signals of the same pixel in the same frame of video image, respectively, and the convolution results are added to finally obtain the corresponding pixel (x). m ,y q ,t k Motion response information D(x) at the location m ,y q ,t k );

[0048] D: Using a spatiotemporal accumulation module, the motion information of spatiotemporally correlated pixels in different video frames is accumulated in four motion directions: up, down, left, and right, to obtain motion information values ​​in each direction. Finally, the largest motion information value is selected as the motion direction information at that pixel location, and the corresponding direction is the motion direction of the moving target at that pixel location. If the motion information values ​​in each direction are all below a set motion information threshold E... e If no moving target is found at the pixel location, then it is assumed that there is no moving target at that pixel location; finally, the corresponding pixel (x) is obtained. m ,y q ,t k Enhanced motion response information F(x) at position m ,y q ,t k ,d);

[0049] E: Using the motion direction modulation module and brightness modulation module in the peripheral modulation module, the corresponding pixel points (x) are obtained respectively. m ,y q ,t k The contrast coefficient R of the direction of motion at the position dir (x m ,y q ,t k ) and brightness contrast factor R lum (x m ,y q ,t k The final response coefficient is obtained as R(x). m ,y q ,t k );

[0050] F: Using the information fusion module, the data from pixel (x) m ,y q ,t k Enhanced motion response information R at the location dir (x m ,y q ,t k ) and final response coefficient R(x m ,y q ,t k Convolution is performed to obtain the fused value S(x) m ,y q ,t k The maximum value among the fused values ​​is used as the fusion comparison value. Then, the fusion comparison value is compared with the set motion detection threshold. If the fusion comparison value is greater than the motion detection threshold, it is considered that there is a moving target in the video frame.

[0051] G: Using the information fusion module, the pixel position corresponding to the fusion comparison value is used as the reference pixel. The fusion comparison value of each pixel around the reference pixel is calculated, and it is determined whether it is greater than 80% of the fusion comparison value of the reference pixel. Then, the positions of all pixels corresponding to the fusion comparison values ​​greater than 80% and the reference pixel are used as the positions of the moving target. The positions of the moving target are then sent to the display device.

[0052] H: The location information of the moving target is marked and displayed using a display device, and an alarm module is used to provide a reminder.

[0053] This invention calculates the brightness information of each pixel in each frame of a video image using a brightness perception module. Then, a motion detection module acquires motion response information for both brightness increases and decreases in each pixel within each frame. Subsequently, a spatiotemporal accumulation module uses the motion response information of spatiotemporally correlated pixels across different video frames to obtain enhanced motion response information and motion direction information for the moving target. Simultaneously, a peripheral modulation module obtains the final response coefficients used to adjust the enhanced motion response information. Finally, an information fusion module integrates the enhanced motion response information with the final response coefficients to output the position information of the moving target in the video. Based on the design concept of spatiotemporal accumulation-dynamic modulation, this invention can detect moving targets more accurately and reliably in natural image backgrounds and real natural videos. This invention requires no training samples and is simple to implement, enabling accurate and effective detection of moving targets in security systems, achieving an early warning effect. Attached Figure Description

[0054] Figure 1 This is a schematic diagram of the motion detection and early warning system based on the bird visual canopy motion coding mechanism in this invention.

[0055] Figure 2 This is a flowchart illustrating the motion detection and early warning method based on the bird optic canopy motion coding mechanism in this invention.

[0056] Figure 3 This is a schematic diagram illustrating the signal processing principle of the motion detection module in this invention. Detailed Implementation

[0057] The present invention will now be described in detail with reference to the accompanying drawings and embodiments:

[0058] like Figure 1 As shown, the motion detection and early warning system based on the bird optic canopy motion coding mechanism of the present invention includes a video acquisition device, a data processing device, and a display device;

[0059] A video capture device is used to capture video and convert the captured video into digital signals, which are then sent to a data processing device.

[0060] The data processing device is used to calculate the brightness information of each pixel in each frame of video image through a brightness perception module, and then obtain the motion response information of each pixel in each frame of video image through a motion detection module. The motion response information includes motion response information for both brightness increase and brightness decrease. Subsequently, the spatiotemporal accumulation module is used to obtain the enhanced motion response information and motion direction information of the moving target by using the motion response information of pixels with spatiotemporal correlation in different frames of video image. The data processing device also obtains the final response coefficient for adjusting the enhanced motion response information through a peripheral modulation module. The final response coefficient includes a motion direction contrast coefficient and a brightness contrast coefficient. After the information fusion module integrates the enhanced motion response information and the final response coefficient, it outputs the position information of the moving target in the video.

[0061] A display device is used to identify and display the position information of a moving target output by a data processing device; the display device can be an existing display screen.

[0062] In this invention, an alarm module can also be added in conjunction with the display device to warn of any moving targets. The alarm module can be an existing audible and visual alarm device.

[0063] In this invention, the video acquisition device includes a video sensing module and a data transmission module. The video sensing module converts the light signals of the acquired video images into digital signals, and the data transmission module transmits the digital signals output by the video sensing module to a data processing device. In this embodiment, the video sensing module can be a camera or a monitor with video recording capabilities; the data transmission module can be a wired or wireless transmission device, such as a network cable; or a wireless transmitter or Bluetooth transmitter.

[0064] In this invention, the data processing device includes a brightness sensing module, a motion detection module, a spatiotemporal accumulation module, a peripheral modulation module, and an information fusion module;

[0065] The brightness perception module is used to calculate the brightness information of each pixel in each frame of the input video and to smooth the brightness information.

[0066] The brightness perception module is used to simulate the neurons in the retinal layer of birds. It can be represented by a matrix of M×Q photoreceptors, where M and Q represent the size of each frame of video image. Each photoreceptor receives the brightness information of the corresponding pixel in the video image and performs Gaussian blur (GB) processing to simulate the brightness perception and Gaussian processing mechanism of the retina.

[0067] In this embodiment, the brightness perception module can employ a spatial domain Gaussian filter to smooth the brightness signal of each pixel in each frame of the video image. The video image sequence constituting the video is represented as... The output P(x,y,t) of the brightness sensing module is the input I(x,y,t) and the Gaussian function. Convolution;

[0068]

[0069]

[0070] Where x and y represent the horizontal and vertical coordinates of a pixel in the video image, and t represents time. Let (u,v) represent the set of real numbers, (x,y) represent the variable (x,y) in the integral with variable limits, and σ1 be the standard deviation of the Gaussian function.

[0071] The motion detection module is used to calculate the ON-OFF signal of brightness change for each pixel in each frame of video image, and perform delay processing on the ON signal and OFF signal after separating them; then, the delay of the obtained ON signal and the delay of the OFF signal are convolved with the OFF signal and ON signal of the same pixel in the same frame of video image, and the convolution results are added together to finally obtain the motion response information at the corresponding pixel position; in this invention, the motion response information includes motion response information for both brightness increase and brightness decrease.

[0072] The motion detection module simulates the superficial neurons of the optic tectum in birds, directly receiving signals from the retinal layer and exhibiting a strong response to decreases (OFF) and increases (ON) in brightness. Within the motion detection module, a time bandpass filter (HPF) is used to calculate the neuron's response to changes in brightness.

[0073] In this embodiment, considering the advantages of gamma kernels in time processing, such as trivial stability, easy self-adaptation, and decoupling of impulse response support region and order, the difference between two gamma kernels is used to define the impulse response of the time bandpass filter H(t), i.e.

[0074]

[0075]

[0076] Among them, Γ n,τ (t) represents an nth-order gamma kernel function with a time constant τ, where the subscripts in n1, τ1, n2, and τ2 represent the first and second gamma kernels, respectively; then the response output L(x,y,t) of the brightness change in the motion detection module is:

[0077] L(x,y,t)=∫P(x,y,s)H(ts)ds;

[0078] Where s represents the variable t in the integral with variable limits;

[0079] Then, the ON and OFF signals are separated:

[0080] S ON (x,y,t)=[L(x,y,t)] + ;

[0081] S OFF (x,y,t)=[-L(x,y,t)] + ;

[0082] Among them, S ON (x,y,t) and S OFF (x,y,t) represent the positive and negative values ​​of L(x,y,t), respectively, and are called the ON and OFF signals, reflecting the increase and decrease in brightness of each pixel in each frame of the video image; [L(x,y,t)] + This means that L(x,y,t) takes a positive value when it is positive, and takes a value of 0 when it is not positive; [-L(x,y,t)] + This means that when L(x,y,t) is a positive number, the value is 0, and when L(x,y,t) is a non-positive number, the value is the absolute value of its value.

[0083] Subsequently, the motion detection module performs delay processing on the separated ON and OFF signals, i.e., gamma convolution, to obtain the delay of the ON signal. Delay of OFF signal for:

[0084]

[0085]

[0086] In this context, the subscripts in n3, τ3 and n4, τ4 represent the third and fourth gamma kernels, respectively, and s represents the variable t in the variable limit integral.

[0087] Then, the motion response information D(x,y,t) at pixel position (x,y,t) is:

[0088] D(x,y,t)=S ON (x,y,t)·S D-OFF (x,y,t)+S OFF (x,y,t)·S D-ON (x,y,t);

[0089] To understand the calculation strategy of the motion detection module simply and intuitively, such as Figure 3 As shown, taking a black moving target as an example, the signal processing process in the motion detection module is illustrated. When the black target passes through position (x0 y0), the brightness signal at pixel position (x0 y0) in the brightness sensing module changes over time as follows: Figure 3 As shown in step (a); the brightness change signal generated by the motion detection module in response to brightness changes is as follows: Figure 3 As shown in step (b), the brightness change is caused by the arrival and departure of the black target; therefore, the brightness change signal exhibits a trend of first decreasing and then increasing. The brightness change signal is then separated into ON and OFF signals as follows: Figure 3 As shown in step (c), the arrival and departure of the black target at pixel position (x0 y0) occur in different video frames, meaning the peaks of the ON and OFF signals appear in different video frames. To detect the passing of the moving target within the same video frame, we perform delay processing on the ON and OFF signals separately. Figure 3 Step (c) illustrates the ON and OFF signals, as well as the delayed OFF signal. The first row of the subgraph shows the ON signal, and in the second row, the long dashes represent the OFF signal, and the short dashes represent the delayed OFF signal. Figure 3 As can be seen, the peak value of the ON signal and the peak value of the delayed OFF signal appear in the same video frame. Finally, the delayed ON and OFF signals are convolved and added to the convolution result of the delayed OFF and ON signals to obtain the motion response information at pixel location (x0 y0). Figure 3 As shown in step (d). For a black moving target, the peak value of the OFF signal and the delay peak value of the OFF signal cannot appear in the same video frame, therefore in Figure 3 It will no longer be displayed.

[0090] The spatiotemporal accumulation module accumulates the motion information of spatiotemporally correlated pixels in different video frames in four motion directions: up, down, left, and right. This yields motion information values ​​for each direction, which are the enhanced motion response information. Finally, the largest motion information value is selected as the motion direction information for that pixel location, and the corresponding direction is the motion direction of the moving target at that pixel location. If the motion information values ​​in each direction are all below a set motion information threshold E... e If a pixel is not found to be moving, it is considered that there is no moving target at that pixel location. Pixels with spatiotemporal correlation refer to pixels where a moving target appears in several consecutive frames of video images.

[0091] In this invention, the spatiotemporal accumulation module is used to simulate the deep layers of the visual canopy of birds.

[0092] In this embodiment, the pixel position (x) m ,y q ,t k The motion information values ​​in each direction of motion are shown below:

[0093] The motion information value in the rightward direction is:

[0094]

[0095] The motion information value in the leftward direction is:

[0096]

[0097] The motion information value in the downward direction is:

[0098]

[0099] The motion information value in the upward direction is:

[0100]

[0101] Where, x m ,y q ,t k ,x m±i ,y q±i ,t k-i The subscript values ​​represent the position coordinates of the pixel, and M and Q represent the size of each video frame; W i This represents the cumulative weight value of the current video image frame.

[0102] Finally, the motion information value E(x,y,t,d) at the location of a single pixel in each frame of the video image is obtained, where d represents the direction. During the accumulation operation, the accumulation weight W... i It is represented by an exponential function with the mathematical constant e as the base and a range of (0, 1).

[0103] For a given pixel in each frame of video image, its motion information value includes four values ​​(representing the response in four motion directions). The largest motion information value is selected as the motion direction information at that pixel location, and the motion direction corresponding to the largest motion information value is recorded as the motion direction of the moving target at that pixel location. If the motion information values ​​in all four directions are lower than the set motion information threshold E, the motion information is not considered complete. e If the value is zero, it is assumed that there is no moving target at that pixel location.

[0104] Pixel (x) m ,yq ,t k The motion information value (enhanced motion response information) at the location is:

[0105]

[0106] When dir = 0, it indicates that there is no moving target at the pixel location; dir = 1, 2, 3, 4 indicate that the movement direction is right, left, down, and up, respectively; d represents the pixel (x m ,y q ,t k The direction of movement at the location.

[0107] The peripheral modulation module is used to obtain the final response coefficient of the enhanced motion response information through division normalization operation. The final response coefficient is composed of motion direction contrast coefficient and brightness contrast coefficient with different weights.

[0108] In this invention, the peripheral modulation module simulates the peripheral modulation mechanism of optic tectum neurons, that is, the neuron's response to stimuli in the receptive field is modulated by stimuli outside the receptive field. This modulation effect is positively correlated with the center-periphery contrast intensity and is also related to the target's state attributes (static or dynamic).

[0109] We model the peripheral modulation of the visual tectum as a process of mutual inhibition of similar detection features by neurons. That is, a detection feature that can induce neuronal excitation will inhibit the excitation of neurons by surrounding similar detection features. The response coefficient is achieved through a biologically plausible division normalization method. In the division normalization model, the neuronal response is described as the ratio between the neuron's excitatory input and the normalized signal; in other words, a neuron's response depends on the stimulus within its excitatory field and the overall response induced by all visual stimuli within its modulation field.

[0110] Therefore, the normalized response R of neuron j can be obtained. j Depends on its unnormalized input B k The standardization definition is:

[0111]

[0112]

[0113] Among them, molecule B j For a specific stimulus input to neuron j, generally it is a weighted sum of specific stimulus inputs, where w f I represents the weighted value of neuron j receiving the f-th stimulus. f This indicates a specific stimulus received by a neuron. This specific stimulus can come from external stimuli (such as vision, hearing, smell, and olfaction) or from other neurons, where I...f The subscript f represents the number of specific stimuli, γ represents a constant; the denominator is a constant σ and all input stimuli. The sum of, where B h This represents all stimulus inputs to neuron j, including specific stimuli and other stimuli.

[0114] In this invention, the peripheral modulation module includes a motion direction modulation module and a brightness modulation module;

[0115] The motion direction modulation module is used to obtain the pixel position (x) m ,y q ,t k The motion direction contrast coefficient R dir (x m ,y q ,t k ):

[0116]

[0117] Where, N dir (x m ,y q ,t k ) represents a pixel (x) m ,y q ,t k The frequency of the direction of motion at a given location in the current video image. This represents the sum of the frequencies of all directions of motion. There are 5 groups of directions, where d = 0 indicates no motion, and d = 1, 2, 3, 4 indicate the directions of motion as right, left, down, and up, respectively.

[0118] The brightness modulation module is used to obtain the pixel position (x) m ,y q ,t k The brightness contrast factor R lum (x m ,y q ,t k ):

[0119]

[0120] Where, N lum (x m ,y q ,t k ) represents a pixel (x) m ,y q ,t k The frequency of the brightness value at the given location in the current video image. This represents the sum of the frequencies of all brightness values, where there are 256 brightness values.

[0121] After the above division normalization operation, the maximum values ​​of both the motion direction contrast coefficient and the brightness contrast coefficient are 1. In this calculation method, if a certain feature appears frequently, the response coefficients of all pixels corresponding to that feature will be relatively small after division normalization. Similarly, if a certain feature appears infrequently, the response coefficients of all pixels corresponding to that feature will be relatively large after division normalization. Therefore, this process simulates the peripheral modulation effect in the optic tectum neurons.

[0122] Finally, to prevent weak signals that are almost imperceptible to biological vision from being excessively amplified during this processing, in this invention, features with a frequency of less than 10 in the motion direction are set to zero after division normalization. Therefore, the final output of the peripheral modulation module includes the motion direction contrast coefficient R. dir (x m ,y q ,t k ) and brightness contrast factor R lum (x m ,y q ,t k The final response coefficient is:

[0123] R(x m ,y q ,t k ) = w dir *R dir (x m ,y q ,t k )+(1-w dir )*R lum (x m ,y q ,t k );

[0124] Where R(x) m ,y q ,t k ) represents the pixel (x) in the peripheral modulation module. m ,y q ,t k The final response coefficient at location w dir This represents the weighting coefficient for comparing the directions of motion. 1-w dir Weighting coefficients representing brightness contrast;

[0125] In this embodiment, when motion information exists, the weighting coefficient for motion direction comparison is set to w. dir =0.8, then the brightness contrast weighting coefficient is 1-w dir =1 - 0.8 = 0.2. That is...

[0126]

[0127] This invention adjusts the enhanced motion response information using the final response coefficient output by the peripheral modulation module. According to the definition of the final response coefficient, the neural response induced when there is a contrast between the stimulus at the center of the receptive field and the peripheral stimulus is stronger than the neural response induced when the stimuli in both regions are identical. This is consistent with the modulation pattern of a single type of peripheral stimulus on the optic tectum in neurophysiological studies. Therefore, we need to prove that this calculation strategy conforms to the coupling and joint modulation patterns induced by brightness contrast and motion direction contrast.

[0128] 1) Dynamic peripheral modulation strategy for brightness contrast under static and motion conditions:

[0129] According to the definition of the modulation index, the relationship between the modulation index and the response coefficient can be obtained as follows:

[0130]

[0131] Among them, S oddball S represents the fusion motion information value when there is a contrast between the stimulus in the receptive field center and the peripheral stimulus. uniform This represents the fused motion information value when the stimulus at the center of the receptive field is consistent with the peripheral stimulus. R1 is the response coefficient when there is contrast between the stimulus at the center of the receptive field and the peripheral stimulus, and R2 is the response coefficient when the stimulus at the center of the receptive field is consistent with the peripheral stimulus. F1 and F2 are the outputs of the motion detection pathway when there is contrast and consistency between the stimulus at the center of the receptive field and the peripheral stimulus, respectively. Under these two stimulus conditions, the stimulus parameters (brightness and direction of motion) at the central site remain unchanged, therefore the output of the motion detection pathway F1 = F2. The above formula simplifies to:

[0132]

[0133] Under conditions where there is no motion information but only brightness contrast, the neuron's response modulation index CI L for:

[0134]

[0135] Among them, CI L R represents the modulation index under this condition. L1 and R L2 These represent the brightness contrast response coefficients when there is contrast and consistency between the stimulus at the center of the receptive field and the peripheral stimulus, respectively.

[0136] When all targets move in the same direction and at the same speed, and there is a brightness contrast, the neuron's response modulation index is:

[0137]

[0138] Among them, R m1 and R m2 R represents the motion direction contrast response coefficient when there is contrast and coherence between the stimulus at the center of the receptive field and the peripheral stimulus, respectively. Since all targets move in the same direction and at the same speed under these conditions, R... m1 =R m2 Simplifying the above equation, we get:

[0139]

[0140] Because R m1 >0, therefore

[0141]

[0142] Therefore, the luminance contrast modulation index under motion conditions is smaller than that under static conditions, which is consistent with the results of neuroelectrophysiology, indicating that the calculation strategy conforms to the dynamic peripheral modulation law of luminance contrast under static and motion conditions.

[0143] 2) Dynamic peripheral adjustment strategy based on a combination of brightness contrast and motion direction contrast

[0144] Under conditions of combined contrast of brightness and motion direction, the stimulation parameters (brightness and motion direction) at the central site remain unchanged; therefore, the output of the motion detection pathway is identical. From the definition of the peripheral modulation index, we can obtain:

[0145]

[0146] The simplified formula shows that, under the condition of combined contrast of brightness and motion direction, the greater the neuronal response induced by the contrast between the center and the periphery, the greater its modulation index. This is because the final model output is the product of the output of the motion detection pathway and the output of the peripheral modulation pathway, and the output of the motion detection pathway at the central site is the same. Therefore, the larger the response coefficient in the peripheral modulation pathway, the greater its modulation index. In this model, we set the weight of the response coefficient of motion contrast to be greater than the weight of the response coefficient of brightness contrast, so that the modulation index induced by motion direction contrast is greater than the modulation index induced by brightness contrast. This is consistent with the electrophysiological results of combined modulation by ambient brightness and ambient motion direction.

[0147] The information fusion module integrates the enhanced motion response information output by the spatiotemporal accumulation module and the final response coefficients output by the peripheral modulation module. Specifically, it convolves the enhanced motion response information and the final response coefficients from the same pixel location to obtain a fusion value. The maximum value in the fusion value is used as the fusion comparison value. Then, the fusion comparison value is compared with a set motion detection threshold. If the fusion comparison value is greater than the motion detection threshold, it is considered that there is a moving target in the video frame.

[0148] When a moving target is present in the video frame, the information fusion module also uses the pixel position corresponding to the corresponding fusion comparison value as the reference pixel, calculates the fusion comparison value corresponding to each pixel around the reference pixel, and determines whether it is greater than the set comparison threshold of the reference pixel's fusion comparison value. Then, the positions of all pixels corresponding to the fusion comparison values ​​greater than the set comparison threshold and the reference pixel are used as the location of the moving target.

[0149] In this embodiment, the comparison threshold is set to 80%;

[0150] The fused value output by the information fusion module is represented as follows:

[0151] S(x m ,y q ,t k )=F(x m ,y q ,t k )·R(x m ,y q ,t k );

[0152] Where F(x) m ,y q ,t k R(x) represents the enhanced motion response information output by the spatiotemporal accumulation module. m ,y q ,t k ) represents the final response coefficient output by the peripheral modulation module.

[0153] After this calculation, the fusion value of each frame of video image output by this invention is a two-dimensional matrix. In performance testing of computational models simulating insect vision, multiple target locations are typically detected in each frame. While this method achieves high accuracy, it also results in a particularly high error rate. Furthermore, this method does not conform to the functional characteristics of the pigeon optic tectum. Existing literature indicates that the pigeon optic tectum and isthmus together form a midbrain salience network, responsible for calculating the highest-priority stimuli and outputting them through the tectum. Here, we consider moving targets as the highest-priority stimuli in the visual field; therefore, only one target location is output in each frame.

[0154] Therefore, this invention includes a detection threshold; if the two-dimensional matrix... If the maximum value is greater than the set motion detection threshold, the video frame corresponding to that maximum value is considered to contain a moving target. Otherwise, the video frame is considered to contain no moving target.

[0155] If there is a moving target in the video frame, the position corresponding to the maximum value in the video frame is further determined as the target reference position, i.e., the reference pixel point, in the current frame. The area around the reference pixel point that is not lower than the set comparison threshold (80%) of the fusion comparison value at the reference pixel point is taken as the detected target area.

[0156] like Figure 2 As shown, the early warning method of the motion detection and early warning system based on the motion coding mechanism of bird optic canopy according to the present invention includes the following steps in sequence:

[0157] A: Use a video capture device to acquire real-time video;

[0158] B: The brightness perception module is used to calculate the brightness information of each pixel in each frame of video image, and the brightness information is smoothed.

[0159] C: The motion detection module calculates the ON-OFF signal of brightness change for each pixel in each frame of video image, and performs delay processing on the ON and OFF signals after separation; then, the delays of the obtained ON and OFF signals are convolved with the OFF and ON signals of the same pixel in the same frame of video image, respectively, and the convolution results are added to finally obtain the corresponding pixel (x). m ,y q ,t k Motion response information D(x) at the location m ,y q ,t k ), D(x,y,t)=S ON (x,y,t)·S D-OFF (x,y,t)+S OFF (x,y,t)·S D-ON (x,y,t);

[0160] Among them, S ON (x,y,t) and S OFF (x,y,t) represent the positive and negative numbers of L(x,y,t) respectively, where L(x,y,t) is the response output of the brightness change in the motion detection module, and S D-ON (x,y,t) represents the delay of the ON signal, S D-OFF (x,y,t) represents the delay of the OFF signal.

[0161] D: Using the spatial accumulation module, the motion information of spatiotemporally correlated pixels in different video frames is accumulated in four motion directions: up, down, left, and right, to obtain motion information values ​​in each direction. Finally, the largest motion information value is selected as the motion direction information at that pixel location, and the corresponding direction is the motion direction of the moving target at that pixel location. If the motion information value in each direction is lower than the set motion information threshold E... e If no moving target is found at the pixel location, then it is assumed that there is no moving target at that pixel location; finally, the corresponding pixel (x) is obtained. m ,y q ,t k Enhanced motion response information F(x) at position m ,y q ,t k ,d);

[0162] E: Using the motion direction modulation module and brightness modulation module in the peripheral modulation module, the corresponding pixel points (x) are obtained respectively. m ,y q ,t k The contrast coefficient R of the direction of motion at the position dir (x m ,y q ,t k ) and brightness contrast factor R lum (x m ,y q ,t k The final response coefficient is obtained as R(x). m ,y q ,t k );

[0163] F: Using the information fusion module, the data from pixel (x) m ,y q ,t k Enhanced motion response information R at the location dir (x m ,y q ,t k ) and final response coefficient R(x m ,y q ,t k Convolution is performed to obtain the fused value S(x) m ,y q ,t k The maximum value among the fused values ​​is used as the fusion comparison value. Then, the fusion comparison value is compared with the set motion detection threshold. If the fusion comparison value is greater than the motion detection threshold, it is considered that there is a moving target in the video frame.

[0164] G: Using the information fusion module, the pixel position corresponding to the fusion comparison value is used as the reference pixel. The fusion comparison value of each pixel around the reference pixel is calculated, and it is determined whether it is greater than the set comparison threshold of the reference pixel's fusion comparison value. Then, the positions of all pixels corresponding to the fusion comparison values ​​that are greater than the set comparison threshold and the reference pixel are used as the appearance position of the moving target, and the appearance position of the moving target is sent to the display device.

[0165] H: The location information of the moving target is marked and displayed using a display device, and an alarm module is used to provide a reminder.

Claims

1. A motion detection and early warning system based on the motion coding mechanism of bird optic canopy, characterized in that: Includes video capture devices, data processing devices, and display devices; A video capture device is used to capture video and convert the captured video into digital signals, which are then sent to a data processing device. The data processing device is used to calculate the brightness information of each pixel in each frame of video image through a brightness perception module, and then obtain the motion response information of each pixel in each frame of video image through a motion detection module. The motion response information includes motion response information for both brightness increase and brightness decrease. Subsequently, the spatiotemporal accumulation module is used to obtain the enhanced motion response information and motion direction information of the moving target by using the motion response information of pixels with spatiotemporal correlation in different frames of video image. The data processing device also obtains the final response coefficient for adjusting the enhanced motion response information through a peripheral modulation module. The final response coefficient includes a motion direction contrast coefficient and a brightness contrast coefficient. After the information fusion module integrates the enhanced motion response information and the final response coefficient, it outputs the position information of the moving target in the video. The display device is used to identify and display the position information of the moving target output by the data processing device.

2. The motion detection and early warning system based on the bird optic canopy motion coding mechanism according to claim 1, characterized in that: The data processing device includes a brightness sensing module, a motion detection module, a spatiotemporal accumulation module, a peripheral modulation module, and an information fusion module; The brightness perception module is used to calculate the brightness information of each pixel in each frame of the input video and to smooth the brightness information. The motion detection module is used to calculate the ON-OFF signal of the brightness change of each pixel in each frame of video image, and perform delay processing on the ON signal and OFF signal after separating them; then, the delay of the obtained ON signal and the delay of the OFF signal are convolved with the OFF signal and ON signal of the same pixel in the same frame of video image, and the convolution results are added to finally obtain the motion response information at the corresponding pixel position. The spatiotemporal accumulation module is used to accumulate the motion information of spatiotemporally correlated pixels in different frames of video images in four motion directions: up, down, left, and right. This yields motion information values ​​in each of the four directions, which are the enhanced motion response information. Finally, the largest motion information value is selected as the motion direction information at that pixel location, and the corresponding direction is the motion direction of the moving target at that pixel location. If the motion information values ​​in each direction are all below a set motion information threshold... If the value is zero, it is considered that there is no moving target at the location of that pixel. The peripheral modulation module is used to obtain the final response coefficient of the enhanced motion response information through a division normalization operation; the final response coefficient is composed of motion direction contrast coefficient and brightness contrast coefficient with different weights; The information fusion module integrates the enhanced motion response information output by the spatiotemporal accumulation module and the final response coefficients output by the peripheral modulation module. It convolves the enhanced motion response information and the final response coefficients from the same pixel location to obtain a fusion value. The maximum value in the fusion value is used as the fusion comparison value. Then, the fusion comparison value is compared with the set motion detection threshold. If the fusion comparison value is greater than the motion detection threshold, it is considered that there is a moving target in the video image of that frame.

3. The motion detection and early warning system based on the bird optic canopy motion coding mechanism according to claim 2, characterized in that: When a moving target is present in the video frame, the information fusion module also uses the pixel position corresponding to the corresponding fusion comparison value as the reference pixel, calculates the fusion comparison value corresponding to each pixel around the reference pixel, and determines whether it is greater than the set comparison threshold of the reference pixel's fusion comparison value. Then, the positions of all pixels corresponding to the fusion comparison values ​​greater than the set comparison threshold and the reference pixel are used as the location of the moving target.

4. The motion detection and early warning system based on the bird optic canopy motion coding mechanism according to claim 2, characterized in that, When the spatiotemporal accumulation module acquires motion information values ​​in four motion directions, the pixel points... Motion information values ​​of the position in each direction of motion As shown below: The motion information value in the rightward direction is: ； The motion information value in the leftward direction is: ； The motion information value in the downward direction is: ； The motion information value in the upward direction is: ； Where d represents direction, and d = 1, 2, 3, 4 represent the directions of motion as right, left, down, and up, respectively; The subscript value represents the position coordinate of the pixel. and This indicates the size of each video frame. Represents the cumulative weight value of the current video image frame; pixel. Motion response information at location , and They represent Positive and negative numbers, This is the response output for brightness changes in the motion detection module. For the delay of the ON signal, This is the delay for the OFF signal.

5. The motion detection and early warning system based on the bird optic canopy motion coding mechanism according to claim 4, characterized in that: In the spatiotemporal accumulation module, the largest motion information value is selected as the motion direction information at the pixel location, and the motion direction corresponding to the largest motion information value is recorded as the motion direction of the moving target at the pixel location; if the motion information values ​​in all four directions are lower than the set motion information threshold... If the value is zero, it is considered that there is no moving target at the location of that pixel. pixel Enhanced motion response information at location for: ； when This indicates that there is no moving target at the location of that pixel; =1,2,3,4 represent the directions of movement as right, left, down, and up, respectively; Represents pixels The direction of movement at the location.

6. The motion detection and early warning system based on the bird optic canopy motion coding mechanism according to claim 1, characterized in that: The peripheral modulation module includes a motion direction modulation module; The motion direction modulation module is used to acquire pixel points. Contrast coefficient of movement direction at position : ； in, Represents pixels The frequency of the direction of motion at a given location in the current video image. This represents the sum of the frequencies of all directions of motion, with a total of 5 groups of directions. Indicates no movement. These represent the directions of movement as right, left, down, and up, respectively.

7. The motion detection and early warning system based on the bird optic canopy motion coding mechanism according to claim 6, characterized in that: The peripheral modulation module further includes a brightness modulation module; The brightness modulation module is used to acquire pixels. Brightness contrast coefficient at the location : ； in, Represents pixels The frequency of the brightness value at a given location in the current video image. This represents the sum of the frequencies of all brightness values.

8. The motion detection and early warning system based on the bird optic canopy motion coding mechanism according to claim 7, characterized in that: The final output of the peripheral modulation module includes the motion direction contrast coefficient. and brightness contrast factor The final response coefficient is: ； in, Represents the pixels in the peripheral modulation module The final response coefficient at the location, The weighting coefficients represent the comparison of motion directions. This represents the weighting coefficient for brightness contrast.

9. The motion detection and early warning system based on the bird optic canopy motion coding mechanism according to claim 1, characterized in that, The fused value output by the information fusion module is represented as follows: ； in, This represents the enhanced motion response information output by the spatiotemporal accumulation module. This represents the final response coefficient output by the peripheral modulation module.

10. A warning method for a motion detection and warning system based on the motion coding mechanism of bird optic tectonics according to any one of claims 1 to 9, characterized in that: The steps are as follows: A: Use a video capture device to acquire real-time video; B: The brightness perception module is used to calculate the brightness information of each pixel in each frame of video image, and the brightness information is smoothed. C: The motion detection module calculates the ON-OFF signal of brightness change for each pixel in each frame of video image, and performs delay processing on the ON and OFF signals after separation; then, the delayed ON and OFF signals are convolved with the OFF and ON signals of the same pixel in the same frame of video image, respectively, and the convolution results are added to obtain the corresponding pixel. Motion response information at location ; D: Using a spatiotemporal accumulation module, the motion information of spatiotemporally correlated pixels in different video frames is accumulated in four motion directions: up, down, left, and right, to obtain motion information values ​​in each direction. Finally, the largest motion information value is selected as the motion direction information at that pixel location, and the corresponding direction is the motion direction of the moving target at that pixel location. If the motion information values ​​in each direction are all below a set motion information threshold... If no moving target is detected at the pixel location, then it is assumed that there is no moving target at that pixel location; finally, the corresponding pixel is obtained. Enhanced motion response information at location ; E: Using the motion direction modulation module and brightness modulation module in the peripheral modulation module, the corresponding pixel points are obtained respectively. Contrast coefficient of movement direction at position and brightness contrast factor The final response coefficient is obtained as ; F: Using the information fusion module, the data from the pixels... Enhanced motion response information at location and final response coefficient Convolution is performed to obtain the fusion value The maximum value among the fused values ​​is used as the fusion comparison value. Then, the fusion comparison value is compared with the set motion detection threshold. If the fusion comparison value is greater than the motion detection threshold, it is considered that there is a moving target in the video image of that frame. G: Using the information fusion module, the pixel position corresponding to the fusion comparison value is used as the reference pixel. The fusion comparison value of each pixel around the reference pixel is calculated, and it is determined whether it is greater than the set comparison threshold of the reference pixel's fusion comparison value. Then, the positions of all pixels corresponding to the fusion comparison values ​​that are greater than the set comparison threshold and the reference pixel are used as the appearance position of the moving target, and the appearance position of the moving target is sent to the display device. H: The location information of the moving target is marked and displayed using a display device, and an alarm module is used to provide a reminder.

Images