Image processing method and device for detecting target motion based on multi-layer inhibition network
By using an improved multilayer suppression network and Gaussian filter processing, the accuracy problem of target detection in complex motion scenes is solved, and high-precision target motion detection is achieved in dynamic backgrounds and high-order motion scenes.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANGHAI UNIV
- Filing Date
- 2024-05-16
- Publication Date
- 2026-07-07
AI Technical Summary
Existing object detection algorithms struggle to distinguish between foreground and background motion in complex motion scenarios, resulting in low detection accuracy. This is especially true in applications such as drone photography, where both the background and the object are in motion, making it difficult to accurately detect object motion.
An improved multilayer suppression network is adopted, which combines a temporal-space smoothing network and a Gaussian filter. The background motion response is suppressed by a multilayer EMD model, the internal texture of the foreground target is filled, and the contour and motion direction of the foreground target are extracted after Gaussian filtering.
It improves the accuracy and precision of target motion detection in complex scenes, and can effectively detect the motion of foreground targets in dynamic backgrounds and high-order motion scenes, while reducing interference from internal texture changes.
Smart Images
Figure CN118537770B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and in particular to a brain-inspired image processing method and device for detecting target motion based on a multilayer recurrent inhibition network. Background Technology
[0002] In fields such as military and autonomous driving, videos contain numerous complex phenomena, such as motion blur, occlusion, morphological diversity, internal texture motion, and lighting variations, which increases the demands on object detection algorithms. Existing mainstream feature-based detection algorithms are insufficient to distinguish complex moving scenes. These moving scenes share the common characteristic that the moving objects themselves possess complex patterns, and these patterns are not static relative to the objects themselves, but rather exhibit different speeds and directions of motion. Therefore, the spatiotemporal correlation of brightness is disrupted. In such cases, if common object detection algorithms based on optical flow variations are used, they will encode the motion of internal or edge patterns, rather than the motion of the object itself. Furthermore, in practical applications, such as drones equipped with cameras, the movement of the drone causes both the background and the object to be in motion, further increasing the difficulty of accurate object detection.
[0003] Motion determined by the relationship between contrast changes, texture changes, and flickering from two or more stimulus locations is called second-order motion or non-Fourier motion. Common examples of second-order motion include theta motion, equilibrium drift motion, and inverse phi motion. This type of motion is important in everyday life, such as the freely swaying ropes of soldiers wearing ghillie suits during military field operations, the rotating and moving striped spheres, or the moving transparent glass panels.
[0004] Interestingly, insects, zebrafish, cats, guinea pigs, and other animals can effectively recognize this second-order movement, which is crucial for their adaptation to the environment and survival. These organisms' visual processing systems, including photoreceptors and visual lobes, can process specific moving objects and features. In the visual systems of insects such as fruit flies and hoverflies, different photoreceptor cells perceive different wavelengths to identify colors. The perception of localized motion signals occurs in the medulla and lobules, and downstream localized motion signals are integrated in the lobular lamina. Then, the optic ganglia and the central brain integrate this information to form their own motion perception.
[0005] Current moving object detection algorithms can be broadly categorized into two types: (1) extracting motion information by employing differences between consecutive frames; and (2) determining the contour and motion information of objects by extracting object features such as texture, edges, and shape. The first type includes classical methods such as time difference methods, background subtraction, optical flow methods, and Elementary Motion Detector (EMD) algorithms inspired by biological neural mechanisms. Time difference, background subtraction, and optical flow methods have difficulties in handling moving camera scenes because the background is also in motion, making it difficult to distinguish moving objects from the background. The second type includes feature-based algorithms, such as widely used deep learning algorithms and algorithms combined with wavelet transform, for extracting object features. Deep learning algorithms require prior training to identify obvious features, making it difficult to detect object motion in complex texture foreground-background and missing features. Similarly, wavelet transform algorithms rely on contour extraction and need to be combined with methods such as time difference, background subtraction, K-means, or Gaussian mixture models (GMM) to extract motion information, resulting in less than ideal performance in complex textures and moving camera scenes.
[0006] The EMD algorithm, based on the neural computation mechanism of the fruit fly's visual system, identifies direction-selective motion-sensitive neurons in the leaflet lamina by detecting intracellular and extracellular electrical signals. This global direction selectivity is achieved by integrating the outputs of local motion detectors T4 and T5 cells. Each unit acts as a specialized matched filter, extracting directional information by detecting changes in optical flow during motion. Compared to mainstream algorithms, the EMD algorithm relies on the extraction of motion discontinuities, enabling it to recognize patterns against textured backgrounds. Several derivative algorithms have been developed based on the basic EMD algorithm, such as the Basic Small Object Motion Detector (ESTMD) algorithm for small object detection and the EMD algorithm for second-order motion. However, most of these algorithms are customized for specific application scenarios and lack sufficient biological evidence to support them.
[0007] Improving the accuracy of target motion detection in complex motion scenarios has become a technical problem that needs to be solved. Summary of the Invention
[0008] The purpose of this invention is to overcome the shortcomings of the existing technology and provide an image processing method and device for detecting target motion based on a multilayer suppression network.
[0009] The objective of this invention can be achieved through the following technical solutions:
[0010] According to one aspect of the present invention, an image processing method for detecting target motion based on a multilayer inhibition network is provided, the method comprising the following steps:
[0011] Step S1: Convert the input image to grayscale and perform preprocessing to obtain the differentiated image data;
[0012] Step S2: The image processed in step S1 is processed using a multilayer suppression network based on an improved traditional EMD model.
[0013] Step S3: Apply time-sliding mean filtering to the result processed by the multi-layer suppression network to fill in the texture inside the foreground target;
[0014] Step S4: Input the result processed in step S3 into a Gaussian filter for spatial filtering to blur the texture inside the foreground target.
[0015] Step S5: The first layer of a multi-layer suppression network is used to process the result after step S4 to extract the contour and motion direction of the foreground target.
[0016] Preferably, the multilayer suppression network includes a multilayer EMD model, and high-pass filtering is performed only on the first layer of the multilayer EMD model. The multilayer EMD model suppresses the background motion response according to the velocity gradient, thereby distinguishing between foreground motion and background motion.
[0017] More preferably, in step S2, the process of processing the image processed in step S1 using a multilayer suppression network based on an improved traditional EMD model includes inputting the horizontally adjacent pixels of all images processed in step S1 into the multilayer suppression network for processing, and inputting the vertically adjacent pixels of all images processed in step S1 into the multilayer suppression network for processing.
[0018] More preferably, the input multilayer suppression network processes the input by sequentially passing adjacent pixels through multiple layers of the multilayer suppression network until the background response is suppressed to the minimum.
[0019] More preferably, the number of layers in the multilayer suppression network is set according to the suppression requirements of the background motion response.
[0020] Preferably, in step S3, performing time-sliding mean filtering on the result after processing by the multilayer suppression network to fill the internal texture of the foreground target specifically involves: arranging each pixel of the result after processing in step S2 according to time sequence and selecting the corresponding time window width, so that the time window slides on the time sequence of one pixel, and averaging the values within the time window each time it slides as the output.
[0021] More preferably, the standard deviation of the Gaussian filter is one-sixth of the operator side length.
[0022] Preferably, in step S1, the preprocessing specifically involves preprocessing the grayscale image using a Gaussian filter.
[0023] Preferably, in step S1, obtaining the differentiated image data specifically involves: subtracting the grayscale values of two adjacent frames to obtain the differentiated image data.
[0024] According to a second aspect of the present invention, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the method described thereon.
[0025] Compared with the prior art, the present invention has the following beneficial effects:
[0026] 1) This invention improves the traditional EMD model. The constructed multi-layer suppression network can suppress the background motion response according to the velocity gradient, thereby distinguishing between foreground motion and background motion. This makes up for the inability of the traditional EMD model to extract the motion of the foreground target in scenes where the foreground and background are moving simultaneously. This improves the accuracy and precision of target motion detection in complex scenes and can be applied to fields such as autonomous driving and drone photography.
[0027] 2) This invention adds a temporal-spatial smoothing network and a Gaussian filter after the multi-layer suppression network to fill and blur the internal texture of the foreground target, respectively, eliminating the interference of the internal texture change of the target on the motion detection of the target itself. Therefore, when faced with feature changes caused by partial occlusion of the target, motion blur, brightness change, and change of the target's own shape and texture, the motion of the target itself can be accurately detected, further improving the accuracy of target motion detection.
[0028] 3) After processing the texture inside the foreground target, the present invention uses the first layer of the multilayer suppression network for further direction detection, which further improves the accuracy of target motion detection. Attached Figure Description
[0029] Figure 1 This is a schematic diagram of the image processing flow for detecting target motion based on a multilayer suppression network in this invention;
[0030] Figure 2 This is a schematic diagram of the structure of an existing traditional EMD model;
[0031] Figure 3 This is a schematic diagram of the structure of the multilayer suppression network in this invention;
[0032] Figure 4 This is a schematic diagram of time-sliding mean filtering in this invention;
[0033] Figure 5 This is a schematic diagram of spatial Gaussian filtering in this invention;
[0034] In the attached diagram, HP represents a high-pass filter; LP represents a low-pass filter; DC represents DC processing; L1 and L2 represent positive and negative ramp functions, corresponding to different layers of cells in Drosophila, respectively; and M represents multiplication. Detailed Implementation
[0035] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0036] This invention improves upon the traditional EMD model by constructing a multi-layer recursive suppression network to enhance the accuracy of target motion detection in complex motion scenarios. The innovation lies in utilizing the structures within the leaflets and leaf lamina of the fruit fly visual system to construct a multi-layer recursive structure and a temporal-space smoothing network, thereby enhancing the accuracy of traditional EMD in moving backgrounds and high-order motion scenarios.
[0037] Example 1
[0038] This embodiment relates to an image processing method for detecting target motion based on a multilayer suppression network, such as... Figure 1 The method includes:
[0039] Step S1: Convert the input image to grayscale and preprocess it using Gaussian filtering. Subtract the grayscale values from adjacent frames to obtain the differentiated image data.
[0040] Step S1 specifically includes:
[0041] Step S1-1: Read the image frame by frame and convert the RGB image to a grayscale image;
[0042] Step S1-2: Preprocess the image using a Gaussian filter. The Gaussian operator size is 12*12 and the standard deviation is 3.5.
[0043] Step S1-3: Subtract the gray values of two adjacent frames to obtain the differenced image data.
[0044] Step S2: Improve the EMD model by constructing a multi-layer suppression network to process the result after the difference in step S1 until the background response is suppressed to a minimum, which is beneficial for extracting the motion of the foreground target in scenes where the foreground and background move simultaneously.
[0045] Multilayer suppression networks (MLNs) suppress background motion responses based on velocity gradients. Specifically, they suppress background motion responses according to the magnitude of the moving object's velocity. The background motion response of a fast-moving object can be suppressed to zero within one or two layers of the EMD model, while the background motion response of a slow-moving object requires several more EMD layers to be suppressed to zero. This is because the suppression is not based on the exact value of the velocity, but rather on discrete and equally spaced suppression according to the number of EMD layers—a process known as gradient suppression.
[0046] like Figure 3 Multilayer suppression networks (MDNs) include multilayer EMD models. Except for the first layer, the other layers of the EMD model do not undergo high-pass filtering (HP) processing. Traditional EMD models include... Figure 2 As shown.
[0047] like Figure 2 and Figure 3 Step S2 specifically includes:
[0048] Step S2-1: Process the horizontally adjacent pixels of all images after difference using the first layer of a multilayer suppression network (same as the traditional EMD model) to obtain the EMD results of the horizontal pixels. Specifically, take two horizontally adjacent pixels of the differenced image as input, and perform noise reduction on the input using DC processing (DC) and high-pass filtering (HP) respectively, then add them together. Use a ramp function to separate the ON signal (increased gray level) and OFF signal (decreased gray level) in the difference data. The positive and negative ramp functions correspond to different layers of cells in Drosophila (…). Figure 2 The L1 and L2 in the image are used to input the ON signal of two pixels to the left and right arms of one EMD unit, and the OFF signal to the left and right arms of another EMD unit. Then, a low-pass filter (LP) is applied sequentially to the left and right arms of the two EMD units to delay the time sequence. Within one EMD unit, the delayed left arm signal is multiplied by the undelayed right arm signal. Figure 3 In the M), the delayed right arm signal is multiplied by the undelayed left arm signal, and the two arm signals are subtracted to obtain the ON / OFF result. The ON and OFF results are added together to obtain one EMD result.
[0049] Step S2-2: Input the result obtained in step S2-1 into other layers of the multilayer suppression network (except for the first layer, other layers no longer use high-pass filters for noise reduction) until the background response is suppressed to the minimum;
[0050] Step S2-3: Input the vertically adjacent pixels of all images after difference into a multilayer suppression network and process them in the same way as the horizontally adjacent pixels until the background response is suppressed to the minimum.
[0051] Step S3: Use a sliding time window to slide through the results processed by the multi-layer suppression network frame by frame, and perform mean processing on the sequence within the window to fill the internal texture of the foreground target and eliminate the interference of the internal texture changes of the foreground target on the high-order motion detection of the target itself.
[0052] like Figure 4 Step S3 specifically includes:
[0053] Step S3-1: Arrange each pixel obtained from S2 in chronological order and select the corresponding time window width. Slide the time window over the time sequence of a pixel. Each time the time window slides, calculate the average value within the time window and use it as the output.
[0054] Step S3-2: Repeat step S3-1, performing the same processing on each pixel.
[0055] Step S4: Input the result processed in step S3 into a Gaussian filter with a suitable operator and standard deviation. The standard deviation of the Gaussian filter is one-sixth of the operator's side length. By using a Gaussian filter (such as...) Figure 5 The texture inside the foreground target is blurred, which facilitates the extraction of motion information of the foreground target in the next step and further eliminates the interference of the texture changes inside the foreground target on the high-order motion detection of the target itself.
[0056] Step S5: Use the first layer of the multilayer suppression network (same as the traditional EMD model) to process the result after step S4, and extract the outline and motion direction of the foreground target.
[0057] Compared to the classic EMD model, this invention uses a more layered classic EMD model to simulate the recurrent feedback structure of Drosophila T4 and T5 cells, enabling the detection of moving targets in a dynamic context; it incorporates a temporal-spatial smoothing network that simulates the neural firing process and the receptive field structure of Drosophila, enabling the detection of real target motion in scenarios containing higher-order motion; and based on the functional characteristics of tangential cells in the leaflet plate of Drosophila, an additional orientation detection layer is added (step S5), retaining the orientation detection function of the classic EMD model.
[0058] Example 2
[0059] This embodiment involves a comparison with other motion detection algorithms.
[0060] The present invention, the EMD leaflet network model proposed in 2023, and the 3D discrete wavelet algorithm proposed in 2019 were compared in three practical scenarios. The comparison results are shown in Table 1.
[0061] Table 1
[0062]
[0063] We use the F-measure to measure the algorithm's ability to extract targets and the PI to measure the algorithm's ability to suppress moving backgrounds. The F-measure is calculated as 2TP / (2TP + FP + FN), where TP is a true positive, FP is a false positive, and FN is a false negative. The PI is calculated by dividing the sum of all output values in the target region by the sum of all response values in the object and background regions.
[0064] As can be seen from the comparison results in Table 1, the present invention is superior to the two methods mentioned above in terms of suppressing dynamic backgrounds and extracting targets in complex scenes.
[0065] Example 3
[0066] The electronic device of this invention includes a central processing unit (CPU), which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) or loaded from a storage unit into random access memory (RAM). The RAM may also store various programs and data required for device operation. The CPU, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.
[0067] Multiple components in the device are connected to the I / O interface, including: input units such as keyboards and mice; output units such as various types of displays and speakers; storage units such as disks and optical discs; and communication units such as network interface cards (NICs), modems, and wireless transceivers. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0068] The processing unit performs the various methods and processes described above, which are tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on the device via ROM and / or a communication unit. When the computer program is loaded into RAM and executed by the CPU, one or more steps of the methods described above may be performed. Alternatively, in other embodiments, the CPU may be configured to execute the methods by any other suitable means (e.g., by means of firmware).
[0069] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. An image processing method for detecting target motion based on a multilayer inhibition network, characterized in that, The method includes the following steps: Step S1: Convert the input image to grayscale and perform preprocessing to obtain the differentiated image data; Step S2 involves processing the image from step S1 using a multilayer suppression network based on an improved traditional EMD model, including: Step S2-1: Process the horizontally adjacent pixels of all images after difference using the first layer of a multilayer suppression network to obtain the EMD result of the horizontal pixels. Specifically, take two horizontally adjacent pixels of the differenced image as input, and perform DC processing and high-pass filtering for noise reduction on the inputs respectively, and then add them together. Use a ramp function to separate the ON and OFF signals in the difference data. The positive and negative ramp functions correspond to different layers of cells in Drosophila. The ON signal of the two pixels is input to the left and right arms of one EMD unit, and the OFF signal is input to the left and right arms of another EMD unit. Then, apply a low-pass filter to the left and right arms of the two EMD units in sequence to delay the time series. In one EMD unit, the delayed left arm signal is multiplied by the undelayed right arm signal, and the delayed right arm signal is multiplied by the undelayed left arm signal. Subtract the signals of the two arms to obtain the ON / OFF result. Add the ON and OFF results to obtain one EMD result. Step S2-2: Input the result obtained in step S2-1 into other layers of the multilayer suppression network until the background response is suppressed to the minimum. Except for the first layer, other layers no longer use high-pass filters for noise reduction. Step S2-3: Input the vertically adjacent pixels of all images after difference into a multilayer suppression network and process the horizontally adjacent pixels in the same way until the background response is suppressed to the minimum. Step S3: Apply time-sliding mean filtering to the result processed by the multi-layer suppression network to fill in the texture inside the foreground target; Step S4: Input the result processed in step S3 into a Gaussian filter for spatial filtering to blur the texture inside the foreground target. Step S5: The first layer of a multi-layer suppression network is used to process the result after step S4 to extract the contour and motion direction of the foreground target.
2. The image processing method for detecting target motion based on a multilayer inhibition network according to claim 1, characterized in that, The number of layers in the multilayer suppression network is set according to the suppression requirements of the background motion response.
3. The image processing method for detecting target motion based on a multilayer inhibition network according to claim 1, characterized in that, In step S3, the time-sliding mean filtering of the result after processing by the multi-layer suppression network to fill the internal texture of the foreground target is specifically performed as follows: each pixel of the result after processing in step S2 is arranged in time sequence and the corresponding time window width is selected, so that the time window slides on the time sequence of one pixel, and the average value within the time window is calculated as the output each time it slides.
4. The image processing method for detecting target motion based on a multilayer inhibition network according to claim 1, characterized in that, The standard deviation of the Gaussian filter is one-sixth of the operator side length.
5. The image processing method for detecting target motion based on a multilayer inhibition network according to claim 1, characterized in that, In step S1, the preprocessing specifically involves using Gaussian filtering to preprocess the grayscale image.
6. The image processing method for detecting target motion based on a multilayer inhibition network according to claim 1, characterized in that, In step S1, obtaining the differentiated image data specifically involves subtracting the grayscale values of two adjacent frames to obtain the differentiated image data.
7. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 6.