A human body movement detection method and system based on pulse neural network

By applying pulse neural network in human body movement detection, image processing is combined with pulse neural network, and video pixel values ​​are converted into pulse sequences, solving the problems of high hardware cost and energy consumption in the existing technology, and achieving high efficiency and low energy consumption Human movement detection.

CN116681725BActive Publication Date: 2024-11-22CHENGDU NISHENG TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310623075.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-05-30
Publication Date
2024-11-22
Estimated Expiration
2043-05-30

AI Technical Summary

Technical Problem

In the prior art, the human body movement detection method based on image recognition is relatively high in hardware deployment cost and energy consumption, and is difficult to widely use in home scenarios.

Method used

The human body movement detection method based on pulse neural network is adopted, and the target video is image sampling, edge extraction and background differential calculation are performed, and the human body movement detection is performed using the low power consumption characteristics of pulse neural network.

Benefits of technology

It reduces hardware requirements and energy consumption, improves detection accuracy and robustness, is suitable for lightweight edge computing devices, reduces the overall number of pulses of neural networks, and reduces the computational complexity.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116681725B_ABST
    Figure CN116681725B_ABST
Patent Text Reader

Abstract

The present invention discloses a method and system for detecting human movement based on a pulse neural network, which relates to the field of image processing, extracts effective subject information and background information from video data; converts the obtained information into a pulse sequence; constructs a multi-layer neural network using pulse neurons; sets pulse neuron parameters; uses the pulse sequence as input to generate corresponding pulse output; the method combines SNN with image processing methods, and is applied to the field of human movement detection, providing a new idea for lightweight and efficient intelligent detection equipment, while expanding the application scope of brain-like chips based on pulse neural networks.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a method and system for detecting human movement based on a pulse neural network. Background Art

[0002] Nowadays, many occasions require monitoring and reminders of human movement and approach. By obtaining information about changes in human movement and performing intelligent judgment and analysis, the human movement detection and reminder function can be realized. At present, the methods for obtaining human movement information mainly include position change detection, human body sensors, etc. Although the strategies implemented by the human body sensor method are more diverse and can be used in different scenarios, the position change detection method in the home context has stronger competitiveness in system cost, such as home smart switch lights. The main reason is that although the monitoring method based on image recognition has the characteristics of more accuracy, the hardware deployment cost is higher, and it also has higher equipment working energy consumption. Summary of the invention

[0003] In order to at least overcome the above-mentioned deficiencies in the prior art, the purpose of the present application is to provide a method and system for detecting human movement based on a pulse neural network.

[0004] In a first aspect, an embodiment of the present application provides a method for detecting human movement based on a pulse neural network, comprising:

[0005] Performing image sampling on the target video to obtain multiple target images;

[0006] Performing edge extraction on the target image to form an edge matrix, and performing background difference calculation on the target image according to a background image to form a background difference matrix; the background image is one of the target images or a preset background image;

[0007] Using a first threshold coding to encode all the edge matrices of the target video to form an edge pulse sequence, and using a second threshold coding to encode all the background difference matrices of the target video to form a detection pulse sequence; the elements in the edge pulse sequence and the detection pulse sequence are all 0 or 1;

[0008] The edge pulse sequence and the detection pulse sequence are subjected to ignition quantity detection, and the ignition quantity detection result is used as the human body movement detection result of the target video.

[0009] In the embodiment of the present application, a scheme combining a spiking neural network and image processing is used for human recognition. Image processing is an image processing method that distinguishes different categories of targets according to their different features reflected in the image information. It uses a computer to perform quantitative analysis on the image, and classifies the image or each pixel or region in the image into one of several categories to replace human visual interpretation. As a third-generation artificial neural network, the spiking neural network (SNN) transmits time and space information through a pulse sequence, and highly fits the dynamic information processing process of neurons in biological neural networks. As the most basic computing unit, the spiking neuron integrates the pulse signals transmitted by all synapses (weights) in space, accumulates the influence of historical information in time, and has a strong spatiotemporal processing capability. At the same time, thanks to the special information transmission method of SNN, the SNN model is much lower in power consumption than the conventional deep learning model, and is very suitable for deployment on lightweight edge computing devices. Combining SNN with the image processing method and applying it to the field of human motion detection can draw on the advantages of convolutional neural network (CNN) in extracting spatial visual features, and use the special features of SNN in extracting timing information and the characteristics of ultra-low power consumption.

[0010] Specifically, the target image is generated by extracting multiple static images from the target video, which is a prior art and will not be repeated in the embodiments of this application. Similarly, edge detection and background difference calculation are performed on the target image to extract the edge matrix and the background difference matrix, which are also mature technologies and will not be repeated in the embodiments of this application.

[0011] In the embodiment of the present application, it is necessary to reduce the hardware requirements and energy consumption deployed in the edge device, and human body recognition is not performed directly through the edge matrix and the background difference matrix, that is, the edge matrix and the background difference matrix are not processed by the convolutional neural network that needs to be trained, but the edge matrix and the background difference matrix are converted into a pulse sequence with only 0 and 1 elements. Specifically, when the data to be encoded is greater than or equal to the corresponding threshold, a pulse is issued, that is, it is encoded as 1, and when the data to be encoded is less than the corresponding threshold, no pulse is issued, that is, it is encoded as 0. This encoding method can save a lot of computing power for subsequent detection or information transmission, and the energy consumption is very low. At the same time, based on the dual recognition scheme of edge detection and background difference, when calculating in the SNN network, the robustness of the embodiment of the present application can be well improved, and the detection accuracy can be improved. In the embodiment of the present application, it is necessary to detect the number of ignitions of the edge pulse sequence and the detection pulse sequence based on the SNN network, and finally determine whether there is a human body moving in the target video.

[0012] In a possible implementation, performing ignition quantity detection on the edge pulse sequence and the detection pulse sequence includes:

[0013] inputting the edge pulse sequence into the edge input neuron and inputting the detection pulse sequence into the main input neuron;

[0014] Inhibiting the pulse output of the main input neuron within a preset inhibition radius by the edge input neuron;

[0015] The main detection neuron generates a first pulse sequence by judging the pulse sequence output by the main input neuron based on a first threshold within a preset neuron receptive field window;

[0016] The edge detection neuron generates a second pulse sequence by performing a judgment based on a second threshold within the preset neuron receptive field window on the pulse sequence output by the edge input neuron;

[0017] The first pulse sequence and the second pulse sequence are detected for ignition quantity by using human body detection neurons, and when the ignition quantity exceeds an ignition quantity threshold, it is determined that a human body exists in the target video corresponding to the first pulse sequence and the second pulse sequence.

[0018] In a possible implementation, generating a first pulse sequence by judging the pulse sequence output by the main input neuron based on a first threshold within a preset neuron receptive field window through the main detection neuron includes:

[0019] Performing sliding window detection on the pulse sequence output by the subject input neuron through the preset neuron receptive field window to form the first pulse sequence;

[0020] When the number of elements that are 1 in the pulse sequence within the preset neuron receptive field window is greater than or equal to the first threshold, the element of the first pulse sequence corresponding to the preset neuron receptive field window is recorded as 1;

[0021] When the number of elements that are 1 in the pulse sequence within the current preset neuron receptive field window is less than a first threshold, the element of the first pulse sequence corresponding to the preset neuron receptive field window is recorded as 0.

[0022] In a possible implementation, generating a second pulse sequence by performing a judgment based on a second threshold within the preset neuron receptive field window on the pulse sequence output by the edge input neuron through the edge detection neuron includes:

[0023] Performing sliding window detection on the pulse sequence output by the edge input neuron through the preset neuron receptive field window to form the second pulse sequence;

[0024] When the number of elements that are 1 in the pulse sequence within the preset neuron receptive field window is greater than or equal to the second threshold, the element of the second pulse sequence corresponding to the preset neuron receptive field window is recorded as 1;

[0025] When the number of elements that are 1 in the pulse sequence within the current preset neuron receptive field window is less than the second threshold, the elements of the second pulse sequence corresponding to the preset neuron receptive field window are recorded as 0.

[0026] In a possible implementation, performing ignition number detection on the first pulse sequence and the second pulse sequence through a human body detection neuron, and determining that a human body exists in the target video corresponding to the first pulse sequence and the second pulse sequence when the ignition number exceeds an ignition number threshold, includes:

[0027] The number of elements that are 1 in the first pulse sequence and the second pulse sequence is counted by human body detection neurons as the firing number.

[0028] In a possible implementation manner, using a first threshold coding to encode all the edge matrices of the target video to form an edge pulse sequence includes:

[0029] Encode the elements in the edge matrix that are higher than or equal to the first threshold code as 1, and encode the elements in the edge matrix that are lower than the first threshold code as 0;

[0030] All the encoded edge matrices of the target video are used as the three-dimensional edge pulse sequence.

[0031] In a possible implementation manner, using the second threshold coding to encode all the background difference matrices of the target video to form a detection pulse sequence includes:

[0032] Encode the elements in the background difference matrix that are higher than or equal to the second threshold code as 1, and encode the elements in the background difference matrix that are lower than the second threshold code as 0;

[0033] All the encoded background difference matrices of the target video are used as the three-dimensional detection pulse sequence.

[0034] In a second aspect, an embodiment of the present application provides a human movement detection system based on a pulse neural network, comprising:

[0035] An acquisition unit is configured to perform image sampling on a target video to acquire a plurality of target images;

[0036] a matrix generating unit, configured to perform edge extraction on the target image to form an edge matrix, and perform background difference calculation on the target image according to a background image to form a background difference matrix; the background image is one of the target images or a preset background image;

[0037] The encoding unit is configured to use a first threshold encoding to encode all the edge matrices of the target video to form an edge pulse sequence, and use a second threshold encoding to encode all the background difference matrices of the target video to form a detection pulse sequence; the elements in the edge pulse sequence and the detection pulse sequence are all 0 or 1;

[0038] The detection unit is configured to perform ignition quantity detection on the edge pulse sequence and the detection pulse sequence, and use the ignition quantity detection result as the human body movement detection result of the target video.

[0039] In a possible implementation, the detection unit is further configured to:

[0040] inputting the edge pulse sequence into the edge input neuron and inputting the detection pulse sequence into the main input neuron;

[0041] Inhibiting the pulse output of the main input neuron within a preset inhibition radius by the edge input neuron;

[0042] The main detection neuron generates a first pulse sequence by judging the pulse sequence output by the main input neuron based on a first threshold within a preset neuron receptive field window;

[0043] The edge detection neuron generates a second pulse sequence by performing a judgment based on a second threshold within the preset neuron receptive field window on the pulse sequence output by the edge input neuron;

[0044] The first pulse sequence and the second pulse sequence are detected for ignition quantity by using human body detection neurons, and when the ignition quantity exceeds an ignition quantity threshold, it is determined that a human body exists in the target video corresponding to the first pulse sequence and the second pulse sequence.

[0045] In a possible implementation, the detection unit is further configured to:

[0046] Performing sliding window detection on the pulse sequence output by the subject input neuron through the preset neuron receptive field window to form the first pulse sequence;

[0047] When the number of elements that are 1 in the pulse sequence within the preset neuron receptive field window is greater than or equal to the first threshold, the element of the first pulse sequence corresponding to the preset neuron receptive field window is recorded as 1;

[0048] When the number of elements that are 1 in the pulse sequence within the current preset neuron receptive field window is less than a first threshold, the element of the first pulse sequence corresponding to the preset neuron receptive field window is recorded as 0.

[0049] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0050] The present invention discloses a human motion detection method and system based on a pulse neural network, which converts video pixel values ​​into pulse sequences. It not only takes into account the relationship between static background and human dynamics, and highlights the difference between background and moving human body, but also ensures that pulse signals are generated only when human body movement enters the monitoring range, thereby reducing the overall number of pulses of the neural network and reducing the computational complexity. At the same time, it does not require the training of traditional neural networks, but only needs to adjust individual parameters according to actual conditions, which is convenient for deployment on edge devices, does not require additional computing power investment, and has lower energy consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0052] Figure 1 It is a schematic diagram of the steps of the method of the embodiment of the present application;

[0053] Figure 2 It is a schematic diagram of the architecture of an embodiment of the present application. DETAILED DESCRIPTION

[0054] To make the purpose, technical scheme and advantages of the embodiment of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the embodiment of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0055] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0056] Please refer to Figure 1 , which is a flow chart of a method for detecting human movement based on a pulse neural network provided in an embodiment of the present invention. Furthermore, the method for detecting human movement based on a pulse neural network may specifically include the contents described in the following steps S1 to S4.

[0057] S1: Perform image sampling on the target video to obtain multiple target images;

[0058] S2: extracting edges of the target image to form an edge matrix, and performing background difference calculation on the target image according to a background image to form a background difference matrix; the background image is one of the target images or a preset background image;

[0059] S3: using a first threshold coding to encode all the edge matrices of the target video to form an edge pulse sequence, and using a second threshold coding to encode all the background difference matrices of the target video to form a detection pulse sequence; the elements in the edge pulse sequence and the detection pulse sequence are all 0 or 1;

[0060] S4: performing ignition quantity detection on the edge pulse sequence and the detection pulse sequence, and using the ignition quantity detection result as the human body movement detection result of the target video.

[0061] In the embodiment of the present application, a scheme combining a spiking neural network and image processing is used for human recognition. Image processing is an image processing method that distinguishes different categories of targets according to their different features reflected in the image information. It uses a computer to perform quantitative analysis on the image, and classifies the image or each pixel or region in the image into one of several categories to replace human visual interpretation. As a third-generation artificial neural network, the spiking neural network (SNN) transmits time and space information through a pulse sequence, and highly fits the dynamic information processing process of neurons in biological neural networks. As the most basic computing unit, the spiking neuron integrates the pulse signals transmitted by all synapses (weights) in space, accumulates the influence of historical information in time, and has a strong spatiotemporal processing capability. At the same time, thanks to the special information transmission method of SNN, the SNN model is much lower in power consumption than the conventional deep learning model, and is very suitable for deployment on lightweight edge computing devices. Combining SNN with the image processing method and applying it to the field of human motion detection can draw on the advantages of convolutional neural network (CNN) in extracting spatial visual features, and use the special features of SNN in extracting timing information and the characteristics of ultra-low power consumption.

[0062] Specifically, the target image is generated by extracting multiple static images from the target video, which is a prior art and will not be repeated in the embodiments of this application. Similarly, edge detection and background difference calculation are performed on the target image, where the edge extraction method includes but is not limited to the Canny edge detection algorithm, the Sobel edge detection algorithm, and the DoG edge detection algorithm, and the extraction of the edge matrix and the background difference matrix is ​​also a mature technology, which will not be repeated in the embodiments of this application.

[0063] In the embodiment of the present application, it is necessary to reduce the hardware requirements and energy consumption deployed in the edge device, and human body recognition is not performed directly through the edge matrix and the background difference matrix, that is, the edge matrix and the background difference matrix are not processed by the convolutional neural network that needs to be trained, but the edge matrix and the background difference matrix are converted into a pulse sequence with only 0 and 1 elements. Specifically, when the data to be encoded is greater than or equal to the corresponding threshold, a pulse is issued, that is, it is encoded as 1, and when the data to be encoded is less than the corresponding threshold, no pulse is issued, that is, it is encoded as 0. This encoding method can save a lot of computing power for subsequent detection or information transmission, and the energy consumption is very low. At the same time, based on the dual recognition scheme of edge detection and background difference, when calculating in the SNN network, the robustness of the embodiment of the present application can be well improved, and the detection accuracy can be improved. In the embodiment of the present application, it is necessary to detect the number of ignitions of the edge pulse sequence and the detection pulse sequence based on the SNN network, and finally determine whether there is a human body moving in the target video.

[0064] In a possible implementation, performing ignition quantity detection on the edge pulse sequence and the detection pulse sequence includes:

[0065] inputting the edge pulse sequence into the edge input neuron and inputting the detection pulse sequence into the main input neuron;

[0066] Inhibiting the pulse output of the main input neuron within a preset inhibition radius by the edge input neuron;

[0067] The main detection neuron generates a first pulse sequence by judging the pulse sequence output by the main input neuron based on a first threshold within a preset neuron receptive field window;

[0068] The edge detection neuron generates a second pulse sequence by performing a judgment based on a second threshold within the preset neuron receptive field window on the pulse sequence output by the edge input neuron;

[0069] The first pulse sequence and the second pulse sequence are detected for ignition quantity by using human body detection neurons, and when the ignition quantity exceeds an ignition quantity threshold, it is determined that a human body exists in the target video corresponding to the first pulse sequence and the second pulse sequence.

[0070] When implementing the embodiments of this application, please refer to Figure 2 , showing the relationship between each neuron, edge detection neurons, subject detection neurons and human body detection neurons are preferably LIF models, and when pulse sequences of edge input neurons and subject input neurons are input, their main function is to suppress the output of subject input neurons through edge input neurons; because in the pulse sequence generated by edge detection, the element represented as 1 may also be represented as 1 in the pulse sequence generated by background differential detection, and there may be a situation where multiple elements are repeated as 1, so in order to improve the detection accuracy, the embodiment of the present application sets a preset suppression radius for the edge input neurons, and suppresses the pulse output of the subject input neurons within this range, that is, the subject input neurons cannot generate pulse signals within this range.

[0071] In an embodiment of the present application, the detection neuron needs to sample the pulse sequence of the corresponding input neuron, and specifically adopts a sequence of preset neuron receptive field windows for sliding window sampling. The sliding window sampling itself can reduce false alarm points caused by algorithm errors. The first threshold and the second threshold are used to generate a pulse sequence for the data generated by the sliding window sampling process, thereby generating a first pulse sequence and a second pulse sequence.

[0072] When the embodiment of the present application is implemented, for human body detection neurons, after obtaining the first pulse sequence and the second pulse sequence, the ignition number detection can be performed, that is, the number of elements with a value of 1 in the pulse sequence is counted, and the final judgment is made through the ignition number threshold.

[0073] In a possible implementation, generating a first pulse sequence by judging the pulse sequence output by the main input neuron based on a first threshold within a preset neuron receptive field window through the main detection neuron includes:

[0074] Performing sliding window detection on the pulse sequence output by the subject input neuron through the preset neuron receptive field window to form the first pulse sequence;

[0075] When the number of elements that are 1 in the pulse sequence within the preset neuron receptive field window is greater than or equal to the first threshold, the element of the first pulse sequence corresponding to the preset neuron receptive field window is recorded as 1;

[0076] When the number of elements that are 1 in the pulse sequence within the current preset neuron receptive field window is less than a first threshold, the element of the first pulse sequence corresponding to the preset neuron receptive field window is recorded as 0.

[0077] In a possible implementation, generating a second pulse sequence by performing a judgment based on a second threshold within the preset neuron receptive field window on the pulse sequence output by the edge input neuron through the edge detection neuron includes:

[0078] Performing sliding window detection on the pulse sequence output by the edge input neuron through the preset neuron receptive field window to form the second pulse sequence;

[0079] When the number of elements that are 1 in the pulse sequence within the preset neuron receptive field window is greater than or equal to the second threshold, the element of the second pulse sequence corresponding to the preset neuron receptive field window is recorded as 1;

[0080] When the number of elements that are 1 in the pulse sequence within the current preset neuron receptive field window is less than the second threshold, the elements of the second pulse sequence corresponding to the preset neuron receptive field window are recorded as 0.

[0081] In a possible implementation, performing ignition number detection on the first pulse sequence and the second pulse sequence through a human body detection neuron, and determining that a human body exists in the target video corresponding to the first pulse sequence and the second pulse sequence when the ignition number exceeds an ignition number threshold, includes:

[0082] The number of elements that are 1 in the first pulse sequence and the second pulse sequence is counted by human body detection neurons as the firing number.

[0083] In a possible implementation manner, using a first threshold coding to encode all the edge matrices of the target video to form an edge pulse sequence includes:

[0084] Encode the elements in the edge matrix that are higher than or equal to the first threshold code as 1, and encode the elements in the edge matrix that are lower than the first threshold code as 0;

[0085] All the encoded edge matrices of the target video are used as the three-dimensional edge pulse sequence.

[0086] In a possible implementation manner, using the second threshold coding to encode all the background difference matrices of the target video to form a detection pulse sequence includes:

[0087] Encode the elements in the background difference matrix that are higher than or equal to the second threshold code as 1, and encode the elements in the background difference matrix that are lower than the second threshold code as 0;

[0088] All the encoded background difference matrices of the target video are used as the three-dimensional detection pulse sequence.

[0089] When implementing the embodiments of this application, please refer to Figure 2 , shows a more specific implementation scheme, including:

[0090] Shoot a video with a camera, the resolution is 120x120, the frame rate is 25 frames / second, the length is 10 seconds, and the video sampling interval is 0.2 seconds. Get a 120x120x500 video data;

[0091] Use the Canny edge extraction algorithm for each frame except the first frame in the video to obtain an edge matrix E of size 120x120x499;

[0092] Use the Canny edge extraction algorithm for each frame except the first frame in the video to obtain an edge matrix E of size 120x120x499;

[0093] Subtract each frame of the video from the first frame of the background image, then take the absolute value to get a background difference matrix D of size 120x120x499;

[0094] The threshold θ is set to 0.5 and the encoding is obtained to obtain a detection pulse sequence Sl of size 120x120x500, with only elements of 0 or 1. At this point, the original video of size 120x120x2500 is converted into a pulse sequence of size 120x120x499x2, with only elements of 0 or 1.

[0095] Then you need to set the pulse neuron parameters and use the pulse neurons to build a detection network:

[0096] Build 120x120 main input neurons and 120x120 edge input neurons, and set the edge neuron inhibition radius r = 1. The weight of each edge input neuron connected to the main input neuron within the pixel space distance 1 is set to -1, and the others are set to 1;

[0097] Build the subject detection neuron, set the neuron receptive field window size w=5, threshold θl=11, and weight 1; after receiving the pulse sequence of the subject input neuron, a pulse sequence of size 116x116 with elements of 0 or 1 can be obtained.

[0098] Build an edge detection neuron, set the neuron receptive field window size w=5, threshold θe=6, and weight 1; after receiving the pulse sequence of the edge input neuron, a pulse sequence of size 116x116 with elements of 0 or 1 can be obtained.

[0099] Build a human detection neuron, set the neuron threshold θ = 2, and the weight is 1. The human detection neuron receives pulse sequences from the body detection neuron and the edge detection neuron.

[0100] Set the threshold value of the number of firing neurons for human detection θf = 145. When the number of firing neurons for human detection exceeds 145, it means that someone appears or passes through the detection area.

[0101] Based on the same inventive concept as above, a human movement detection system based on a pulse neural network is also provided. A detailed description of the human movement detection system based on a pulse neural network is as follows.

[0102] A human movement detection system based on a pulse neural network, comprising:

[0103] An acquisition unit is configured to perform image sampling on a target video to acquire a plurality of target images;

[0104] a matrix generating unit, configured to perform edge extraction on the target image to form an edge matrix, and perform background difference calculation on the target image according to a background image to form a background difference matrix; the background image is one of the target images or a preset background image;

[0105] The encoding unit is configured to use a first threshold encoding to encode all the edge matrices of the target video to form an edge pulse sequence, and use a second threshold encoding to encode all the background difference matrices of the target video to form a detection pulse sequence; the elements in the edge pulse sequence and the detection pulse sequence are all 0 or 1;

[0106] The detection unit is configured to perform ignition quantity detection on the edge pulse sequence and the detection pulse sequence, and use the ignition quantity detection result as the human body movement detection result of the target video.

[0107] In a possible implementation, the detection unit is further configured to:

[0108] inputting the edge pulse sequence into the edge input neuron and inputting the detection pulse sequence into the main input neuron;

[0109] Inhibiting the pulse output of the main input neuron within a preset inhibition radius by the edge input neuron;

[0110] The main detection neuron generates a first pulse sequence by judging the pulse sequence output by the main input neuron based on a first threshold within a preset neuron receptive field window;

[0111] The edge detection neuron generates a second pulse sequence by performing a judgment based on a second threshold within the preset neuron receptive field window on the pulse sequence output by the edge input neuron;

[0112] The first pulse sequence and the second pulse sequence are detected for ignition quantity by using human body detection neurons, and when the ignition quantity exceeds an ignition quantity threshold, it is determined that a human body exists in the target video corresponding to the first pulse sequence and the second pulse sequence.

[0113] In a possible implementation, the detection unit is further configured to:

[0114] Performing sliding window detection on the pulse sequence output by the subject input neuron through the preset neuron receptive field window to form the first pulse sequence;

[0115] When the number of elements that are 1 in the pulse sequence within the preset neuron receptive field window is greater than or equal to the first threshold, the element of the first pulse sequence corresponding to the preset neuron receptive field window is recorded as 1;

[0116] When the number of elements that are 1 in the pulse sequence within the current preset neuron receptive field window is less than a first threshold, the element of the first pulse sequence corresponding to the preset neuron receptive field window is recorded as 0.

[0117] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0118] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or it can be an electrical, mechanical or other form of connection.

[0119] The units described as separate components may or may not be physically separated. As units, it is obvious that a person of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0120] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0121] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a grid device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0122] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A human body movement detection method based on pulse neural network, characterized in that: include: Performing image sampling on the target video to obtain multiple target images; Performing edge extraction on the target image to form an edge matrix, and performing background difference calculation on the target image according to the background image to form a background difference matrix; The background image is one of the target images or a preset background image; Using a first threshold coding to encode all the edge matrices of the target video to form an edge pulse sequence, and using a second threshold coding to encode all the background difference matrices of the target video to form a detection pulse sequence; Elements in the edge pulse sequence and the detection pulse sequence are all 0 or 1; Performing ignition quantity detection on the edge pulse sequence and the detection pulse sequence, and using the ignition quantity detection result as the human body movement detection result of the target video; Performing background difference calculation on the target image according to the background image to form a background difference matrix includes: Subtract each frame of the video from the first frame of the background image, then take the absolute value to get the background difference matrix; Detecting the ignition quantity of the edge pulse sequence and the detection pulse sequence includes: inputting the edge pulse sequence into the edge input neuron and inputting the detection pulse sequence into the main input neuron; The edge input neurons are used to suppress the pulse output of the main input neurons within a preset suppression radius; the weight of each edge input neuron connected to the main input neurons within a pixel space distance of 1 is set to -1, and the others are set to 1; The main detection neuron generates a first pulse sequence by judging the pulse sequence output by the main input neuron based on a first threshold within a preset neuron receptive field window; The edge detection neuron generates a second pulse sequence by performing a judgment based on a second threshold within the preset neuron receptive field window on the pulse sequence output by the edge input neuron; The first pulse sequence and the second pulse sequence are detected for ignition quantity by using human body detection neurons, and when the ignition quantity exceeds an ignition quantity threshold, it is determined that a human body exists in the target video corresponding to the first pulse sequence and the second pulse sequence.

2. The method for detecting human movement based on a pulse neural network according to claim 1, characterized in that: The step of generating a first pulse sequence by judging the pulse sequence output by the main input neuron through the main detection neuron based on a first threshold within a preset neuron receptive field window includes: Performing sliding window detection on the pulse sequence output by the subject input neuron through the preset neuron receptive field window to form the first pulse sequence; When the number of elements that are 1 in the pulse sequence within the preset neuron receptive field window is greater than or equal to the first threshold, the element of the first pulse sequence corresponding to the preset neuron receptive field window is recorded as 1; When the number of elements that are 1 in the pulse sequence within the current preset neuron receptive field window is less than a first threshold, the element of the first pulse sequence corresponding to the preset neuron receptive field window is recorded as 0.

3. The method for detecting human movement based on a pulse neural network according to claim 1, characterized in that: The step of generating a second pulse sequence by performing a judgment based on a second threshold within the preset neuron receptive field window on the pulse sequence output by the edge input neuron through the edge detection neuron includes: Performing sliding window detection on the pulse sequence output by the edge input neuron through the preset neuron receptive field window to form the second pulse sequence; When the number of elements that are 1 in the pulse sequence within the preset neuron receptive field window is greater than or equal to the second threshold, the element of the second pulse sequence corresponding to the preset neuron receptive field window is recorded as 1; When the number of elements that are 1 in the pulse sequence within the current preset neuron receptive field window is less than the second threshold, the elements of the second pulse sequence corresponding to the preset neuron receptive field window are recorded as 0.

4. The method for detecting human movement based on a pulse neural network according to claim 1, characterized in that: Performing ignition number detection on the first pulse sequence and the second pulse sequence by using human body detection neurons, and determining that a human body exists in the target video corresponding to the first pulse sequence and the second pulse sequence when the ignition number exceeds an ignition number threshold, comprises: The number of elements that are 1 in the first pulse sequence and the second pulse sequence is counted by human body detection neurons as the firing number.

5. The method for detecting human movement based on a pulse neural network according to claim 1, characterized in that: Using the first threshold coding to encode all the edge matrices of the target video to form an edge pulse sequence includes: Encode the elements in the edge matrix that are higher than or equal to the first threshold code as 1, and encode the elements in the edge matrix that are lower than the first threshold code as 0; All the encoded edge matrices of the target video are used as the three-dimensional edge pulse sequence.

6. The method for detecting human movement based on a pulse neural network according to claim 1, characterized in that: Using the second threshold coding to encode all the background difference matrices of the target video to form a detection pulse sequence includes: Encode the elements in the background difference matrix that are higher than or equal to the second threshold code as 1, and encode the elements in the background difference matrix that are lower than the second threshold code as 0; All the encoded background difference matrices of the target video are used as the three-dimensional detection pulse sequence.

7. A human movement detection system based on a pulse neural network using the method described in any one of claims 1 to 6, characterized in that: include: An acquisition unit is configured to perform image sampling on a target video to acquire a plurality of target images; A matrix generating unit is configured to perform edge extraction on the target image to form an edge matrix, and perform background difference calculation on the target image according to the background image to form a background difference matrix; The background image is one of the target images or a preset background image; An encoding unit is configured to encode all the edge matrices of the target video using a first threshold encoding to form an edge pulse sequence, and to encode all the background difference matrices of the target video using a second threshold encoding to form a detection pulse sequence; Elements in the edge pulse sequence and the detection pulse sequence are all 0 or 1; The detection unit is configured to perform ignition quantity detection on the edge pulse sequence and the detection pulse sequence, and use the ignition quantity detection result as the human body movement detection result of the target video.

8. The human body movement detection system based on pulse neural network according to claim 7 is characterized in that: The detection unit is further configured to: inputting the edge pulse sequence into the edge input neuron and inputting the detection pulse sequence into the main input neuron; Inhibiting the pulse output of the main input neuron within a preset inhibition radius by the edge input neuron; The main detection neuron generates a first pulse sequence by judging the pulse sequence output by the main input neuron based on a first threshold within a preset neuron receptive field window; The edge detection neuron generates a second pulse sequence by performing a judgment based on a second threshold within the preset neuron receptive field window on the pulse sequence output by the edge input neuron; The first pulse sequence and the second pulse sequence are detected for ignition quantity by using human body detection neurons, and when the ignition quantity exceeds an ignition quantity threshold, it is determined that a human body exists in the target video corresponding to the first pulse sequence and the second pulse sequence.

9. The human body movement detection system based on pulse neural network according to claim 8, characterized in that: The detection unit is further configured to: Performing sliding window detection on the pulse sequence output by the subject input neuron through the preset neuron receptive field window to form the first pulse sequence; When the number of elements that are 1 in the pulse sequence within the preset neuron receptive field window is greater than or equal to the first threshold, the element of the first pulse sequence corresponding to the preset neuron receptive field window is recorded as 1; When the number of elements that are 1 in the pulse sequence within the current preset neuron receptive field window is less than a first threshold, the element of the first pulse sequence corresponding to the preset neuron receptive field window is recorded as 0.