Infrared blocking feature fusion personnel counting method and system

Abstract

This invention relates to the fields of infrared data processing and personnel counting technology, specifically to a personnel counting method and system based on infrared blocking feature fusion. The method includes: preprocessing the detected signal using an adaptive noise cancellation algorithm; identifying saturated channels in real time using ambient light sampling and compensating with adjacent channels, combined with continuous filtering using a multi-channel blocking spatiotemporal map; and introducing an environmental coding supernetwork to dynamically generate the weights and bias parameters of the main fusion network based on real-time environmental noise and light interference characteristics. This invention solves the technical problems in existing technologies, such as the inability to separate environmental noise at the signal level and the inability to dynamically reconstruct fusion parameters based on environmental conditions at the fusion decision level.

Description

Technical Field

[0001] This invention relates to the fields of infrared data processing and personnel counting technology, and more specifically, to a personnel counting method and system based on infrared blocking feature fusion. Background Technology

[0002] In the field of people counting, active infrared light curtains count and determine direction by detecting the obstruction of infrared beams by human bodies, but they cannot distinguish between human bodies and inanimate moving objects (such as suitcases and trolleys). Passive pyroelectric (PIR) sensors identify heat source attributes by sensing changes in human infrared radiation, but they have weak spatial resolution and suffer from severe signal aliasing when multiple people are operating simultaneously.

[0003] Existing technologies have the following problems: First, at the signal level, PIR sensors are highly susceptible to environmental temperature disturbances. Air conditioning airflow, heat waves in the lobby, etc., can cause baseline drift and spurious waveforms in the output signal. Existing technologies include solutions that use an additional environmental reference sensor and direct differential subtraction to eliminate such interference. However, the direct differential method implicitly assumes that the amplitude and phase of environmental interference reaching the working sensor and the reference sensor are completely identical. In actual deployments, due to differences in sensor installation locations and slight variations in circuit characteristics, the amplitude and phase of the interference signal often deviate. Simple subtraction cannot accurately eliminate noise components, and the residual noise will distort the subsequently extracted pyroelectric characteristics (such as peak and valley amplitudes, rising slope, etc.), leading to feature distortion.

[0004] Second, at the fusion decision level, existing fusion methods, whether using fixed rules or deep learning networks, have fixed fusion parameters (or weights) once trained, making it impossible to adaptively adjust them according to real-time environmental interference. Even with the introduction of attention mechanisms, their modulation capability is limited to channel-level weighting of extracted features, failing to address the nonlinear and coupling effects of complex environmental interference (such as the simultaneous occurrence of strong light and heat waves) on sensor signals.

[0005] Therefore, there is an urgent need for a personnel counting method that can accurately separate environmental noise at the signal level and dynamically reconstruct fusion parameters based on environmental conditions at the fusion decision level. Summary of the Invention

[0006] The purpose of this application is to provide a method and system for personnel counting based on infrared blocking feature fusion, which solves the technical problems existing in the prior art, such as the inability to separate environmental noise at the signal level and the inability to dynamically reconstruct fusion parameters according to environmental conditions at the fusion decision level.

[0007] To solve the above-mentioned technical problems, the solution adopted in this application is as follows:

[0008] A method for personnel counting based on infrared blocking feature fusion, comprising:

[0009] Acquire multi-channel blocking signals, multi-channel thermal radiation signals, environmental thermal noise reference signals, and ambient light interference sampling signals;

[0010] Using an adaptive noise cancellation algorithm, with the ambient thermal noise reference signal as the reference noise input, a thermal radiation signal sequence with the ambient noise component eliminated is obtained;

[0011] Based on the ambient light interference sampling signal, invalid signal channels that are continuously interfered with by ambient light are identified. The invalid signal channels are compensated by adjacent valid signal channels to obtain a compensated blocking signal sequence. The blocking signal sequence is then subjected to spatiotemporal continuity filtering to obtain an effective occlusion event set.

[0012] Blocking features are extracted from the set of effective blocking events, pyroelectric features are extracted from the thermal radiation signal sequence, and the blocking features and pyroelectric features are concatenated to obtain a joint feature vector;

[0013] The environmental thermal noise reference signal and the ambient light interference sampling signal are encoded to generate an environmental feature vector;

[0014] The environmental feature vector is input into the environmental coding supernetwork, and the environmental coding supernetwork outputs the weight parameter matrix and bias parameters of the main fusion network.

[0015] The joint feature vector is input into the main fusion network, which performs forward computation using the weight parameter matrix and bias parameters, and outputs the number and category of people passing through.

[0016] The number of identified personnel is accumulated by direction.

[0017] Preferably, the method of using an adaptive noise cancellation algorithm to denoise the thermal radiation signal of each channel to obtain a thermal radiation signal sequence with environmental noise components eliminated includes:

[0018] An adaptive filter is constructed by using an environmental thermal noise reference signal as the reference input signal of the adaptive filter and one of the thermal radiation signals from the multi-channel thermal radiation signal as the desired response signal of the adaptive filter.

[0019] At each sampling moment, the adaptive filter calculates the output signal based on the reference input signal and the current weighting coefficients; the output signal is a real-time estimate of the environmental noise component contained in the thermal radiation signal.

[0020] The error signal is obtained by subtracting the output signal of the adaptive filter from the thermal radiation signal of this channel; the error signal is the pure thermal radiation signal after eliminating the environmental noise component corresponding to the thermal radiation signal of this channel.

[0021] Using the error signal, the weight coefficients of the adaptive filter are iteratively updated according to the weight coefficient update formula of the least mean square algorithm;

[0022] Repeat the above steps at each sampling time to make the weight coefficients of the adaptive filter converge and its output signal close to the actual environmental noise component contained in the thermal radiation signal, thereby suppressing the environmental noise component in the error signal obtained through the output signal.

[0023] The above processing is performed on each of the multi-channel thermal radiation signals to obtain multiple pure thermal radiation signals, which together form a thermal radiation signal sequence that eliminates environmental noise components.

[0024] Preferably, the specific implementation method for identifying invalid signal channels subject to continuous ambient light interference based on the ambient light interference sampling signal, and compensating the invalid signal channels using adjacent valid signal channels to obtain a compensated blocking signal sequence includes:

[0025] The ambient light interference sampling signal is compared with a preset light interference threshold. When the amplitude of the ambient light interference sampling signal exceeds the light interference threshold, it is determined that there is strong infrared light interference in the current environment.

[0026] Monitor the output signal of the signal channel and mark the signal channel whose output signal is continuously in a saturated state as an invalid signal channel;

[0027] For invalid signal channels, logical compensation is performed using the infrared beam blocking state of adjacent valid signal channels at the same sampling time to obtain a compensated blocking signal sequence.

[0028] Preferably, the method for performing spatiotemporal continuity filtering on the blocking signal sequence to obtain an effective occlusion event set includes the following steps:

[0029] The compensated blocking signal sequence is constructed into a multi-channel blocking spatiotemporal diagram within a preset time window;

[0030] In a multi-channel blocking spatiotemporal map, a region of pixels that is continuously occluded in time and space is marked as a candidate occlusion event;

[0031] For each candidate occlusion event, calculate its time span; if its time span is less than the preset minimum human passage time, then mark the candidate occlusion event as an interference event and exclude it.

[0032] Calculate the spatial height of the remaining candidate occlusion events; if the spatial height is less than the preset minimum human height, mark the candidate occlusion event as an interference event and exclude it.

[0033] For candidate occlusion events filtered by time span and spatial height, if there is a vertical gap inside, the occlusion time synchronization of the upper and lower parts of the vertical gap is calculated. Based on the occlusion time synchronization, the candidate occlusion times are merged or segmented to obtain the effective occlusion event set.

[0034] Preferably, the specific implementation method for extracting blocking features from the effective blocking event set, extracting pyroelectric features from the thermal radiation signal sequence, and concatenating the blocking features and pyroelectric features to obtain a joint feature vector includes:

[0035] Blocking features are extracted from the effective occlusion event set. The blocking features include the spatial height, spatial width, and duration of the occlusion area.

[0036] Pyroelectric features are extracted from the thermal radiation signal sequence. The pyroelectric features include the peak and valley amplitudes, peak widths, rising slopes, and falling slopes of the waveforms of each thermal radiation signal.

[0037] The blocking feature and the pyroelectric feature are concatenated to obtain a joint feature vector.

[0038] Preferably, the method for encoding the environmental thermal noise reference signal and the environmental light interference sampling signal to generate an environmental feature vector includes:

[0039] Thermal noise features are extracted from the environmental thermal noise reference signal. The thermal noise features include thermal noise amplitude and thermal noise fluctuation frequency.

[0040] Light interference features are extracted from ambient light interference sampling signals. These features include light interference intensity and the proportion of invalid channels.

[0041] The thermal noise features and optical interference features are spliced ​​together to obtain the environmental feature vector.

[0042] Preferably, the method for inputting the environmental feature vector into the environmental encoding supernetwork and outputting the weight parameters and bias parameters of the main fusion network includes the following steps:

[0043] Construct an environment coding supernetwork, which is a fully connected neural network;

[0044] The environmental feature vector is input into the constructed environmental coding supernetwork, which outputs the weight parameter matrix and bias vector of each layer of the main fusion network through forward computation.

[0045] Preferably, the joint feature vector is input into the main fusion network, which performs forward computation using the weight parameter matrix and bias parameters to output the number and category of people passing through. The specific implementation method includes:

[0046] Construct a main fusion network, which is a fully connected classification network;

[0047] The joint feature vector is input into the main fusion network, which performs layer-by-layer forward computation based on the weight parameter matrix and bias vector generated by the environment coding supernetwork to obtain the output vector.

[0048] Input the output vector into the Softmax function to obtain the probability distribution of the number of people in each category. Take the category with the highest probability as the classification result of the number of people.

[0049] Preferably, the method for accumulating the number of identified personnel by direction includes the following steps:

[0050] The direction of passage for pedestrians is determined by the order in which the infrared beams are blocked.

[0051] Set the entry counter and exit counter, both with an initial value of zero;

[0052] When the number of people output by the main fusion network is When the direction of travel is determined to be the inbound direction, the count value of the inbound counter is incremented. ;

[0053] When the number of people output by the main fusion network is When the direction of travel is determined to be the departure direction, the departure counter value is incremented. ;

[0054] When the number of people in the category output by the main fusion network is 0, no counters are updated.

[0055] An infrared blocking feature fusion-based personnel counting system, applicable to the aforementioned infrared blocking feature fusion-based personnel counting method, includes:

[0056] The signal acquisition unit is used to acquire multi-channel blocking signals, multi-channel thermal radiation signals, environmental thermal noise reference signals, and ambient light interference sampling signals.

[0057] The denoising unit, connected to the signal acquisition unit, is used to obtain a thermal radiation signal sequence with environmental noise components eliminated by using an adaptive noise cancellation algorithm and taking the environmental thermal noise reference signal as the reference noise input.

[0058] The signal processing unit, connected to the noise reduction unit, is used to identify invalid signal channels that are continuously interfered with by ambient light based on the ambient light interference sampling signal, compensate the invalid signal channels using adjacent valid signal channels to obtain a compensated blocking signal sequence, and perform spatiotemporal continuity filtering on the blocking signal sequence to obtain an effective occlusion event set.

[0059] The feature extraction and splicing unit, connected to the signal processing unit, is used to extract blocking features from the effective blocking event set, extract pyroelectric features from the thermal radiation signal sequence, and splice the blocking features and pyroelectric features to obtain a joint feature vector.

[0060] The environmental feature encoding unit, connected to the feature extraction and splicing unit, is used to encode the environmental thermal noise reference signal and the environmental light interference sampling signal to generate an environmental feature vector;

[0061] The environment coding supernetwork, connected to the environment feature coding unit, is used to receive environment feature vectors and output the weight parameter matrix and bias parameters of the main fusion network.

[0062] The main fusion network, connected to the environment coding supernetwork, is used to receive the joint feature vector and perform forward computation using the weight parameter matrix and bias parameters output by the environment coding supernetwork to output the number and category of people passing through.

[0063] The directional accumulation unit, connected to the main fusion network, is used to accumulate the number of identified personnel by direction.

[0064] The technical solution of this application has at least the following advantages and beneficial effects:

[0065] The personnel counting method provided by this invention first uses an adaptive noise cancellation algorithm to preprocess the detection signal, using an environmental thermal noise reference signal as the reference noise input to filter out the environmental noise component in the detection signal, avoiding distortion of the pyroelectric characteristics in the signal caused by residual thermal noise. On this basis, ambient light sampling is used to identify saturated channels in real time and compensate with adjacent channels. Combined with a multi-channel blocking spatiotemporal map, continuous filtering is performed to eliminate interference from strong light saturation, small objects, and local occlusion. Furthermore, an environmental coding supernetwork is introduced to dynamically generate fusion network parameters based on real-time environmental noise and light interference characteristics. This enables the main fusion network to adaptively fuse the preprocessed and interference-removed multi-channel signals according to the fusion network parameters, solving the technical problems in the prior art, such as the inability to separate environmental noise at the signal level and the inability to dynamically reconstruct fusion parameters based on environmental conditions at the fusion decision level. Attached Figure Description

[0066] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0067] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0068] Example 1

[0069] See Figure 1 The present invention provides a method for personnel counting based on infrared blocking feature fusion, comprising the following steps:

[0070] Acquire multi-channel blocking signals, multi-channel thermal radiation signals, environmental thermal noise reference signals, and ambient light interference sampling signals;

[0071] Using an adaptive noise cancellation algorithm, with the ambient thermal noise reference signal as the reference noise input, a thermal radiation signal sequence with the ambient noise component eliminated is obtained;

[0072] Based on the ambient light interference sampling signal, invalid signal channels that are continuously interfered with by ambient light are identified. The invalid signal channels are compensated by adjacent valid signal channels to obtain a compensated blocking signal sequence. The blocking signal sequence is then subjected to spatiotemporal continuity filtering to obtain an effective occlusion event set.

[0073] Blocking features are extracted from the set of effective blocking events, pyroelectric features are extracted from the thermal radiation signal sequence, and the blocking features and pyroelectric features are concatenated to obtain a joint feature vector;

[0074] The environmental thermal noise reference signal and the ambient light interference sampling signal are encoded to generate an environmental feature vector;

[0075] The environmental feature vector is input into the environmental coding supernetwork, and the environmental coding supernetwork outputs the weight parameter matrix and bias parameters of the main fusion network.

[0076] The joint feature vector is input into the main fusion network, which performs forward computation using the weight parameter matrix and bias parameters, and outputs the number and category of people passing through.

[0077] The number of identified personnel is accumulated by direction.

[0078] In this embodiment, the acquisition of multi-channel blocking signals, multi-channel thermal radiation signals, environmental thermal noise reference signals, and ambient light interference sampling signals is specifically implemented as follows:

[0079] An active infrared light curtain, a passive pyroelectric sensor array, a passive infrared environmental reference sensor, and an active infrared ambient light sampling receiver are deployed in the detection area; the detection area is a passageway for personnel.

[0080] Infrared emitting and receiving arrays are arranged opposite each other on both sides of the passage. The infrared emitting array consists of multiple infrared emitting tubes arranged at vertical intervals, and the receiving array consists of a corresponding number of infrared receiving tubes arranged at the same intervals, forming an active infrared light curtain composed of multiple parallel infrared beams. This active infrared light curtain can cover the vertical detection section of the passage. When a person passes through the light curtain, each infrared beam is blocked, generating a multi-channel blocking signal. The multi-channel blocking signal is composed of the blocking status of each infrared beam, reflecting whether the corresponding infrared beam is blocked at each sampling moment.

[0081] Among them, at least one infrared receiver in the receiving array is used to collect the infrared light intensity in the environment within the channel and output the ambient light interference sampling signal, and its corresponding infrared transmitter is turned off or not set.

[0082] Unlike pedestrian walkways which serve as detection areas, the channels in multi-channel blocking signals refer to signal channels formed by infrared emitting diodes and infrared receiving diodes that generate blocking signals. One infrared emitting diode and one infrared receiving diode constitute one signal channel.

[0083] It should be noted that the active infrared light curtains are deployed in two rows along the passage direction, referred to as the first light curtain and the second light curtain. When a person enters the inner side from the outside, the first light curtain is triggered first, followed by the second light curtain. When a person leaves the outer side from the inner side, the second light curtain is triggered first, followed by the first light curtain. The passage direction is determined by the triggering order of the two rows of light curtains.

[0084] Several pyroelectric infrared sensors are arranged above the channel to form a passive pyroelectric sensor array. The pyroelectric infrared sensors are arranged at intervals along the width of the channel, and the field of view of the passive pyroelectric sensor array covers the horizontal detection section of the channel. Each pyroelectric infrared sensor outputs a thermal radiation signal that reflects the change of thermal radiation within its field of view, forming a multi-channel thermal radiation signal.

[0085] At least one infrared environmental reference sensor for detecting changes in ambient temperature is installed in the channel. The infrared environmental reference sensor can be a pyroelectric infrared sensor. Its infrared detection end is physically shielded by a light shield so that it does not respond to personnel passing through the channel, but only detects changes in ambient temperature and outputs an ambient thermal noise reference signal.

[0086] Specifically, after the above-mentioned sensors collect signals, a signal conditioning circuit needs to be set up. The signal conditioning circuit includes a signal amplification circuit, a filtering circuit, and an analog-to-digital conversion circuit. The signal amplification circuit, filtering circuit, and analog-to-digital conversion circuit all use existing circuits, so their specific circuit structures and signal processing processes will not be described in detail here. Those skilled in the art can use existing conventional signal conditioning circuits to implement it. The innovation of this invention lies in the personnel counting method based on infrared blocking feature fusion. The hardware circuit structure is not within the protection scope of this invention.

[0087] As an example, the pyroelectric infrared sensor can be an Excelitas PYD 1398 pyroelectric infrared sensor; the infrared emitting tube can be an OSRAM SFH 4546 infrared emitting tube; and the infrared receiving tube can be an OSRAM SFH 213FA infrared receiving tube. The above is only an exemplary description of the selection of some component models. This invention does not limit the specific component models and can be changed according to the actual scenario and requirements.

[0088] In this embodiment, an adaptive noise cancellation algorithm is used to denoise the thermal radiation signal of each channel to obtain a thermal radiation signal sequence with environmental noise components eliminated. The specific implementation method includes:

[0089] An adaptive filter is constructed by using an environmental thermal noise reference signal as the reference input signal of the adaptive filter and one of the thermal radiation signals from the multi-channel thermal radiation signal as the desired response signal of the adaptive filter.

[0090] At each sampling moment, the adaptive filter calculates the output signal based on the reference input signal and the current weighting coefficients; the output signal is a real-time estimate of the environmental noise component contained in the thermal radiation signal.

[0091] The error signal is obtained by subtracting the output signal of the adaptive filter from the thermal radiation signal of this channel; the error signal is the pure thermal radiation signal after eliminating the environmental noise component corresponding to the thermal radiation signal of this channel.

[0092] Using the error signal, the weight coefficients of the adaptive filter are iteratively updated according to the weight coefficient update formula of the least mean square algorithm;

[0093] Repeat the above steps at each sampling time to make the weight coefficients of the adaptive filter converge and its output signal close to the actual environmental noise component contained in the thermal radiation signal, thereby suppressing the environmental noise component in the error signal obtained through the output signal.

[0094] The above processing is performed on each of the multi-channel thermal radiation signals to obtain multiple pure thermal radiation signals, which together form a thermal radiation signal sequence that eliminates environmental noise components.

[0095] Specifically, in the above processing steps, the adaptive filter adopts a finite impulse response (FIR) filter structure; the order of the FIR filter is... As an example, the order of a finite impulse response filter. Set to order 32; its initial weight coefficients are set to zero vector; the step size factor of the least mean square algorithm is set to... As an example, the step size factor for the least mean square algorithm is set to 0.01.

[0096] Define the adaptive filter in the first... The weight coefficient vector at each sampling time point for:

[0097] ;

[0098] Reference input signal vector for:

[0099] ;

[0100] in, The environmental thermal noise reference signal is at the first The value at each sampling time;

[0101] At each sampling time, the output signal of the adaptive filter is :

[0102] ;

[0103] in, This is a real-time estimate of the environmental noise component contained in the thermal radiation signal of this path;

[0104] Assume that the thermal radiation signal of this path is at the th The value at each sampling time is Then the error signal for:

[0105] ;

[0106] Error signal That is, the pure thermal radiation signal after eliminating the environmental noise component corresponding to this thermal radiation signal is at the 1st... The value at each sampling time;

[0107] Using error signals The weight coefficients of the adaptive filter are iteratively updated according to the weight coefficient update formula of the least mean square algorithm:

[0108] ;

[0109] The above calculation is repeated at each sampling time, causing the weight coefficients of the adaptive filter to gradually converge, and the output signal of the adaptive filter... The error signal is closer to the actual environmental noise component contained in the thermal radiation signal. The environmental noise component is suppressed;

[0110] The above processing is performed on each of the multi-channel thermal radiation signals to obtain multiple clean thermal radiation signals. These multiple clean thermal radiation signals constitute a thermal radiation signal sequence that eliminates environmental noise components, which serves as input data for subsequent processing.

[0111] In this step, the multi-channel thermal radiation signal contains two types of signal components: one is the information on thermal radiation changes caused by people passing through the channel, and the other is the thermal radiation fluctuation caused by changes in ambient temperature. This step uses an adaptive filter based on the least mean square algorithm, with the ambient thermal noise reference signal as the reference signal, and automatically adjusts the weight coefficients of the adaptive filter iteratively. During the convergence of the weight coefficients, the adaptive filter learns the amplitude and phase relationship between the reference signal and the ambient noise component, making the output signal of the adaptive filter approximate the waveform of the actual noise component. Subtracting this output signal from the thermal radiation signal yields a high-fidelity, clean thermal radiation signal.

[0112] In this embodiment, invalid signal channels subject to continuous ambient light interference are identified based on the ambient light interference sampling signal. The invalid signal channels are then compensated using adjacent valid signal channels to obtain a compensated blocking signal sequence. The specific implementation method includes:

[0113] The ambient light interference sampling signal is compared with a preset light interference threshold. When the amplitude of the ambient light interference sampling signal exceeds the light interference threshold, it is determined that there is strong infrared light interference in the current environment.

[0114] Monitor the output signal of the signal channel and mark the signal channel whose output signal is continuously in a saturated state as an invalid signal channel;

[0115] For invalid signal channels, logical compensation is performed using the infrared beam blocking state of adjacent valid signal channels at the same sampling time to obtain a compensated blocking signal sequence.

[0116] As an example, the logical compensation method is as follows:

[0117] When an invalid signal channel is located between two adjacent valid channels in space, if both adjacent valid signal channels are blocked at the same time, then the blocking state of the invalid signal channel at that sampling time is set to the blocked state.

[0118] If the two adjacent valid signal channels are both in an unobstructed state at the same sampling time, then the blocking state of the invalid signal channel at that sampling time is set to an unobstructed state.

[0119] If one of the two adjacent valid signal channels is blocked while the other is unblocked, compensation is made based on the number of valid signal channels on both sides of the invalid signal channel that are blocked and unblocked, and the invalid signal channel is set to the state of having more valid signal channels.

[0120] If an invalid signal channel is located at the edge of the active infrared light curtain and there is only one adjacent valid signal channel on one side, the invalid signal channel will be directly set to the state of its adjacent valid signal channel.

[0121] Other logical compensation methods can also be used, and this invention does not limit them.

[0122] Specifically, the output signal of each signal channel is the original voltage value output by each infrared receiver tube, and the original voltage value can be used to determine whether the infrared receiver tube is in saturation; among them, the multi-channel blocking signal is obtained by binarizing the output signal of each signal channel;

[0123] As an example, the light interference threshold can be set to 80% of the saturation voltage of the infrared receiver tube. This invention does not limit it and it can be adjusted according to the actual scenario and accuracy requirements.

[0124] In this embodiment, the spatiotemporal continuity filtering of the blocking signal sequence is performed to obtain an effective occlusion event set. The specific implementation method includes the following steps:

[0125] The compensated blocking signal sequence is constructed into a multi-channel blocking spatiotemporal diagram within a preset time window;

[0126] In a multi-channel blocking spatiotemporal map, a region of pixels that is continuously occluded in time and space is marked as a candidate occlusion event;

[0127] For each candidate occlusion event, calculate its time span; if its time span is less than the preset minimum human passage time, then mark the candidate occlusion event as an interference event and exclude it.

[0128] Calculate the spatial height of the remaining candidate occlusion events; if the spatial height is less than the preset minimum human height, mark the candidate occlusion event as an interference event and exclude it.

[0129] For candidate occlusion events filtered by time span and spatial height, if there is a vertical gap inside, the occlusion time synchronization of the upper and lower parts of the vertical gap is calculated, and the candidate occlusion time is merged or segmented according to the occlusion time synchronization to obtain the effective occlusion event set.

[0130] Candidate occlusion times are merged or segmented based on the synchronicity of occlusion time, specifically as follows:

[0131] If the occlusion time synchronization of the upper and lower parts of the vertical gap is greater than or equal to the preset synchronization threshold, the upper and lower parts of the vertical gap are merged into one candidate occlusion event; if the occlusion time synchronization of the upper and lower parts of the vertical gap is lower than the preset synchronization threshold, the upper and lower parts of the vertical gap are divided into two candidate occlusion events.

[0132] Among them, the vertical gap is the vertically separated region formed by the unblocked infrared beam that exists inside the candidate occlusion event in the multi-channel blocking spatiotemporal diagram;

[0133] Specifically, the synchronicity of the occlusion time of the upper and lower parts of the vertical gap is recorded as follows: The preset synchronization threshold is denoted as The starting time of occlusion of the upper part of the vertical gap is recorded as . The end of the occlusion is recorded as The starting time of the shading of the lower part of the vertical gap is recorded as . The end of the occlusion is recorded as The event span of a candidate occlusion event is denoted as . Then the occlusion time synchronization for:

[0134] ;

[0135] when If the start and end times of occlusion in the upper and lower parts of the vertical gap are the same, it indicates that the occlusion is the same candidate event and is determined to be the same human body; otherwise, it is determined to be different candidate events, and the upper and lower parts are divided into two candidate events.

[0136] As an example, the preset minimum human passage time is set to 0.3 seconds, and the preset minimum human height is 30 centimeters. These can be modified according to the actual situation of the width, height, length, etc. of the pedestrian passage being monitored. This invention does not limit these modifications.

[0137] As an example, the preset synchronization threshold is set to 0.8, which is dimensionless. Those skilled in the art can modify it according to the common characteristics of the actual deployment scenario. This invention does not limit its value.

[0138] More specifically, the horizontal axis of the multi-channel blocking spatiotemporal diagram represents the sampling time, and the vertical axis represents the infrared beam number, which is also the signal channel number. The value of each pixel in the multi-channel blocking spatiotemporal diagram represents the occlusion state of the corresponding infrared beam number at the corresponding sampling time.

[0139] It should be noted that a candidate occlusion event is marked as a pixel region that is continuously occluded in time and space. Time refers to the horizontal axis direction of the multi-channel blocking spatiotemporal map (time), and space refers to the vertical axis direction of the multi-channel blocking spatiotemporal map (infrared beam number).

[0140] In this embodiment, blocking features are extracted from the set of effective blocking events, pyroelectric features are extracted from the thermal radiation signal sequence, and the blocking features and pyroelectric features are concatenated to obtain a joint feature vector. The specific implementation method includes:

[0141] Blocking features are extracted from the effective occlusion event set. The blocking features include the spatial height, spatial width, and duration of the occlusion area.

[0142] Pyroelectric features are extracted from the thermal radiation signal sequence. The pyroelectric features include the peak and valley amplitudes, peak widths, rising slopes, and falling slopes of the waveforms of each thermal radiation signal.

[0143] The blocking feature and the pyroelectric feature are concatenated to obtain a joint feature vector.

[0144] In this embodiment, the environmental thermal noise reference signal and the ambient light interference sampling signal are encoded to generate an environmental feature vector. The specific implementation method includes:

[0145] Thermal noise features are extracted from the environmental thermal noise reference signal. The thermal noise features include thermal noise amplitude and thermal noise fluctuation frequency.

[0146] Light interference features are extracted from ambient light interference sampling signals. These features include light interference intensity and the proportion of invalid channels.

[0147] The thermal noise features and optical interference features are concatenated and combined to obtain the environmental feature vector;

[0148] Wherein, the thermal noise amplitude is the difference between the peak and valley values ​​of the environmental thermal noise reference signal within the current time window; the thermal noise fluctuation frequency is the frequency corresponding to the component with the largest amplitude in the spectrum obtained after Fourier transform of the environmental thermal noise reference signal within the current time window;

[0149] The light interference intensity is the average value of the ambient light interference sampling signal within the current time window; the invalid channel ratio is marked as the ratio of the number of invalid signal channels to the total number of signal channels in the active infrared light curtain.

[0150] In this embodiment, the environmental feature vector is input into the environmental coding supernetwork, and the environmental coding supernetwork outputs the weight parameters and bias parameters of the main fusion network. The specific implementation method includes the following steps:

[0151] Construct an environment coding supernetwork, which is a fully connected neural network;

[0152] The environmental feature vector is input into the constructed environmental coding supernetwork, and the environmental coding supernetwork outputs the weight parameter matrix and bias vector of each layer of the main fusion network through forward computation.

[0153] Specifically, the structure of the environment coding hypernetwork is as follows: the input layer dimension is... Each of these corresponds to one of the four components of the environmental feature vector; the output layer dimension is the total number of elements in the weight parameter matrix and bias vector of each layer of the fusion network; several hidden layers are set between the input layer and the output layer.

[0154] In this embodiment, the joint feature vector is input into the main fusion network. The main fusion network performs forward computation using the weight parameter matrix and bias parameters, and outputs the number and category of people passing through. The specific implementation method includes:

[0155] Construct a main fusion network, which is a fully connected classification network;

[0156] The joint feature vector is input into the main fusion network, which performs layer-by-layer forward computation based on the weight parameter matrix and bias vector generated by the environment coding supernetwork to obtain the output vector.

[0157] Input the output vector into the Softmax function to obtain the probability distribution of the number of people in each category. Take the category with the highest probability as the classification result of the number of people.

[0158] Specifically, the main converged network has Fully connected layer, the first The weight parameter matrix of the layer is The bias vector is ,in ;

[0159] During each inference iteration, the context-encoding supernetwork outputs the weight matrices of each layer of the main fusion network through forward computation. and bias vector :

[0160] ;

[0161] in, This represents the forward computation function of the environment coding hypernetwork. Encode the network parameters of the hypernetwork itself for the environment; This represents the environmental feature vector.

[0162] The joint feature vector is input into the main fusion network. The main fusion network performs layer-by-layer forward computation based on the weight parameter matrix and bias vector generated by the environment coding supernetwork. The computation process is as follows:

[0163] ;

[0164] ;

[0165] ;

[0166] in, For joint feature vectors, For the first The output of the layer, For activation function, The output vector of the main fusion network;

[0167] As an example, the main fusion network is configured as a two-layer fully connected structure, i.e. The first layer has an input dimension equal to the dimension of the joint feature vector, and an output dimension of 64; the second layer has an input dimension of 64, and an output dimension equal to the number of people and categories; activation function. This invention uses the ReLU function and does not... The specific values ​​of parameters are restricted, and those skilled in the art can modify them according to the actual situation.

[0168] In this embodiment, the number of identified personnel is accumulated by direction, and the specific implementation method includes the following steps:

[0169] The direction of passage for pedestrians is determined by the order in which the infrared beams are blocked.

[0170] Set the entry counter and exit counter, both with an initial value of zero;

[0171] When the number of people output by the main fusion network is When the direction of travel is determined to be the inbound direction, the count value of the inbound counter is incremented. ;

[0172] When the number of people output by the main fusion network is When the direction of travel is determined to be the departure direction, the departure counter value is incremented. ;

[0173] When the number of people in the category output by the main fusion network is 0, no counters are updated.

[0174] Another aspect of the present invention provides a personnel counting system based on infrared blocking feature fusion, comprising:

[0175] The signal acquisition unit is used to acquire multi-channel blocking signals, multi-channel thermal radiation signals, environmental thermal noise reference signals, and ambient light interference sampling signals.

[0176] The denoising unit, connected to the signal acquisition unit, is used to obtain a thermal radiation signal sequence with environmental noise components eliminated by using an adaptive noise cancellation algorithm and taking the environmental thermal noise reference signal as the reference noise input.

[0177] The signal processing unit, connected to the noise reduction unit, is used to identify invalid signal channels that are continuously interfered with by ambient light based on the ambient light interference sampling signal, compensate the invalid signal channels using adjacent valid signal channels to obtain a compensated blocking signal sequence, and perform spatiotemporal continuity filtering on the blocking signal sequence to obtain an effective occlusion event set.

[0178] The feature extraction and splicing unit, connected to the signal processing unit, is used to extract blocking features from the effective blocking event set, extract pyroelectric features from the thermal radiation signal sequence, and splice the blocking features and pyroelectric features to obtain a joint feature vector.

[0179] The environmental feature encoding unit, connected to the feature extraction and splicing unit, is used to encode the environmental thermal noise reference signal and the environmental light interference sampling signal to generate an environmental feature vector;

[0180] The environment coding supernetwork, connected to the environment feature coding unit, is used to receive environment feature vectors and output the weight parameter matrix and bias parameters of the main fusion network.

[0181] The main fusion network, connected to the environment coding supernetwork, is used to receive the joint feature vector and perform forward computation using the weight parameter matrix and bias parameters output by the environment coding supernetwork to output the number and category of people passing through.

[0182] The directional accumulation unit, connected to the main fusion network, is used to accumulate the number of identified personnel by direction.

[0183] The various embodiments of the present invention have now been described in detail. To avoid obscuring the concept of the invention, some details known in the art have not been described. Those skilled in the art will fully understand how to implement the technical solutions of this invention based on the above description, and the scope of the invention is defined by the appended claims.

Claims

1. An infrared-blocking feature fused people counting method, characterized by, include: Acquire multi-channel blocking signals, multi-channel thermal radiation signals, environmental thermal noise reference signals, and ambient light interference sampling signals; Using an adaptive noise cancellation algorithm, with the ambient thermal noise reference signal as the reference noise input, a thermal radiation signal sequence with the ambient noise component eliminated is obtained; Based on the ambient light interference sampling signal, invalid signal channels that are continuously interfered with by ambient light are identified. The invalid signal channels are compensated by adjacent valid signal channels to obtain a compensated blocking signal sequence. The blocking signal sequence is then subjected to spatiotemporal continuity filtering to obtain an effective occlusion event set. Blocking features are extracted from the set of effective blocking events, pyroelectric features are extracted from the thermal radiation signal sequence, and the blocking features and pyroelectric features are concatenated to obtain a joint feature vector; The environmental thermal noise reference signal and the ambient light interference sampling signal are encoded to generate an environmental feature vector; The environmental feature vector is input into the environmental coding supernetwork, and the environmental coding supernetwork outputs the weight parameter matrix and bias parameters of the main fusion network. The joint feature vector is input into the main fusion network, which performs forward computation using the weight parameter matrix and bias parameters, and outputs the number and category of people passing through. The number of identified personnel is accumulated by direction.

2. The method of claim 1, wherein the infrared-blocking feature is a fusion of a person counting method, characterized in that, The method for using an adaptive noise cancellation algorithm, with an environmental thermal noise reference signal as the reference noise input, to obtain a thermal radiation signal sequence with environmental noise components eliminated, includes the following specific implementation methods: An adaptive filter is constructed by using an environmental thermal noise reference signal as the reference input signal of the adaptive filter and one of the thermal radiation signals from the multi-channel thermal radiation signal as the desired response signal of the adaptive filter. At each sampling moment, the adaptive filter calculates the output signal based on the reference input signal and the current weighting coefficients; the output signal is a real-time estimate of the environmental noise component contained in the thermal radiation signal. The error signal is obtained by subtracting the output signal of the adaptive filter from the thermal radiation signal of this channel; the error signal is the pure thermal radiation signal after eliminating the environmental noise component corresponding to the thermal radiation signal of this channel. Using the error signal, the weight coefficients of the adaptive filter are iteratively updated according to the weight coefficient update formula of the least mean square algorithm; Repeat the above steps at each sampling time to make the weight coefficients of the adaptive filter converge and its output signal close to the actual environmental noise component contained in the thermal radiation signal, thereby suppressing the environmental noise component in the error signal obtained through the output signal. The above processing is performed on each of the multi-channel thermal radiation signals to obtain multiple pure thermal radiation signals, which together form a thermal radiation signal sequence that eliminates environmental noise components.

3. The method of claim 2, wherein the infrared-blocking feature is a fusion of a person counter, and wherein the infrared-blocking feature is a fusion of a person counter. The method for identifying invalid signal channels subject to continuous ambient light interference based on ambient light interference sampling signals, and compensating for the invalid signal channels using adjacent valid signal channels to obtain a compensated blocking signal sequence, specifically includes: The ambient light interference sampling signal is compared with a preset light interference threshold. When the amplitude of the ambient light interference sampling signal exceeds the light interference threshold, it is determined that there is strong infrared light interference in the current environment. Monitor the output signal of the signal channel and mark the signal channel whose output signal is continuously in a saturated state as an invalid signal channel; For invalid signal channels, logical compensation is performed using the infrared beam blocking state of adjacent valid signal channels at the same sampling time to obtain a compensated blocking signal sequence.

4. The method of claim 3, wherein the infrared-blocking feature is a fusion of a person counter, and wherein the infrared-blocking feature is a fusion of a person counter. The method for performing spatiotemporal continuity filtering on the blocking signal sequence to obtain an effective occlusion event set includes the following steps: The compensated blocking signal sequence is constructed into a multi-channel blocking spatiotemporal diagram within a preset time window; In a multi-channel blocking spatiotemporal map, a region of pixels that is continuously occluded in time and space is marked as a candidate occlusion event; For each candidate occlusion event, calculate its time span; if its time span is less than the preset minimum human passage time, then mark the candidate occlusion event as an interference event and exclude it. Calculate the spatial height of the remaining candidate occlusion events; if the spatial height is less than the preset minimum human height, mark the candidate occlusion event as an interference event and exclude it. For candidate occlusion events filtered by time span and spatial height, if there is a vertical gap inside, the occlusion time synchronization of the upper and lower parts of the vertical gap is calculated. Based on the occlusion time synchronization, the candidate occlusion times are merged or segmented to obtain the effective occlusion event set.

5. The personnel counting method based on infrared blocking feature fusion according to claim 4, characterized in that, The specific implementation method for extracting blocking features from the effective blocking event set, extracting pyroelectric features from the thermal radiation signal sequence, and concatenating the blocking features and pyroelectric features to obtain a joint feature vector includes: Blocking features are extracted from the effective occlusion event set. The blocking features include the spatial height, spatial width, and duration of the occlusion area. Pyroelectric features are extracted from the thermal radiation signal sequence. The pyroelectric features include the peak and valley amplitudes, peak widths, rising slopes, and falling slopes of the waveforms of each thermal radiation signal. The blocking feature and the pyroelectric feature are concatenated to obtain a joint feature vector.

6. The personnel counting method based on infrared blocking feature fusion according to claim 5, characterized in that, The specific implementation method for encoding the environmental thermal noise reference signal and the ambient light interference sampling signal to generate an environmental feature vector includes: Thermal noise features are extracted from the environmental thermal noise reference signal. The thermal noise features include thermal noise amplitude and thermal noise fluctuation frequency. Light interference features are extracted from ambient light interference sampling signals. These features include light interference intensity and the proportion of invalid channels. The thermal noise features and optical interference features are spliced ​​together to obtain the environmental feature vector.

7. The personnel counting method based on infrared blocking feature fusion according to claim 6, characterized in that, The method for inputting environmental feature vectors into an environmental encoding supernetwork, and for the environmental encoding supernetwork to output the weight parameters and bias parameters of the main fusion network, includes the following steps: Construct an environment coding supernetwork, which is a fully connected neural network; The environmental feature vector is input into the constructed environmental coding supernetwork, which outputs the weight parameter matrix and bias vector of each layer of the main fusion network through forward computation.

8. The personnel counting method based on infrared blocking feature fusion according to claim 7, characterized in that, The joint feature vector is input into the main fusion network, which performs forward computation using the weight parameter matrix and bias parameters to output the number and category of people passing through. The specific implementation method includes: Construct a main fusion network, which is a fully connected classification network; The joint feature vector is input into the main fusion network, which performs layer-by-layer forward computation based on the weight parameter matrix and bias vector generated by the environment coding supernetwork to obtain the output vector. Input the output vector into the Softmax function to obtain the probability distribution of the number of people in each category. Take the category with the highest probability as the classification result of the number of people.

9. The personnel counting method based on infrared blocking feature fusion according to claim 8, characterized in that, The method for accumulating the number of identified personnel by direction includes the following steps: The direction of passage for pedestrians is determined by the order in which the infrared beams are blocked. Set the entry counter and exit counter, both with an initial value of zero; When the number of people output by the main fusion network is When the direction of travel is determined to be the inbound direction, the count value of the inbound counter is incremented. ; When the number of people output by the main fusion network is When the direction of travel is determined to be the departure direction, the departure counter value is incremented. ; When the number of people in the category output by the main fusion network is 0, no counters are updated.

10. A personnel counting system based on infrared blocking feature fusion, applicable to the personnel counting method based on infrared blocking feature fusion as described in any one of claims 1-9, characterized in that, include: The signal acquisition unit is used to acquire multi-channel blocking signals, multi-channel thermal radiation signals, environmental thermal noise reference signals, and ambient light interference sampling signals. The denoising unit, connected to the signal acquisition unit, is used to obtain a thermal radiation signal sequence with environmental noise components eliminated by using an adaptive noise cancellation algorithm and taking the environmental thermal noise reference signal as the reference noise input. The signal processing unit, connected to the noise reduction unit, is used to identify invalid signal channels that are continuously interfered with by ambient light based on the ambient light interference sampling signal, compensate the invalid signal channels using adjacent valid signal channels to obtain a compensated blocking signal sequence, and perform spatiotemporal continuity filtering on the blocking signal sequence to obtain an effective occlusion event set. The feature extraction and splicing unit, connected to the signal processing unit, is used to extract blocking features from the effective blocking event set, extract pyroelectric features from the thermal radiation signal sequence, and splice the blocking features and pyroelectric features to obtain a joint feature vector. The environmental feature encoding unit, connected to the feature extraction and splicing unit, is used to encode the environmental thermal noise reference signal and the environmental light interference sampling signal to generate an environmental feature vector; The environment coding supernetwork, connected to the environment feature coding unit, is used to receive environment feature vectors and output the weight parameter matrix and bias parameters of the main fusion network. The main fusion network, connected to the environment coding supernetwork, is used to receive the joint feature vector and perform forward computation using the weight parameter matrix and bias parameters output by the environment coding supernetwork to output the number and category of people passing through. The directional accumulation unit, connected to the main fusion network, is used to accumulate the number of identified personnel by direction.

Images