A crowd density detection method fusing optical flow and texture features

Abstract

The application discloses a crowd density detection method fusing optical flow and texture features, and relates to the technical field of computer vision, and steps are as follows: collecting a real-time video stream of a camera, and carrying out preprocessing through greying, Gaussian filtering and perspective correction; using image registration and affine transformation to do motion compensation; modeling based on a Gaussian mixture model, extracting foreground and generating a binary mask; analyzing foreground coverage, optical flow and texture features; predefining multiple ROIs and density early warning standards; inputting the features into a regression model to output initial density; dynamically calibrating and correcting deviation; and generating a heat map to be superimposed on a video to realize visualization. The application fuses optical flow and texture features to improve density estimation accuracy, and is resistant to light and dynamic background interference; dynamic calibration can maintain long-term accuracy, and dynamic ROIs can adapt to scene changes; a heat map can quickly identify risks, adapt to multiple scenes and meet real-time monitoring requirements.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method for detecting crowd density by fusing optical flow and texture features. Background Technology

[0002] Crowd density detection is a core technological support for public safety management, transportation hub scheduling, and large-scale event organization. Its detection accuracy and real-time performance directly affect public safety risk prevention and resource allocation efficiency. With the acceleration of urbanization, the scale of pedestrian flow in public places such as shopping malls, squares, subway stations, and train stations continues to expand, leading to frequent occurrences of complex situations such as high-density gatherings, severe crowd obstruction, and dynamic changes in pedestrian flow. This places higher demands on the robustness and adaptability of density detection technology. Accurate crowd density detection can provide managers with real-time pedestrian distribution information, timely warnings of congestion risks, and prevention of safety accidents such as stampedes. It also provides data support for decisions on traffic management, police deployment, and facility optimization, possessing significant practical application value.

[0003] Traditional crowd density detection methods are mainly divided into two categories: person-by-person detection and counting methods, and density regression methods based on single features. Person-by-person detection and counting methods rely on object detection algorithms to identify individual pedestrians and count their numbers to calculate density. While this method can achieve some success in low-density, unobstructed scenes, in high-density crowd scenes, where people severely obstruct each other, the object detection algorithm is prone to missed detections and false detections. Furthermore, the computational load of pixel-by-pixel scanning and feature matching is extremely high, making it difficult to meet the frame rate requirements of real-time monitoring and unsuitable for dynamic crowd detection scenarios in large public places. In addition, this method requires high image resolution; when the pixels of monitoring equipment are limited, the detection accuracy drops significantly, further limiting its applicability.

[0004] Density regression methods based on single features attempt to establish a mapping relationship between global or local image features and density, circumventing the occlusion problem in person-by-person detection. However, they still face numerous technical bottlenecks. Some methods rely solely on texture features, estimating density by analyzing the texture complexity of the heads and shoulders of crowds in an image, but neglecting the impact of crowd movement on density. When the crowd is dense and stationary, texture features are prone to saturation, and the texture differences at different densities shrink, leading to increased estimation bias. Methods relying solely on optical flow features, while capturing crowd movement information, are susceptible to interference in scenarios with changing lighting and dynamic backgrounds, generating random noise and reducing detection accuracy. Furthermore, perspective distortion causes a "near-to-far" effect, resulting in inconsistent pixel scales and optical flow vector lengths for the same object at different locations, directly interfering with feature extraction and density mapping. Camera shake can easily misjudge global motion as foreground, introducing invalid features. In addition, existing methods mostly use a globally uniform threshold for early warning, without considering the functional differences of different regions, which can easily lead to missed warnings in dangerous areas or false warnings in safe areas. Furthermore, they lack a dynamic calibration mechanism, and the model remains fixed after training, making it difficult to adapt to feature shifts caused by changes in the scene, resulting in a continuous decline in accuracy after long-term operation. Summary of the Invention

[0005] This invention proposes a crowd density detection method that integrates optical flow and texture features to solve the problems mentioned in the prior art.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a crowd density detection method that integrates optical flow and texture features, comprising the following steps:

[0007] Video stream acquisition and preprocessing steps: Acquire real-time video stream from camera, convert color image to grayscale image; use Gaussian filter for noise reduction, perform perspective transformation for correction, and map image to a uniform scale top view plane;

[0008] Motion compensation steps: Image registration technology is used to detect camera shake, and affine transformation is used to perform motion compensation on video frames to stabilize the image and eliminate the interference of global motion on foreground judgment;

[0009] Background modeling and foreground extraction steps: Construct a dynamic background model based on Gaussian mixture model, model the brightness distribution of each pixel and update the mean and variance in real time; obtain the initial foreground region by background subtraction, distinguish shadows from real moving objects by combining color information, and generate a binarized foreground mask after thresholding.

[0010] Foreground feature analysis steps: Calculate the foreground region coverage, obtain the optical flow vector using the Lucas-Kanade algorithm, statistically analyze the motion intensity and motion consistency index, extract texture features and calculate texture complexity;

[0011] ROI partitioning steps: Based on the actual physical spatial distribution of the scene, predefine regions of interest with independent safety thresholds, clarify the physical area and density warning standards for each region, and configure the warning standards for different regions according to functional differences;

[0012] Feature fusion and density regression steps: Normalize the foreground region coverage, motion intensity, motion consistency and texture complexity features, input them into the trained gradient boosting regression model, establish the mapping relationship between features and actual number of people, and output the initial population density of each ROI;

[0013] Dynamic calibration steps: Regularly input real human headcount data from manual sampling to drive incremental learning of the regression model; simultaneously, integrate environmental data collected from multiple sensors to perform cross-validation and correct density estimation bias.

[0014] Heatmap generation and visualization steps: Normalize the density values ​​of each ROI, map the corresponding colors through a color lookup table, generate a semi-transparent color layer, and overlay it onto the original video stream to achieve real-time visualization of the crowd density heatmap.

[0015] Furthermore, it also includes a precise calculation step for motion consistency, quantifying the degree of disorder in the distribution of optical flow vector direction angles through information entropy, as shown in the formula:

[0016]

[0017] in Represents the motion consistency information entropy; Indicates the number of intervals divided by the direction angle; Indicates the first The proportion of optical flow vectors within each directional angle interval is obtained by statistically assigning the directional angles of all optical flow vectors in the foreground region.

[0018] Furthermore, it also includes a comprehensive calculation step for texture complexity, integrating the core indicators of local binary mode and gray-level co-occurrence matrix, with the formula as follows:

[0019]

[0020] in This represents the overall texture complexity value; Represents the LBP feature weights. Represents the feature weights of the gray-level co-occurrence matrix. and The values ​​are all between 0.4 and 0.6 and High-density scenes Take 0.6, Take 0.4, for low-density scenes Take 0.4, Take 0.6; This represents the energy normalization value of the local binary pattern histogram; These represent the normalized values ​​of contrast, energy, and entropy of the gray-level co-occurrence matrix, respectively.

[0021] Furthermore, in the preprocessing step, the kernel size of the Gaussian filter is set to 3×3, and the standard deviation is 0.8-1.2; perspective correction establishes a mapping relationship between pixel coordinates and physical coordinates by selecting four non-collinear physical reference points in the image, and the mapping error is controlled within 1 pixel.

[0022] Furthermore, in the background modeling step of the Gaussian mixture model, the number of mixture components for each pixel is set to 3-5, the weight update rate of the components is set to 0.001-0.005, and the initial variance is set to 20-30. The pixel is determined to belong to the background or the foreground by an adaptive threshold. The threshold is dynamically adjusted based on the average pixel brightness. When the average brightness is higher than 200, the threshold is set to 3 times the standard deviation, and when it is lower than 100, the threshold is set to 2 times the standard deviation.

[0023] Furthermore, the optical flow vector calculation adopts an improved Lucas-Kanade algorithm, selects a 3×3 window to perform local feature matching, and calculates the gray-level gradient of the pixels within the window using the Sobel operator; at the same time, a pyramid layering strategy is adopted, with the image resolution of each layer downsampled by 1 / 2, the bottom image capturing large displacement optical flow, and the top image optimizing detail accuracy.

[0024] Furthermore, in the texture feature extraction step, the neighborhood radius of the local binary mode is set to 1, the number of neighboring pixels is set to 8, an 8-bit binary feature code is generated, and a histogram is calculated; the distance of the gray-level co-occurrence matrix is ​​set to 1, and the angle is set to 0°, 45°, 90°, and 135°, respectively, and the contrast, energy, and entropy indices at the four angles are calculated and averaged; both texture features are normalized to the 0-1 interval using min-max normalization, and the normalization range is determined based on the statistical analysis of 1000 frames of scene samples with different densities.

[0025] Furthermore, the ROI partitioning process supports both static predefined and dynamic adjustment modes. In static mode, partitioning is completed by manually annotating the physical region boundary coordinates. In dynamic mode, based on the distribution of foreground features, the K-means clustering algorithm is used to identify areas where people gather. The initial cluster center is determined by the elbow rule, and the iteration termination condition is set to the cluster center offset being less than 2 pixels. The number of clusters is dynamically set to 3-8 based on the scene area, and the boundary coordinates of each ROI are updated in real time at a frequency consistent with the video frame rate.

[0026] Furthermore, in the dynamic calibration step, incremental learning employs a stochastic gradient descent algorithm, updating only a portion of the model's parameters after each input of real-world population data to prevent overfitting. Cross-validation fuses the population statistics from infrared sensors with density estimations from video images, correcting the final density value through a weighted average. The correction formula is as follows:

[0027]

[0028] in This is the corrected density value. To estimate density for video, For infrared headcount counting, For the physical area of ​​the ROI, Weighting of infrared data; Adjust according to ambient light intensity.

[0029] Furthermore, in the heatmap generation step, the normalization process employs piecewise function optimization, with the following piecewise rule:

[0030]

[0031]

[0032]

[0033]

[0034]

[0035]

[0036] To preset the maximum safe density, it is configured according to regional functions and supports manual dynamic adjustment; the color lookup table adopts the Jet color space, which divides the density level into 5 levels, corresponding to the five colors of blue, cyan, yellow, orange and red respectively, and the brightness of each color changes linearly with the density value; the transparency of the semi-transparent color layer is set to 0.6-0.8.

[0037] Compared with existing technologies, the beneficial effects of this invention are:

[0038] In terms of detection accuracy, this invention achieves complementary synergy of multi-dimensional features by fusing optical flow and texture features. Optical flow features can accurately capture the movement state of crowds, distinguishing between stationary and densely moving crowds, as well as orderly queues and chaotic crowds; texture features can effectively characterize the details of personnel distribution in high-density areas, reflecting the "quantity" of the crowd. The fusion of the two avoids the limitations of single features, significantly improving the robustness of density estimation. Even in scenarios with severe personnel occlusion and uneven density distribution, it can maintain stable detection results and effectively reduce estimation bias. At the same time, the dynamic calibration mechanism drives incremental model learning through manual sampling data and combines multi-sensor cross-validation to continuously correct model bias, ensuring that detection accuracy does not decay during long-term operation and further improving the reliability of the results.

[0039] In terms of environmental adaptability, this invention significantly enhances its resistance to interference in complex environments through a series of preprocessing operations and intelligent modeling strategies. Grayscale processing eliminates interference from inconsistent color channel gradients, Gaussian filtering effectively suppresses small-scale noise, and perspective correction unifies the pixel and texture scales at different locations, avoiding the "near-large, far-small" effect. Motion compensation technology corrects global motion misjudgment caused by camera shake, ensuring the accuracy of foreground extraction. Dynamic background modeling based on Gaussian mixture models can adapt to changes in lighting and dynamic backgrounds, accurately distinguishing shadows from real moving objects, reducing interference from invalid features, and enabling the method to work stably under different lighting and background conditions indoors and outdoors, greatly expanding its applicability.

[0040] In terms of real-time performance, this invention employs a macroscopic feature analysis strategy, avoiding the extensive computation required for person-by-person detection. The extraction process for optical flow and texture features is concise and efficient. Furthermore, by dividing the model into Regions of Interest (ROIs) and focusing on key areas for feature analysis and density estimation, unnecessary computational overhead is reduced, significantly improving processing speed and meeting the frame rate requirements of real-time monitoring. This provides a time guarantee for timely warnings and interventions. Simultaneously, the model training process is based on a large number of labeled samples from scenes with varying densities, resulting in high learning efficiency and fast inference speed. After deployment, it can quickly respond to changes in pedestrian flow and dynamically output density results.

[0041] Regarding scenario adaptability, this invention supports two ROI partitioning modes: static predefined and dynamic adjustment. It can set independent safety warning thresholds for different areas based on the physical spatial distribution and functional differences of the actual scenario, making the warning mechanism more scientific and closer to actual safety needs. The dynamic ROI adjustment mode can identify temporary crowd gathering areas in real time, automatically update area boundaries and warning standards, adapt to changes in gathering areas caused by pedestrian flow, and further improve the applicability and flexibility of the method in complex scenarios.

[0042] In terms of visualization, this invention transforms abstract density data into intuitive graphical information through heatmap generation technology. By using variations in color depth or hue, it clearly conveys the density levels of different areas, allowing monitoring personnel to quickly identify high-risk areas and significantly reducing manual identification costs and decision-making time. The presentation method of overlaying a semi-transparent color layer with the original video stream ensures clear visibility of density information without obscuring real-world details, facilitating managers to simultaneously grasp density distribution and on-site conditions, providing an intuitive basis for precise scheduling and intervention. Overall, this invention balances detection accuracy and practical value, meeting the actual needs of public place security monitoring and possessing broad application prospects. Attached Figure Description

[0043] Figure 1 This is a schematic block diagram of a crowd density detection method that integrates optical flow and texture features proposed in this invention.

[0044] Figure 2 A comparison chart of population density estimation errors for different combinations of characteristics;

[0045] Figure 3 This is a graph showing the effect of light intensity on the accuracy of crowd density detection. Detailed Implementation

[0046] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," and "counterclockwise," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0048] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified. Furthermore, the terms "installed," "connected," and "linked" should be interpreted broadly; for example, they may refer to a fixed connection, a detachable connection, or an integral connection; they may refer to a mechanical connection or an electrical connection; they may refer to a direct connection or an indirect connection through an intermediate medium; and they may refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances. The invention will now be described in further detail with reference to the accompanying drawings.

[0049] Reference Figures 1 to 3 A crowd density detection method that integrates optical flow and texture features includes the following steps:

[0050] Video stream acquisition and preprocessing steps: Acquire real-time video stream from camera, first convert color image to grayscale image to reduce channel dimension and retain brightness motion information; use Gaussian filtering to perform noise reduction processing, smooth small-scale textures and random noise, and retain the large-scale structural motion features of the crowd in the image; complete perspective correction through perspective transformation, map the image to a uniform scale top view plane, and eliminate the near-large and far-small effect;

[0051] Motion compensation steps: Image registration technology is used to detect camera shake, and affine transformation is used to perform motion compensation on video frames to stabilize the image and eliminate the interference of global motion on foreground judgment;

[0052] Background modeling and foreground extraction steps: A dynamic background model is constructed based on a Gaussian mixture model. The brightness distribution of each pixel is modeled and the mean and variance are updated in real time to adapt to background changes and illumination fluctuations. The initial foreground area is obtained by background subtraction. The shadow and real moving objects are distinguished by combining color information. A binary foreground mask is generated after thresholding.

[0053] Foreground feature analysis steps: Calculate the ratio of foreground pixels in the foreground mask to the total pixels in the preset area to obtain the foreground area coverage; use the Lucas-Kanade algorithm to calculate the optical flow vector of the foreground area, and calculate the average value of the magnitude of all optical flow vectors as the motion intensity index; analyze the vector direction angle distribution to obtain the motion consistency index; extract the original image area corresponding to the foreground area, extract texture features through local binary mode and gray-level co-occurrence matrix, and calculate texture complexity;

[0054] ROI partitioning steps: Based on the actual physical spatial distribution of the scene, predefine multiple regions of interest with independent safety thresholds, clarify the physical area (accurate to 0.1㎡) and density warning standard of each region, and configure the warning standards of different regions according to functional differences;

[0055] Feature fusion and density regression steps: Normalize the foreground region coverage, motion intensity, motion consistency and texture complexity features and input them into the trained gradient boosting regression model; This model is trained with labeled samples of more than 1,000 frames of different density scenes to establish the mapping relationship between features and actual number of people, and output the initial crowd density of each ROI.

[0056] Dynamic calibration steps: Regularly input real human headcount data from manual sampling to drive incremental learning of the regression model; simultaneously, integrate environmental data collected from multiple sensors to perform cross-validation and correct density estimation bias.

[0057] Heatmap generation and visualization steps: Normalize the density values ​​of each ROI based on the preset maximum safe density, map the normalized density to the corresponding color through a color lookup table, generate a semi-transparent color layer with the same size as the original video frame, and overlay it onto the original video stream to achieve real-time visualization of the crowd density heatmap.

[0058] This invention also includes a precise calculation step for motion consistency, which quantifies the degree of disorder in the distribution of optical flow vector direction angles through information entropy, using the following formula:

[0059]

[0060] in Represents the motion consistency information entropy, with a value range of... The larger the value, the more chaotic the direction of movement; This indicates the number of intervals for the direction angle division, which is evenly divided into 8 intervals from 0 to 360°; Indicates the first The proportion of optical flow vectors within each directional angle interval is obtained by statistically assigning the directional angles of all optical flow vectors in the foreground region. This quantification method can distinguish between orderly queuing crowds and chaotic, crowded crowds, providing more reliable feature support for high-density risk assessment.

[0061] This invention also includes a texture complexity comprehensive calculation step, which integrates the core indicators of local binary patterns and gray-level co-occurrence matrix, and the formula is as follows:

[0062]

[0063] in This represents the overall texture complexity value, ranging from 0 to 1. A larger value indicates a more complex texture. Represents the LBP feature weights. Represents the feature weights of the gray-level co-occurrence matrix. and The values ​​are all between 0.4 and 0.6 and High-density scenes Take 0.6, Take 0.4, for low-density scenes Take 0.4, Set to 0.6; L represents the energy normalization value of the local binary mode histogram; These represent the normalized values ​​of contrast, energy, and entropy of the gray-level co-occurrence matrix, respectively. This comprehensive index can fully characterize the edge texture features of the head and shoulders in densely populated areas, improving the robustness of density estimation.

[0064] In this invention, the preprocessing step uses a Gaussian filter with a kernel size of 3×3 and a standard deviation of 0.8-1.2, which achieves noise suppression without blurring the edges of crowd movement. Perspective correction establishes a mapping relationship between pixel coordinates and physical coordinates by selecting four non-collinear physical reference points in the image (such as ground markings and pillar edges). The mapping error is controlled within one pixel, ensuring that the texture and optical flow features of the crowd at different distances are consistent in scale.

[0065] In this invention, in the background modeling step of the Gaussian mixture model, the number of mixture components for each pixel is set to 3-5, the weight update rate of the components is set to 0.001-0.005, and the initial variance is set to 20-30. The pixel is determined to belong to the background or the foreground by an adaptive threshold. The threshold is dynamically adjusted based on the average pixel brightness. When the average brightness is higher than 200, the threshold is set to 3 times the standard deviation, and when it is lower than 100, the threshold is set to 2 times the standard deviation, thereby improving the accuracy of foreground extraction in complex backgrounds.

[0066] In this invention, the optical flow vector calculation adopts an improved Lucas-Kanade algorithm, selects a 3×3 window for local feature matching, calculates the gray-level gradient of pixels within the window using the Sobel operator, sets the iteration convergence threshold to 0.01, and sets the maximum number of iterations to 10. At the same time, a pyramid layering strategy is adopted, with each layer of image resolution downsampled by 1 / 2. The bottom layer image captures large displacement optical flow, and the top layer image optimizes detail accuracy, thereby improving the calculation accuracy of large displacement optical flow vectors and ensuring the reliability of motion intensity and consistency indicators.

[0067] In this invention, during the texture feature extraction step, the neighborhood radius of the local binary mode is set to 1, the number of neighboring pixels is set to 8, an 8-bit binary feature code is generated, and a histogram is calculated. The distance of the gray-level co-occurrence matrix is ​​set to 1, and the angles are set to 0°, 45°, 90°, and 135°. The contrast, energy, and entropy indices at the four angles are calculated and averaged. Both texture features are normalized to the 0-1 interval using min-max normalization. The normalization range is determined based on 1000 frames of scene samples with different densities, enhancing the ability of texture features to distinguish people of different densities.

[0068] In this invention, the ROI partitioning step supports two modes: static predefinition and dynamic adjustment. In the static mode, partitioning is completed by manually annotating the physical region boundary coordinates. In the dynamic mode, based on the foreground feature distribution, the K-means clustering algorithm is used to identify crowd gathering areas. The initial cluster center is determined by the elbow rule, and the iteration termination condition is set to the cluster center offset being less than 2 pixels. The number of clusters is dynamically set to 3-8 according to the scene area. The boundary coordinates of each ROI are updated in real time at the same frequency as the video frame rate to adapt to changes in gathering areas caused by crowd movement.

[0069] In this invention, the dynamic calibration step employs a stochastic gradient descent algorithm for incremental learning, with a learning rate set to 0.001-0.003. Each time real population data is input, only a portion of the model's parameters are updated to prevent overfitting. Cross-validation fuses the population statistics from infrared sensors with the density estimation results from video images, and the final density value is corrected using a weighted average. The correction formula is as follows:

[0070]

[0071] in This is the corrected density value. To estimate density for video, For infrared headcount counting, For the physical area of ​​the ROI, Weighting of infrared data; Adjusted according to ambient light intensity; when the light intensity (calculated from the average image brightness) is below 500 lux. Set to 0.3-0.4, when it exceeds 500 lux. Set the weight to 0.1-0.2 and update it every 10 frames to improve the density estimation accuracy under different lighting conditions.

[0072] In this invention, the normalization process in the heatmap generation step employs piecewise function optimization, with the piecewise rule being:

[0073]

[0074]

[0075]

[0076]

[0077]

[0078]

[0079] To preset the maximum safe density, it is configured according to the function of the area (5 people / ㎡ in the square, 2 people / ㎡ at the stairwell) and supports manual dynamic adjustment; the color lookup table adopts the Jet color space, which divides the density level into 5 levels, corresponding to the five colors of blue, cyan, yellow, orange and red respectively. The brightness of each color changes linearly with the density value; the transparency of the semi-transparent color layer is set to 0.6-0.8, which ensures that the density information is clearly visible without obscuring the details of crowd movement in the original video, making it easy for monitoring personnel to obtain density distribution and on-site scene information at the same time.

[0080] The following two examples further illustrate specific embodiments of the present invention:

[0081] Example 1: Crowd density detection in the atrium of a large indoor shopping mall

[0082] This embodiment is applied to an indoor atrium scenario in a large shopping mall, covering an area of ​​approximately 800 square meters. It includes two escalators, three main passageways, two rest areas, and one atrium display stand. The peak daily foot traffic reaches 5,000 people, with low peak hours in the morning and evening and a midday peak. Different areas exhibit significant functional differences, with escalator entrances prone to congestion and rest areas experiencing dispersed foot traffic. The system needs to address the issues of accurate ROI segmentation under dense indoor foot traffic, robust density estimation, and real-time visualization of heatmaps. After deployment, the density estimation error must be ≤5%, the ROI dynamic adjustment response time ≤100ms, and the heatmap update frequency must match the video frame rate at 25fps.

[0083] I. System Deployment and Implementation of Core Steps

[0084] 1. Video stream acquisition and preprocessing

[0085] Four high-definition network cameras are deployed 6 meters above the ground around the atrium, with the lenses facing the center of the atrium, covering the escalators, passageways, rest areas and exhibition booths. They capture real-time video streams with a resolution of 1920×1080 and a frame rate of 25fps, and transmit them to the back-end processing server via network cable with a transmission latency of ≤8ms.

[0086] Preprocessing is performed according to the following procedure:

[0087] Grayscale conversion: The image processing library is used to convert each frame of the color image to an 8-bit grayscale image. The calculation formula is as follows: , These are the pixel values ​​for the color channels, respectively, to eliminate Channel gradient inconsistency interferes with optical flow calculation, while preserving brightness motion information.

[0088] Gaussian filtering: A 3×3 Gaussian filter with a standard deviation of 1.0 is used to perform pixel-by-pixel convolution processing on the grayscale image. The filter kernel function is... , The relative coordinates of pixels within the kernel. =1.0, smoothing out small-scale noise such as clothing textures, while preserving the overall structural features of the crowd's movement.

[0089] Perspective Correction: Four non-collinear tile corner points on the mall floor were selected as physical reference points, with coordinates of 1.2m, 0.8m, 1.2m, 15.6m, 22.4m, 0.8m, 22.4m, 15.6m, corresponding to image pixel coordinates of 320, 240, 320, 960, 1600, 240, 1600, 960. The image was mapped to the top plane using a perspective transformation matrix M, with the following mapping formula: , These are the original pixel coordinates. For the corrected coordinates, Using homogeneous coordinates, the mapping error is calculated to be 0.8 pixels ≤ 1 pixel, thus eliminating the optical flow scale deviation caused by near-large and far-small near the escalator.

[0090] 2. Motion Compensation and Background Modeling

[0091] Motion compensation: The mall cameras are fixed but subject to slight vibrations, such as building vibrations caused by escalator operation. The SIFT feature point matching algorithm is used to extract key feature points every 5 frames, with ≥50 matching points extracted each time. The affine transformation matrix is ​​then calculated. , Original coordinates To compensate for the coordinates, a transformation is performed on the current frame. The measured inter-frame offset after compensation is ≤1 pixel, eliminating the interference of global motion on foreground judgment.

[0092] Background Modeling: A dynamic background is constructed based on a Gaussian mixture model, with three mixture components per pixel, a weight update rate of 0.003, and an initial variance of 25. The foreground judgment threshold is dynamically adjusted based on the mean pixel brightness. The mean brightness in the shopping mall is stabilized between 150-180, and the threshold is set to 2.5 times the standard deviation. The initial foreground area obtained by subtracting the background is used to distinguish shadows using the HSV color space. The hue deviation of the shadow area from the background is ≤10°, and the brightness is 30%-50% lower than the background. After removing the shadows, Otsu thresholding is applied to generate a binarized foreground mask, with 255 pixels for the foreground and 0 pixels for the background.

[0093] 3. Foreground Feature Analysis

[0094] Foreground area coverage: Select the main passage ROI in the atrium with a physical area of ​​20㎡, corresponding to an image pixel of 1200×800=960000 pixels. The number of 255 pixels in the foreground mask is 288000, and the coverage rate is 288000 / 960000=30%, reflecting the degree of occupancy of people in the passage.

[0095] Optical flow vector calculation: An improved Lucas-Kanade algorithm is used, with 3×3 window matching and the Sobel operator to calculate the gray-level gradient. Directional gradient Directional gradient The iterative convergence threshold was 0.01, with a maximum of 10 iterations. A three-layer pyramid structure was used, with each layer downsampled at half resolution, to capture the large-displacement optical flow of pedestrians at escalator entrances. 1000 optical flow vectors were statistically analyzed, with an average magnitude of 1.2 pixels per frame, serving as a motion intensity indicator. The directional angles were distributed across eight intervals, from 0-45° to 315-360°, with the percentage of each interval shown. Substituting into the motion consistency information entropy formula Calculated

[0096]

[0097] This indicates that pedestrian movement is quite chaotic during the midday peak, necessitating an upgrade in the congestion prevention warning level.

[0098] Texture complexity calculation: Extract the original grayscale image corresponding to the foreground region, use Local Binary Pattern (LBP) neighborhood radius 1 and number of pixels 8, generate an 8-bit binary code, calculate the histogram, and normalize the energy value. The contrast ratio is calculated based on the gray-level co-occurrence matrix distance of 1 and angles of 0° / 45° / 90° / 135°. ,energy ,entropy The mean is The current scenario is a high-density scene, so we take... Substitute into the texture complexity formula ,have to The text reflects that the head and shoulder textures of people in the passage are dense and of high density.

[0099] 4. ROI segmentation and feature fusion

[0100] ROI Division: Static predefined 3 ROIs: escalator entrance physical area 8㎡, safety threshold 1.5 people / ㎡; main passage 20㎡, safety threshold 2 people / ㎡; rest area 15㎡, safety threshold 3 people / ㎡; Dynamic mode is based on K-means clustering, the initial cluster center is determined by the elbow rule, the error inflection point corresponds to k=5, and the iteration termination condition is that the cluster center offset <2 pixels. During the midday peak, the temporary gathering area around the booth is automatically identified, named ROI5, with an area of ​​12㎡. The boundary coordinates are updated once per frame, synchronized with the video frame rate.

[0101] Feature fusion: Foreground coverage of 30% was normalized to 0.3, motion intensity of 1.2 pixels / frame was normalized to 0.6, motion consistency H=2.58 was normalized to 0.86, and texture complexity C=0.62 was normalized to 0.62. These were input into a gradient boosting regression model, which was trained with 1200 frames of labeled samples. Each ROI was manually counted in each frame to establish a mapping relationship between features and the actual number of people. The initial density of the main channel was 1.9 people / ㎡.

[0102] 5. Dynamic calibration and heat map generation

[0103] Dynamic calibration: Every 2 hours, a manual check of the main passageway revealed 38 people, a density of 38 / 20 = 1.9 people / ㎡. This data was input into the model for incremental learning using a stochastic gradient descent algorithm with a learning rate of 0.002. Data from the escalator entrance infrared sensors was also integrated, showing 16 people, a density of 16 / 8 = 2.0 people / ㎡. The current light intensity was 650 lux. =0.15, substitute into the correction formula ,have to Persons / m², error ≤0.5%.

[0104] Heatmap generation: A piecewise normalization function is used, with the main channel... The current density is 1.915 people / m², 0.3×2.0=0.6<1.915≤2.0, normalized value=1.915 / 2.0=0.957; mapped to orange using Jet color lookup table, density level 4, generating a 1920×1080 semi-transparent layer with transparency 0.7, superimposed on the original video stream, allowing monitoring personnel to intuitively identify that the main channel is approaching the congestion threshold.

[0105] II. Data Representation and Interpretation

[0106] Table 1: Comparison of Crowd Density Detection Results in Indoor Shopping Mall Atriums (Midday Peak Hour)

[0107]

[0108] Table 1 shows that traditional methods relying solely on texture features do not consider motion information and dynamic ROIs. The estimation error at escalator entrances due to pedestrian movement reaches 8.2%, and the method is unsuitable for temporary gathering areas. This invention, through optical flow-texture fusion features, reduces the error to 2.1%, and achieves a dynamic ROI adjustment response time of 92ms, meeting real-time requirements. The overall density estimation accuracy is improved by 13.4%, and the heatmap risk identification time is reduced to 0.3s, allowing monitoring personnel to quickly locate high-risk areas. The data demonstrates that this method can accurately adapt to the density features of different ROIs in complex indoor functional zoning scenarios, improving detection robustness and visualization efficiency.

[0109] Example 2: Crowd density detection in outdoor city squares

[0110] This embodiment is applied to an outdoor urban square with an area of ​​1500㎡. The area is surrounded by trees and streetlights, and the light intensity varies significantly from morning to evening: 300 lux in the morning, 1200 lux at noon, and 400 lux in the evening. The swaying of the trees in the wind creates a dynamic background. The peak pedestrian flow is concentrated in the morning and evening, with people exercising in the morning and relaxing in the evening. It is necessary to solve the problems of light interference, misjudgment of dynamic background, and ROI management in large scenes. The system needs to achieve a density estimation error of ≤6% and a background dynamic interference rate of ≤3% under different lighting conditions.

[0111] I. System Deployment and Implementation of Core Steps

[0112] 1. Video stream acquisition and preprocessing

[0113] High-definition cameras are deployed on four lampposts around the square at a height of 8m. The lenses are equipped with rain covers and the camera angle is 30° downwards. The cameras cover the center of the square, the entrance, the circular walkway and the surrounding green areas. The video stream has a resolution of 1920×1080 and a frame rate of 25fps. It is transmitted to the processing server via 5G with a latency of ≤9ms.

[0114] Preprocessing flow:

[0115] Grayscale processing: Same as in Example 1, eliminate color channel interference, and retain brightness information to adapt to changes in lighting.

[0116] Gaussian filtering: 3×3 kernel, standard deviation 1.2, more outdoor noise, gradient fluctuation of the filtered image ≤5%, improving optical flow stability.

[0117] Perspective correction: Four reference points were selected at the edge of the square fountain, with physical coordinates of 5m, 5m, 5m, 45m, 45m, 5m, 45m, 45m, corresponding to image pixels of 280, 220, 280, 1000, 1640, 220, 1640, 1000. The mapping error after perspective transformation was 0.9 pixels, and the pixel scale of trees and crowds was unified.

[0118] 2. Motion Compensation and Background Modeling

[0119] Motion compensation: Outdoor wind causes slight camera shake, with a maximum offset of 2 pixels / frame. SIFT feature points are extracted every 3 frames, with ≥60 matching points. The Affine transformation matrix is ​​calculated for compensation. After compensation, the inter-frame alignment error is ≤0.5 pixels, avoiding background misjudgment caused by tree swaying and camera shake.

[0120] Background modeling: A Gaussian mixture model with 4 mixture components per pixel is used to adapt to dynamic backgrounds of swaying trees. The weight update rate is 0.005, and the initial variance is 28. The foreground threshold is adjusted according to the lighting conditions: a mean brightness of 80 in the morning corresponds to a threshold of 2 standard deviations; 220 at noon corresponds to a threshold of 3 standard deviations; and 400 in the evening corresponds to a threshold of 2.8 standard deviations. Trees and people are distinguished by combining color and motion. Trees have a movement cycle of 1-2 seconds and a fixed hue, while people move randomly and have diverse hues. After background subtraction, a foreground mask is generated, reducing the dynamic background interference rate to 2.5%.

[0121] 3. Foreground Feature Analysis

[0122] Foreground area coverage: The physical area of ​​the ROI in the center of the square is 100㎡, and the image pixel count is 1500×1000=1.5×10 6 Pixels, foreground pixels 4.5×10 5 Coverage rate = 4.5 × 10 5 / 1.5×10 6 =30%.

[0123] Optical flow vector calculation: The improved Lucas-Kanade algorithm is the same as in Example 1. 1500 optical flow vectors are counted, with an average magnitude of 1.5 pixels / frame as the motion intensity, and the proportion of the 8-degree angle interval is used. Substituting into the motion consistency formula, we get This indicates that pedestrian movement is more chaotic in the evening.

[0124] Texture complexity calculation: LBP energy normalization value Mean of gray-level co-occurrence matrix Currently, this is a medium-to-high density scenario. =0.55、 =0.45, substituting into the formula, we get This reflects the dense flow of people in the central area.

[0125] 4. ROI segmentation and feature fusion

[0126] ROI Division: There are 3 static ROIs, with the central square D being the largest. max 5 people / m², 3 people / m² at the entrance, and 2 people / m² on the walkway; dynamic K-means clustering, k=3, fewer clusters in the evening, iteration termination offset <2 pixels, ROI boundaries updated every frame.

[0127] Feature fusion: After feature normalization, the input is used to the gradient boosting model, and the output is an initial density of 4.8 people / m² at the center of the square.

[0128] 5. Dynamic calibration and heat map generation

[0129] Dynamic calibration: Evening light intensity 380 lux, take =0.35, the infrared sensor counts 470 people in the central area, the density is 470 / 100 = 4.7 people / m², substituting into the correction formula, we get... The accuracy is 0.7% per person per square meter; manual sampling is conducted every hour; incremental learning uses the stochastic gradient descent algorithm with a learning rate of 0.003; and the model adapts to changes in illumination.

[0130] Heat map generation: Plaza center The normalized value corresponding to a density of 4.765 is 4.765 / 5 = 0.953, which is mapped to orange; the normalized value corresponding to an inlet density of 2.8 is 2.8 / 3≈0.933, which is mapped to yellow with a transparency of 0.7. After superposition, the risks in different areas can be clearly distinguished.

[0131] II. Data Representation and Interpretation

[0132] Table 2: Comparison of crowd density detection results in outdoor city squares (at different lighting times)

[0133]

[0134] Table 2 shows that traditional fixed-background methods cannot adapt to changes in outdoor lighting and dynamic backgrounds. In low light conditions in the morning, the error reaches 12.5%, and the interference rate due to tree movement is 15.2%. This invention, through motion compensation and adaptive background modeling, reduces the error to 3.8% in low light, 2.5% in strong light, and only 2.5% in dynamic background interference. After dynamic calibration, the error further decreases to 0.7%, indicating that the method maintains high accuracy even in complex outdoor environments. The data validates the method's resistance to interference from changes in lighting and dynamic backgrounds, meeting the needs of large-scale outdoor crowd density detection.

[0135] Reference Figure 2This image visually demonstrates the advantages of optical flow-texture fusion features: single texture features are greatly affected by static scenes, with errors generally exceeding 8%; while single optical flow features can capture motion information, the error increases to 7.9%-9.2% when people are stationary; this invention, by fusing the two features, complements their weaknesses, controlling the error to within 3.5% in all scenes, maintaining low error even in complex scenes such as subway stations and stadiums. Low error ensures accurate density estimation, providing reliable data support for subsequent risk warnings, avoiding misjudgments or missed detections due to the limitations of single features, and meeting the needs of multi-scene crowd detection.

[0136] Reference Figure 3 This figure verifies the anti-light interference capability of the present invention: traditional methods are significantly affected by light conditions, with an accuracy of only 85.3% in low light and dropping to 84.5% in strong light due to light spots and shadows, failing to stably adapt to changes in outdoor lighting. The present invention, through adaptive background modeling and multi-feature fusion, maintains a stable accuracy of 97.2%-98.5%, with a fluctuation of only 1.3%. This anti-light interference characteristic makes it highly applicable to outdoor scenes, avoiding fluctuations in detection accuracy caused by differences in day and night lighting, and ensuring the reliability of all-weather density detection.

[0137] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for detecting crowd density by fusing optical flow and texture features, characterized in that, Includes the following steps: Video stream acquisition and preprocessing steps: Acquire real-time video stream from camera, convert color image to grayscale image; use Gaussian filter for noise reduction, perform perspective transformation for correction, and map image to a uniform scale top view plane; Motion compensation steps: Image registration technology is used to detect camera shake, and affine transformation is used to perform motion compensation on video frames to stabilize the image and eliminate the interference of global motion on foreground judgment; Background modeling and foreground extraction steps: Construct a dynamic background model based on Gaussian mixture model, model the brightness distribution of each pixel and update the mean and variance in real time; The initial foreground region is obtained by background subtraction, and the shadows and real moving objects are distinguished by combining color information. A binary foreground mask is generated by thresholding. Foreground feature analysis steps: Calculate the foreground region coverage, obtain the optical flow vector using the Lucas-Kanade algorithm, statistically analyze the motion intensity and motion consistency index, extract texture features and calculate texture complexity; ROI partitioning steps: Based on the actual physical spatial distribution of the scene, predefine regions of interest with independent safety thresholds, clarify the physical area and density warning standards for each region, and configure the warning standards for different regions according to functional differences; Feature fusion and density regression steps: Normalize the foreground region coverage, motion intensity, motion consistency and texture complexity features, input them into the trained gradient boosting regression model, establish the mapping relationship between features and actual number of people, and output the initial population density of each ROI; Dynamic calibration steps: Regularly input real human headcount data from manual sampling to drive incremental learning of the regression model; simultaneously, integrate environmental data collected from multiple sensors to perform cross-validation and correct density estimation bias. Heatmap generation and visualization steps: Normalize the density values ​​of each ROI, map the corresponding colors through a color lookup table, generate a semi-transparent color layer, and overlay it onto the original video stream to achieve real-time visualization of the crowd density heatmap. The precise calculation steps for motion consistency involve quantifying the degree of disorder in the distribution of optical flow vector direction angles using information entropy, as shown in the formula: in Represents the motion consistency information entropy; Indicates the number of intervals divided by the direction angle; Indicates the first The proportion of optical flow vectors within each directional angle interval is obtained by statistically assigning the directional angles of all optical flow vectors in the foreground region. The comprehensive calculation steps for texture complexity integrate the core metrics of local binary mode and gray-level co-occurrence matrix, and the formula is as follows: in This represents the overall texture complexity value; Represents the LBP feature weights. Represents the feature weights of the gray-level co-occurrence matrix. and The values ​​are all between 0.4 and 0.6 and High-density scenes Take 0.6, Take 0.4, for low-density scenes Take 0.4, Take 0.6; This represents the energy normalization value of the local binary pattern histogram; These represent the normalized values ​​of contrast, energy, and entropy of the gray-level co-occurrence matrix, respectively.

2. The crowd density detection method fusing optical flow and texture features according to claim 1, characterized in that, In the preprocessing step, the kernel size of the Gaussian filter is set to 3×3, and the standard deviation is 0.8-1.

2. Perspective correction establishes a mapping relationship between pixel coordinates and physical coordinates by selecting four non-collinear physical reference points in the image, and the mapping error is controlled within 1 pixel.

3. The crowd density detection method fusing optical flow and texture features according to claim 1, characterized in that, In the background modeling step of Gaussian mixture model, the number of mixture components for each pixel is set to 3-5, the weight update rate of the components is set to 0.001-0.005, and the initial variance is set to 20-30. The pixel is determined to belong to the background or the foreground by an adaptive threshold. The threshold is dynamically adjusted based on the average pixel brightness. When the average brightness is higher than 200, the threshold is set to 3 times the standard deviation, and when it is lower than 100, the threshold is set to 2 times the standard deviation.

4. The crowd density detection method fusing optical flow and texture features according to claim 1, characterized in that, The optical flow vector calculation adopts an improved Lucas-Kanade algorithm, selects a 3×3 window to perform local feature matching, and calculates the gray-level gradient of the pixels within the window using the Sobel operator; at the same time, a pyramid layering strategy is adopted, with the image resolution of each layer downsampled by 1 / 2, the bottom image capturing large displacement optical flow, and the top image optimizing detail accuracy.

5. The crowd density detection method fusing optical flow and texture features according to claim 1, characterized in that, In the texture feature extraction step, the neighborhood radius of the local binary mode is set to 1, the number of neighboring pixels is set to 8, an 8-bit binary feature code is generated, and a histogram is calculated; the distance of the gray-level co-occurrence matrix is ​​set to 1, and the angle is set to 0°, 45°, 90°, and 135°, respectively, and the contrast, energy, and entropy indices at the four angles are calculated and averaged; both texture features are normalized to the 0-1 interval using min-max normalization, and the normalization range is determined based on the statistical analysis of 1000 frames of scene samples with different densities.

6. The crowd density detection method fusing optical flow and texture features according to claim 1, characterized in that, The ROI partitioning process supports two modes: static predefined and dynamic adjustment. In static mode, the partitioning is completed by manually annotating the physical region boundary coordinates. In dynamic mode, based on the distribution of foreground features, the K-means clustering algorithm is used to identify areas where people gather. The initial cluster center is determined by the elbow rule, and the iteration termination condition is set to the cluster center offset being less than 2 pixels. The number of clusters is dynamically set to 3-8 according to the scene area, and the boundary coordinates of each ROI are updated in real time at the same frequency as the video frame rate.

7. The crowd density detection method fusing optical flow and texture features according to claim 1, characterized in that, In the dynamic calibration step, incremental learning employs a stochastic gradient descent algorithm, updating only a portion of the model's parameters after each input of real-world population data to prevent overfitting. Cross-validation fuses population statistics from infrared sensors with density estimations from video images, correcting the final density value through a weighted average. The correction formula is as follows: in This is the corrected density value. To estimate density for video, For infrared headcount counting, For the physical area of ​​the ROI, Weighting of infrared data; Adjust according to ambient light intensity.

8. The crowd density detection method fusing optical flow and texture features according to claim 1, characterized in that, In the heatmap generation step, the normalization process employs piecewise function optimization, with the following piecewise rule: The maximum safe density is preset, configured according to regional functions, and can be dynamically adjusted manually. The color lookup table uses the Jet color space, dividing the density levels into 5 levels, corresponding to the five colors: blue, cyan, yellow, orange, and red. The brightness of each color changes linearly with the density value; the transparency of the semi-transparent color layer is set to 0.6-0.8.

Images