System and method for monitoring crowd density in multi-scale complex scene

Abstract

The invention discloses a crowd density monitoring system and method in a multi-scale complex scene, and relates to the technical field of data monitoring. Comprising the following steps of: 1, deploying multiple cameras and configuring and calibrating the multiple cameras; 2, performing three-dimensional coordinate reconstruction of a scene by utilizing images acquired by the multiple cameras; 3, performing feature extraction and data fusion; 4, performing discrete element representation and simulation; the crowd distribution condition of the whole monitoring area is displayed, and an efficient and reliable solution is provided for crowd management monitoring in a complex scene.

Description

Technical Field

[0001] The invention discloses a crowd density monitoring system and method in a multi-scale complex scene, and relates to the technical field of data monitoring. Background Art

[0002] In the field of urban governance and public safety, the management of crowd density is particularly critical, especially in large-scale public events and major security scenarios. High-density crowds increase the complexity of security management and are directly related to the life safety of personnel and the stability of order. Therefore, real-time monitoring of on-site personnel density has become an important measure to ensure public safety. By accurately and in real time estimating crowd density, potential safety hazards can be discovered in a timely manner, such as dangerous situations such as trampling caused by overcrowding, so that preventive or evacuation measures can be taken to effectively prevent and control safety risks. In addition, the acquisition of real-time crowd density data also supports more scientific crowd management decisions and resource optimization, enhances the city's emergency response capabilities and scene safety assurance, and has significant practical value and social significance. However, real-world scenarios such as urban governance often have problems such as target occlusion and small target recognition. It is difficult for existing technologies to deal with these problems, resulting in low crowd density estimation accuracy and counting jitter, which affects crowd density monitoring in multi-scale complex scenarios. Summary of the invention

[0003] In response to the problems of the prior art, the present invention provides a crowd density monitoring system and method in multi-scale complex scenes. Based on computer vision technology, discrete element numerical calculation is integrated to effectively overcome the difficulties of occlusion and small target recognition, enhance the understanding and prediction capabilities of complex crowd dynamics, and provide an efficient and reliable solution for crowd management and monitoring in complex scenes.

[0004] The specific scheme proposed by the present invention is:

[0005] The present invention provides a method for monitoring crowd density in a multi-scale complex scene, comprising:

[0006] Step 1: Deploy multiple cameras and perform configuration and calibration.

[0007] Step 2: Reconstruct the 3D coordinates of the scene using images collected by multiple cameras:

[0008] Step 21: Use multiple cameras to simultaneously capture images of the same scene monitoring area from different angles.

[0009] Step 22: Detect and match feature points in the image data.

[0010] Step 23: Estimate disparity based on matched feature points,

[0011] Step 24: Based on the parallax and the known positions and geometric relationships of the cameras, the three-dimensional coordinates of each point in the scene are calculated using a triangulation method, the three-dimensional coordinates are formed into a three-dimensional point set, the three-dimensional point set is integrated into a coherent three-dimensional model, and the spatial structure of the entire monitoring area is represented by the three-dimensional model;

[0012] Step 3: Feature extraction and data fusion:

[0013] Step 31: When performing feature extraction, a feature extraction algorithm is selected to extract features from the collected images according to the scale of the target in different cameras.

[0014] Step 32: Adjust the image size according to the scale ratio so that the images collected by different cameras are comparable when performing feature matching;

[0015] Step 33: When performing data fusion, images collected from different cameras are matched with multi-view features, and then the matched features are integrated;

[0016] Step 4: Perform discrete element characterization and simulation:

[0017] Step 41: Characterize discrete elements: Based on the integrated features, simulate each detected individual as a discrete element, and assign an initial attribute value to each discrete element.

[0018] Step 42: Perform neighbor unit detection on the discrete element and set the force model between neighbor units of the discrete element.

[0019] Step 43: Update the position and velocity of each discrete element based on the force model and the current state;

[0020] Step 5: Estimating the crowd density:

[0021] Discrete metadata is used to calculate the crowd density in the monitoring area. The formula is as follows:

[0022]

[0023] ρ(x,y,z) represents the density value at position (x,y,z), V is the volume unit considered, and δ is the standard function. i It is 1 when it is within V, otherwise it is 0.

[0024] Use crowd density to create a visual chart that shows the distribution of people in the entire monitoring area.

[0025] Furthermore, in step 1 of the method for monitoring crowd density in a multi-scale complex scene, multiple cameras are deployed, specifically including:

[0026] Step 11: Select the camera type and specifications:

[0027] According to the needs of the application scenario, consider the camera's resolution, photosensitivity, and viewing angle, and choose a camera that meets the needs of the scenario.

[0028] Step 12: Determine the camera installation location:

[0029] Scene analysis: Analyze crowded areas based on the geographical and environmental characteristics of the monitored area.

[0030] View planning: Select the position of the camera's view to cover key areas, and cross-cover the cameras to reduce blind spots.

[0031] Step 13: Physical Installation of the Camera:

[0032] According to the preliminary planning, determine the specific installation height and tilt angle of the camera to maximize the field of view coverage efficiency.

[0033] Mount all cameras on a sturdy bracket to prevent vibration from affecting image quality.

[0034] Furthermore, the configuration and calibration of multiple cameras in step 1 of the method for monitoring crowd density in a multi-scale complex scene specifically includes:

[0035] Step 14: Camera Calibration:

[0036] Internal parameter calibration: Calibrate the internal parameters of the camera, including setting the focal length and aperture, adjusting the image clarity and exposure level,

[0037] External parameter calibration: adjust the relative position and angle between cameras so that the fields of view of multiple cameras can overlap effectively.

[0038] Step 15: Synchronize the camera time: Set the time synchronization between cameras to ensure the consistency of all data collection in time.

[0039] Step 16: Test and verify the camera: Perform a preliminary camera functionality test to check if the camera's image quality and coverage are as expected.

[0040] Integration test: Integration test is performed at the system level to verify whether the data fusion and collaborative work of multiple cameras are normal.

[0041] Furthermore, in step 22 of the method for monitoring crowd density in a multi-scale complex scene, feature point detection and matching in the image data are performed, including:

[0042] Identify the correspondence between the same feature points in different images. Let I1 and I2 be images acquired from two different perspectives, where P1 and P2 are the matching feature points in the images.

[0043] The feature point matching function is defined as:

[0044]

[0045] Where descriptor(p) represents the feature descriptor of point p, ‖·‖ represents the Euclidean distance;

[0046] In step 23, the disparity is estimated based on the matched feature points, including:

[0047] The parallax formula is:

[0048] d = x1-x2

[0049] Calculate the disparity, x1 and x2 are the horizontal coordinates of the feature points in the two images respectively.

[0050] Furthermore, the method for monitoring crowd density in a multi-scale complex scene also includes step 25: model optimization and verification: refining the initially constructed three-dimensional model, increasing the level of detail of the triangular model, and improving the accuracy and practicality of spatial reconstruction; verifying the three-dimensional model through field measurements and additional three-dimensional scanning data to ensure the accuracy and reliability of the model;

[0051] Step 26: Integration and Application: Integrate the constructed 3D model into the monitoring system to enhance the spatial reference for crowd density estimation and behavior analysis. Regularly update the 3D model based on new monitoring data to maintain the timeliness and adaptability of the model.

[0052] Further, in step 32 of the method for monitoring crowd density in a multi-scale complex scene, according to the scale ratio formula:

[0053]

[0054] Calculate the scale ratio S,

[0055] f1, f2 are the focal lengths of the two cameras, D1, D2 are the distances from the target to the camera,

[0056] Using the formula:

[0057] I′=resize(I,×S)

[0058] The image size is adjusted using the scale ratio S so that images collected by different cameras are comparable during feature matching.

[0059] Further, step 33 of the method for monitoring crowd density in a multi-scale complex scene includes:

[0060] Perform multi-view feature matching: Use the geometric correction method to ensure the geometric alignment of images from different viewpoints, and use the basic matrix estimation formula:

[0061]

[0062] Describes the epipolar geometric relationship between two perspectives, where p and p' are the homogeneous coordinates of the feature points in the two images, F is the basic matrix,

[0063] To optimize the match:

[0064]

[0065] Where desc(p) is the descriptor of point p, and the matching reliability is enhanced by optimizing the matching score.

[0066] Perform feature fusion: For the intersection area of ​​dual cameras, a fusion method based on geometric correction is adopted, combining multi-view geometric correction and fusion image features; for the single-camera perspective area: a multi-scale fusion network is used to integrate features at different scales.

[0067] Furthermore, in step 41 of the method for monitoring crowd density in a multi-scale complex scene, an initial attribute value is assigned to each discrete element based on the features extracted in step 3, and the formula is as follows:

[0068] E i ={x i ,y i , v i ,θ i}

[0069] E i is the ith discrete element, x i ,y i Represents the position coordinates, v i represents the speed, θ i is the direction of movement,

[0070] In step 42, the force model between neighboring elements of the discrete element is set, including:

[0071] Define the force model using the formula:

[0072]

[0073] The force model is dynamically adjusted according to the distance between neighboring units, where F ij is the force between neighboring units i and j, k is the strength coefficient of the force, σ is the influence range parameter,

[0074] Step 43 includes:

[0075] Using the formula:

[0076]

[0077] Update the position and velocity of each discrete element based on the force model and the current state, where Δt is the time step, which is used to indicate the update state from time t to t+1.

[0078] The present invention also provides a crowd density monitoring system in a multi-scale complex scene, including a multi-camera configuration and calibration module, a three-dimensional space reconstruction module, a feature extraction and data fusion module, a discrete element characterization and simulation module, and a crowd density field estimation module.

[0079] The multi-camera configuration and calibration module deploys multiple cameras and performs multi-camera configuration and calibration.

[0080] The 3D space reconstruction module uses images collected by multiple cameras to reconstruct the 3D coordinates of the scene:

[0081] Step 21: Use multiple cameras to simultaneously capture images of the same scene monitoring area from different angles.

[0082] Step 22: Detect and match feature points in the image data.

[0083] Step 23: Estimate disparity based on matched feature points,

[0084] Step 24: Based on the parallax and the known positions and geometric relationships of the cameras, the three-dimensional coordinates of each point in the scene are calculated using a triangulation method, the three-dimensional coordinates are formed into a three-dimensional point set, the three-dimensional point set is integrated into a coherent three-dimensional model, and the spatial structure of the entire monitoring area is represented by the three-dimensional model;

[0085] The feature extraction and data fusion module performs feature extraction and data fusion:

[0086] Step 31: When performing feature extraction, a feature extraction algorithm is selected to extract features from the collected images according to the scale of the target in different cameras.

[0087] Step 32: Adjust the image size according to the scale ratio so that the images collected by different cameras are comparable when performing feature matching;

[0088] Step 33: When performing data fusion, images collected from different cameras are matched with multi-view features, and then the matched features are integrated;

[0089] Discrete element characterization and simulation module performs discrete element characterization and simulation:

[0090] Step 41: Characterize discrete elements: Based on the integrated features, simulate each detected individual as a discrete element, and assign an initial attribute value to each discrete element.

[0091] Step 42: Perform neighbor unit detection on the discrete element and set the force model between neighbor units of the discrete element.

[0092] Step 43: Update the position and velocity of each discrete element based on the force model and the current state;

[0093] The crowd density field estimation module performs crowd density estimation:

[0094] Discrete metadata is used to calculate the crowd density in the monitoring area. The formula is as follows:

[0095]

[0096] ρ(x,y,z) represents the density value at position (x,y,z), V is the volume unit considered, and δ is the standard function. i It is 1 when it is within V, otherwise it is 0.

[0097] Use crowd density to create a visual chart that shows the distribution of people in the entire monitoring area.

[0098] Furthermore, the crowd density monitoring system in a multi-scale complex scene also includes a system performance evaluation module for testing and evaluating the system performance.

[0099] The benefits of the present invention are:

[0100] 1. Improved small target recognition accuracy: Through cross-scale correction of multi-camera cross-target detection images, the recognition capability of small targets in complex scenes is improved, ensuring the accuracy and reliability of the recognition results.

[0101] 2. Reduced the impact of crowd occlusion: The impact of crowd occlusion is reduced by multi-camera data fusion, and spatial reconstruction of the monitoring scene and crowd discrete element modeling are performed at the same time, realizing full-domain personnel monitoring and improving the system's coverage and monitoring efficiency.

[0102] 3. Avoid crowd counting jitter: It overcomes the physical limitations of single computer vision technology, combines visual recognition with discrete element model numerical calculation, performs scientific calculation correction based on crowd identification, and provides more stable and accurate crowd count data.

[0103] 4. Effectively monitor the crowd density field: By accurately establishing the relationship between the monitoring scene and the crowd distribution, a crowd density field model is actually created, providing an important decision-making support tool for subsequent social governance and emergency response. BRIEF DESCRIPTION OF THE DRAWINGS

[0104] Figure 1 It is a schematic diagram of the application flow of the system of the present invention. DETAILED DESCRIPTION

[0105] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0106] Example 1

[0107] The present invention provides a method for monitoring crowd density in a multi-scale complex scene, comprising:

[0108] Step 1: Deploy multiple cameras and perform configuration and calibration of multiple cameras.

[0109] The deployment of multiple cameras includes:

[0110] Step 11: Select the camera type and specifications:

[0111] According to the needs of the application scenario, consider the camera's resolution, photosensitivity, and viewing angle, and select a camera that meets the scenario requirements, such as fixed focal length or variable focal length, high resolution, infrared night vision, etc.

[0112] Step 12: Determine the camera installation location:

[0113] Scene analysis: Analyze crowded areas based on the geographical and environmental characteristics of the monitored area, such as entrances and exits of public places, important traffic nodes, etc.

[0114] View planning: Select the position of the camera's view to cover key areas, and cross-cover the cameras to reduce blind spots.

[0115] Step 13: Physical Installation of the Camera:

[0116] According to the preliminary planning, determine the specific installation height and tilt angle of the camera to maximize the field of view coverage efficiency.

[0117] Mount all cameras on a sturdy bracket to prevent vibration from affecting image quality.

[0118] The configuration and calibration of multiple cameras may specifically include:

[0119] Step 14: Camera Calibration:

[0120] Internal parameter calibration: Calibrate the internal parameters of the camera, including setting the focal length and aperture, adjusting the image clarity and exposure level,

[0121] External parameter calibration: adjust the relative position and angle between cameras so that the fields of view of multiple cameras can overlap effectively.

[0122] Step 15: Synchronize the camera time: Set the time synchronization between cameras to ensure the consistency of all data collection in time.

[0123] Step 16: Test and verify the camera: Perform a preliminary camera functionality test to check if the camera's image quality and coverage are as expected.

[0124] Integration test: Integration test is performed at the system level to verify whether the data fusion and collaborative work of multiple cameras are normal.

[0125] Step 2: Reconstruct the 3D coordinates of the scene using images collected by multiple cameras:

[0126] Step 21: Use multiple cameras to simultaneously capture images of the same scene monitoring area from different angles. The images can also be pre-processed:

[0127] For example, synchronization and correction: time synchronization and geometric correction of captured images to eliminate perspective distortion caused by differences in camera angles.

[0128] Image enhancement can also be performed: image enhancement techniques such as contrast adjustment and sharpening can be applied to improve image quality and ensure the recognizability of feature points.

[0129] Step 22: Detecting and matching feature points in the image data may specifically include:

[0130] Identify the correspondence between the same feature points in different images. Let I1 and I2 be images acquired from two different perspectives, where P1 and P2 are the matching feature points in the images.

[0131] The feature point matching function is defined as:

[0132]

[0133] Where descriptor(p) represents the feature descriptor of point p, and ‖·‖ represents the Euclidean distance.

[0134] Step 23: Estimating the disparity based on the matched feature points may specifically include:

[0135] The parallax formula is:

[0136] d = x1-x2

[0137] Calculate the disparity, x1 and x2 are the horizontal coordinates of the feature points in the two images respectively.

[0138] Step 24: Based on the parallax and the known positions and geometric relationships of the cameras, use the triangulation method to calculate the three-dimensional coordinates of each point in the scene, form the three-dimensional coordinates into a three-dimensional point set, integrate the three-dimensional point set into a coherent three-dimensional model, and use the three-dimensional model to represent the spatial structure of the entire monitoring area.

[0139] It may further include step 25: model optimization and verification: refining the initially constructed three-dimensional model, increasing the level of detail of the triangulated model, and improving the accuracy and practicality of the spatial reconstruction; verifying the three-dimensional model through field measurements and additional three-dimensional scanning data to ensure the accuracy and reliability of the model.

[0140] And step 26: integration and application: integrate the constructed 3D model into the monitoring system to enhance the spatial reference of crowd density estimation and behavior analysis, and regularly update the 3D model according to new monitoring data to maintain the timeliness and adaptability of the model.

[0141] Step 3: Feature extraction and data fusion:

[0142] Step 31: When performing feature extraction, select a feature extraction algorithm, such as SIFT, SURF, ORB, etc., according to the scale of the target in different cameras, and perform feature extraction on the collected image.

[0143] Step 32: Adjust the image size according to the scale ratio so that the images collected by different cameras are comparable during feature matching.

[0144] In step 32, according to the scale ratio formula:

[0145]

[0146] Calculate the scale ratio S,

[0147] f1, f2 are the focal lengths of the two cameras, D1, D2 are the distances from the target to the camera,

[0148] Using the formula:

[0149] I′=resize(I,×S)

[0150] The image size is adjusted using the scale ratio S so that images collected by different cameras are comparable during feature matching.

[0151] Step 33: When performing data fusion, images collected from different cameras are matched with multi-view features, and then the matched features are integrated. This may specifically include:

[0152] Perform multi-view feature matching: Use the geometric correction method to ensure the geometric alignment of images from different viewpoints, and use the basic matrix estimation formula:

[0153]

[0154] Describes the epipolar geometric relationship between two perspectives, where p and p' are the homogeneous coordinates of the feature points in the two images, F is the basic matrix,

[0155] To optimize the match:

[0156]

[0157] Where desc(p) is the descriptor of point p, and the matching reliability is enhanced by optimizing the matching score.

[0158] Perform feature fusion: For the intersection area of ​​two cameras, a fusion method based on geometric correction is used, combined with multi-view geometry, such as basic matrix, homography matrix, etc. to correct and fuse image features; for the single-camera perspective area: a multi-scale fusion network is used, such as a pyramid network structure, to integrate features at different scales.

[0159] Step 4: Perform discrete element characterization and simulation:

[0160] Step 41: Characterize discrete elements: Based on the integrated features, each detected individual is simulated as a discrete element, and an initial attribute value is assigned to each discrete element.

[0161] Based on the features extracted in step 3, each discrete element can be assigned an initial attribute value, as follows:

[0162] E i ={x i ,y i , v i ,θ i}

[0163] E i is the ith discrete element, x i ,y i Represents the position coordinates, v i represents the speed, θ i is the moving direction.

[0164] Step 42: Detect neighboring units of discrete elements and set the force model between neighboring units of discrete elements. This may specifically include:

[0165] Define the force model using the formula:

[0166]

[0167] The force model is dynamically adjusted according to the distance between neighboring units, where F ijis the force between neighboring elements i and j, k is the strength coefficient of the force, and σ is the influence range parameter.

[0168] Step 43: Update the position and velocity of each discrete element based on the force model and the current state.

[0169] Specifically, it may include:

[0170] Using the formula:

[0171]

[0172] Update the position and velocity of each discrete element based on the force model and the current state, where Δt is the time step, which is used to indicate the update state from time t to t+1.

[0173] Step 5: Estimating the crowd density:

[0174] Discrete metadata is used to calculate the crowd density in the monitoring area. The formula is as follows:

[0175]

[0176] ρ(x,y,z) represents the density value at position (x,y,z), V is the volume unit considered, and δ is the standard function. i It is 1 when it is within V, otherwise it is 0.

[0177] Use crowd density to create a visual chart that shows the distribution of people in the entire monitoring area.

[0178] The embodiment shows that the method of the present invention accurately estimates the density of the crowd in a complex background through the collaborative work of each module to achieve monitoring. First, the position and field of view of the multi-camera are placed and adjusted through the multi-camera configuration and calibration module to ensure that the camera can cover the monitoring area and perform effective data collection. Secondly, the three-dimensional coordinates of the scene are established using the image data collected by the multi-camera. The disparity between the images of different cameras is analyzed using stereo vision technology to calculate the depth information of each object in the scene, thereby realizing the three-dimensional space reconstruction of the entire monitoring area. Thirdly, feature extraction and data fusion are performed, the camera performs multi-level attention feature extraction, and cross-validates and integrates the targets detected in the image data of multiple cameras through scale mapping and data fusion technology. This process enhances the recognition accuracy of small targets and reduces data redundancy and errors through feature matching. Next, discrete element simulation and characterization are performed to simulate the identified crowd by discrete units. In this step, each person is abstracted as a discrete unit, and the interaction relationship between neighbor units is updated according to real-time data, which helps to accurately simulate crowd dynamics and achieve more detailed behavior analysis. After that, system performance evaluation can be performed. Use real scene data to test the functions of each module and the overall system to verify the accuracy of 3D reconstruction, the accuracy of target detection, and the accuracy and stability of crowd density estimation. Adjust the algorithm and parameters according to the evaluation results to optimize system performance and ensure that the expected results are achieved in practical applications. Finally, the crowd density field is estimated by integrating the 3D coordinates of the scene and the discrete unit representation of personnel obtained in the previous steps, and performing numerical calculations of the density characteristics of dense crowds. Based on the visual information obtained from multiple cameras, the crowd density of the entire monitoring area, including visible and invisible areas, is inferred, realizing the global crowd density field estimation, providing accurate data to support subsequent emergency management and social governance decisions.

[0179] Example 2

[0180] The present invention also provides a crowd density monitoring system in a multi-scale complex scene, including a multi-camera configuration and calibration module, a three-dimensional space reconstruction module, a feature extraction and data fusion module, a discrete element characterization and simulation module, and a crowd density field estimation module.

[0181] The multi-camera configuration and calibration module deploys multiple cameras and performs multi-camera configuration and calibration.

[0182] The 3D space reconstruction module uses images collected by multiple cameras to reconstruct the 3D coordinates of the scene:

[0183] Step 21: Use multiple cameras to simultaneously capture images of the same scene monitoring area from different angles.

[0184] Step 22: Detect and match feature points in the image data.

[0185] Step 23: Estimate disparity based on matched feature points,

[0186] Step 24: Based on the parallax and the known positions and geometric relationships of the cameras, the three-dimensional coordinates of each point in the scene are calculated using a triangulation method, the three-dimensional coordinates are formed into a three-dimensional point set, the three-dimensional point set is integrated into a coherent three-dimensional model, and the spatial structure of the entire monitoring area is represented by the three-dimensional model;

[0187] The feature extraction and data fusion module performs feature extraction and data fusion:

[0188] Step 31: When performing feature extraction, a feature extraction algorithm is selected to extract features from the collected images according to the scale of the target in different cameras.

[0189] Step 32: Adjust the image size according to the scale ratio so that the images collected by different cameras are comparable when performing feature matching;

[0190] Step 33: When performing data fusion, images collected from different cameras are matched with multi-view features, and then the matched features are integrated;

[0191] Discrete element characterization and simulation module performs discrete element characterization and simulation:

[0192] Step 41: Characterize discrete elements: Based on the integrated features, simulate each detected individual as a discrete element, and assign an initial attribute value to each discrete element.

[0193] Step 42: Perform neighbor unit detection on the discrete element and set the force model between neighbor units of the discrete element.

[0194] Step 43: Update the position and velocity of each discrete element based on the force model and the current state;

[0195] The crowd density field estimation module performs crowd density estimation:

[0196] Discrete metadata is used to calculate the crowd density in the monitoring area. The formula is as follows:

[0197]

[0198] ρ(x,y,z) represents the density value at position (x,y,z), V is the volume unit considered, and δ is the standard function. i It is 1 when it is within V, otherwise it is 0.

[0199] Use crowd density to create a visual chart that shows the distribution of people in the entire monitoring area.

[0200] As the information interaction and execution process between the modules in the above-mentioned system are based on the same concept as the embodiment of the method of the present invention, the specific contents can be found in the description of the embodiment of the method of the present invention and will not be repeated here.

[0201] Likewise, the benefits of the system of the present invention are:

[0202] 1. Improved small target recognition accuracy: Through cross-scale correction of multi-camera cross-target detection images, the recognition capability of small targets in complex scenes is improved, ensuring the accuracy and reliability of the recognition results.

[0203] 2. Reduced the impact of crowd occlusion: The impact of crowd occlusion is reduced by multi-camera data fusion, and spatial reconstruction of the monitoring scene and crowd discrete element modeling are performed at the same time, realizing full-domain personnel monitoring and improving the system's coverage and monitoring efficiency.

[0204] 3. Avoid crowd counting jitter: It overcomes the physical limitations of single computer vision technology, combines visual recognition with discrete element model numerical calculation, performs scientific calculation correction based on crowd identification, and provides more stable and accurate crowd count data.

[0205] 4. Effectively monitor the crowd density field: By accurately establishing the relationship between the monitoring scene and the crowd distribution, a crowd density field model is actually created, providing an important decision-making support tool for subsequent social governance and emergency response.

[0206] In addition, the system of the present invention may further include a system performance evaluation module for testing and evaluating system performance.

[0207] The system performance evaluation module conducts test design: determine the key indicators for evaluating system performance, including accuracy, stability, response time, and user satisfaction. Design a series of test cases covering various possible usage scenarios and boundary conditions to comprehensively evaluate the performance of the system.

[0208] Functional verification of system performance evaluation modules: Unit test each module in the system to ensure that they work as expected both individually and when integrated. Test the interfaces and data flows between modules to verify whether they can work together correctly after system integration.

[0209] The system performance evaluation module performs performance evaluation:

[0210] For example, time performance test:

[0211] T response =average(t end -t start )

[0212] Among them, t start and tend are the time points at which the processing starts and ends, T response Indicates the average response time of the system.

[0213] Accuracy testing is performed to evaluate how well the system output, such as crowd density estimates, agrees with actual data. This is usually measured by comparing the predictions to the ground truth.

[0214] Optimize and iterate the system performance evaluation module: adjust and optimize the system according to the test results, including software and hardware configuration updates, algorithm adjustments, and function enhancements. Release system update versions to continuously improve system performance and user experience.

[0215] In summary, it should be noted that not all steps and modules in the above-mentioned processes and system structures are necessary, and some steps or modules can be ignored according to actual needs. The execution order of each step is not fixed and can be adjusted as needed. The system structure described in the above-mentioned embodiments can be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or some components in multiple independent devices may be implemented together.

[0216] The above-described embodiments are only preferred embodiments for fully illustrating the present invention, and the protection scope of the present invention is not limited thereto. Equivalent substitutions or changes made by those skilled in the art based on the present invention are within the protection scope of the present invention. The protection scope of the present invention shall be subject to the claims.

Claims

1. A method for monitoring crowd density in multi-scale complex scenes, characterized by include: Step 1: Deploy multiple cameras and perform configuration and calibration. Step 2: Reconstruct the 3D coordinates of the scene using images collected by multiple cameras: Step 21: Use multiple cameras to simultaneously capture images of the same scene monitoring area from different angles. Step 22: Detect and match feature points in the image data. Step 23: Estimate disparity based on matched feature points, Step 24: Based on the parallax and the known positions and geometric relationships of the cameras, the three-dimensional coordinates of each point in the scene are calculated using a triangulation method, the three-dimensional coordinates are formed into a three-dimensional point set, the three-dimensional point set is integrated into a coherent three-dimensional model, and the spatial structure of the entire monitoring area is represented by the three-dimensional model; Step 3: Feature extraction and data fusion: Step 31: When performing feature extraction, a feature extraction algorithm is selected to extract features from the collected images according to the scale of the target in different cameras. Step 32: Adjust the image size according to the scale ratio so that the images collected by different cameras are comparable when performing feature matching; Step 33: When performing data fusion, images collected from different cameras are matched with multi-view features, and then the matched features are integrated; Step 4: Perform discrete element characterization and simulation: Step 41: Characterize discrete elements: Based on the integrated features, simulate each detected individual as a discrete element, and assign an initial attribute value to each discrete element. Step 42: Perform neighbor unit detection on the discrete element and set the force model between neighbor units of the discrete element. Step 43: Update the position and velocity of each discrete element based on the force model and the current state; Step 5: Estimation of crowd density: Discrete metadata is used to calculate the crowd density in the monitoring area. The formula is as follows: ρ(x,y,z) represents the density value at position (x,y,z), V is the volume unit considered, and δ is the standard function. i It is 1 when it is within V, otherwise it is 0. Use crowd density to create a visual chart that shows the distribution of people in the entire monitoring area.

2. According to the method for monitoring crowd density in a multi-scale complex scene according to claim 1, it is characterized in that multiple cameras are deployed in step 1, specifically comprising: Step 11: Select the camera type and specifications: According to the needs of the application scenario, consider the camera's resolution, photosensitivity, and viewing angle, and choose a camera that meets the needs of the scenario. Step 12: Determine the camera installation location: Scene analysis: Analyze crowded areas based on the geographical and environmental characteristics of the monitored area. View planning: Select the position of the camera's view to cover key areas, and cross-cover the cameras to reduce blind spots. Step 13: Physical Installation of the Camera: According to the preliminary planning, determine the specific installation height and tilt angle of the camera to maximize the field of view coverage efficiency. Mount all cameras on a sturdy bracket to prevent vibration from affecting image quality.

3. The method for monitoring crowd density in a multi-scale complex scene according to claim 1 or 2, characterized in that The configuration and calibration of multiple cameras in step 1 specifically includes: Step 14: Camera Calibration: Internal parameter calibration: Calibrate the internal parameters of the camera, including setting the focal length and aperture, adjusting the image clarity and exposure level, External parameter calibration: adjust the relative position and angle between cameras so that the fields of view of multiple cameras can overlap effectively. Step 15: Synchronize the camera time: Set the time synchronization between cameras to ensure the consistency of all data collection in time. Step 16: Test and verify the camera: Perform a preliminary camera functionality test to check if the camera's image quality and coverage are as expected. Integration test: Integration test is performed at the system level to verify whether the data fusion and collaborative work of multiple cameras are normal.

4. The method for monitoring crowd density in a multi-scale complex scene according to claim 1, characterized in that Step 22 performs feature point detection and matching in the image data, including: Identify the correspondence between the same feature points in different images. Let I1 and I2 be images acquired from two different perspectives, where P1 and P2 are the matching feature points in the images. The feature point matching function is defined as: Where descriptor(p) represents the feature descriptor of point p, ‖·‖ represents the Euclidean distance; In step 23, the disparity is estimated based on the matched feature points, including: The parallax formula is: d = x1-x2 Calculate the disparity, x1 and x2 are the horizontal coordinates of the feature points in the two images respectively.

5. The method for monitoring crowd density in a multi-scale complex scene according to claim 1, characterized in that It also includes step 25: model optimization and verification: refine the initially constructed 3D model, increase the level of detail of the triangulated model, and improve the accuracy and practicality of the spatial reconstruction; verify the 3D model through field measurements and additional 3D scanning data to ensure the accuracy and reliability of the model; Step 26: Integration and Application: Integrate the constructed 3D model into the monitoring system to enhance the spatial reference for crowd density estimation and behavior analysis. Regularly update the 3D model based on new monitoring data to maintain the timeliness and adaptability of the model.

6. The method for monitoring crowd density in a multi-scale complex scene according to claim 1, characterized in that In step 32, according to the scale ratio formula: Calculate the scale ratio S, f1, f2 are the focal lengths of the two cameras, D1, D2 are the distances from the target to the camera, Using the formula: I′=resize(I,×S) The image size is adjusted using the scale ratio S so that images collected by different cameras are comparable during feature matching.

7. The method for monitoring crowd density in a multi-scale complex scene according to claim 1, characterized in that Step 33 includes: Perform multi-view feature matching: Use the geometric correction method to ensure the geometric alignment of images from different viewpoints, and use the basic matrix estimation formula: Describes the epipolar geometric relationship between two perspectives, where p and p' are the homogeneous coordinates of the feature points in the two images, F is the basic matrix, To optimize the match: Where desc(p) is the descriptor of point p, and the matching reliability is enhanced by optimizing the matching score. Perform feature fusion: For the intersection area of ​​dual cameras, a fusion method based on geometric correction is adopted, combining multi-view geometric correction and fusion image features; for the single-camera perspective area: a multi-scale fusion network is used to integrate features at different scales.

8. The method for monitoring crowd density in a multi-scale complex scene according to claim 1, characterized in that the steps In 41, based on the features extracted in step 3, each discrete element is assigned an initial attribute value, and the formula is as follows: E i {x i y i v i ,θ i } E i is the ith discrete element, x i ,y i Represents the position coordinates, v i represents the speed, θ i is the direction of movement, In step 42, the force model between neighboring elements of the discrete element is set, including: Define the force model using the formula: The force model is dynamically adjusted according to the distance between neighboring units, where F ij is the force between neighboring units i and j, k is the strength coefficient of the force, σ is the influence range parameter, Step 43 includes: Using the formula: Update the position and velocity of each discrete element based on the force model and the current state, where Δt is the time step, which is used to indicate the update state from time t to t+1.

9. A crowd density monitoring system in a multi-scale complex scene, characterized by It includes multi-camera configuration and calibration module, 3D space reconstruction module, feature extraction and data fusion module, discrete element characterization and simulation module and crowd density field estimation module. The multi-camera configuration and calibration module deploys multiple cameras and performs multi-camera configuration and calibration. The 3D space reconstruction module uses images collected by multiple cameras to reconstruct the 3D coordinates of the scene: Step 21: Use multiple cameras to simultaneously capture images of the same scene monitoring area from different angles. Step 22: Detect and match feature points in the image data. Step 23: Estimate disparity based on matched feature points, Step 24: Based on the parallax and the known positions and geometric relationships of the cameras, the three-dimensional coordinates of each point in the scene are calculated using a triangulation method, the three-dimensional coordinates are formed into a three-dimensional point set, the three-dimensional point set is integrated into a coherent three-dimensional model, and the spatial structure of the entire monitoring area is represented by the three-dimensional model; The feature extraction and data fusion module performs feature extraction and data fusion: Step 31: When performing feature extraction, a feature extraction algorithm is selected to extract features from the collected images according to the scale of the target in different cameras. Step 32: Adjust the image size according to the scale ratio so that the images collected by different cameras are comparable when performing feature matching; Step 33: When performing data fusion, images collected from different cameras are matched with multi-view features, and then the matched features are integrated; Discrete element characterization and simulation module performs discrete element characterization and simulation: Step 41: Characterize discrete elements: Based on the integrated features, simulate each detected individual as a discrete element, and assign an initial attribute value to each discrete element. Step 42: Perform neighbor unit detection on the discrete element and set the force model between neighbor units of the discrete element. Step 43: Update the position and velocity of each discrete element based on the force model and the current state; The crowd density field estimation module performs crowd density estimation: Discrete metadata is used to calculate the crowd density in the monitoring area. The formula is as follows: ρ(x,y,z) represents the density value at position (x,y,z), V is the volume unit considered, and δ is the standard function. i It is 1 when it is within V, otherwise it is 0. Use crowd density to create a visual chart that shows the distribution of people in the entire monitoring area.

10. The crowd density monitoring system in a multi-scale complex scene according to claim 9, characterized in that It also includes a system performance evaluation module for testing and evaluating system performance.

Images