Multi-view visual identification fused intelligent playground group movement behavior analysis system

Abstract

The invention relates to the technical field of behavior detection and recognition methods, and discloses a multi-view visual recognition fused intelligent playground group movement behavior analysis system, which comprises a playground data collection module, a playground group movement behavior analysis module and a multi-view visual recognition fusion module, the data processing module is used for processing the collected playground data to obtain playground group movement data; the analysis and decision module is used for carrying out deep analysis and decision judgment on playground group movement data so as to realize effective control on group movement behaviors and playground safety management; through cooperative work of the playground data collection module, the data processing module and the analysis decision module, playground group movement behaviors can be comprehensively and accurately monitored, analyzed and evaluated, the shielding problem in a crowded scene is considered, and the result accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of behavior detection and recognition methods, and more specifically, to a multi-view visual recognition fusion smart playground group motion behavior analysis system. Background Art

[0002] Multiple high-definition cameras are deployed around the playground to build an all-round visual acquisition network with no blind spots. Each camera adopts an optimized field of view and focal length parameters to synchronously capture video streams at a fixed frequency. The system performs spatiotemporal alignment and synchronous processing of multi-channel video data, uses camera calibration technology to establish a mapping relationship between perspectives, and integrates multi-perspective information into a unified world coordinate system through a 3D reconstruction algorithm. On this basis, the system can extract the spatiotemporal trajectory of moving targets, calculate group motion characteristics including density distribution, velocity field, and acceleration field, and thus identify typical motion patterns and abnormal behaviors. However, in crowded scenes, the target is severely occluded, the target detection accuracy is significantly reduced, the positioning accuracy of key points of the human body is reduced, and the motion trajectory tracking is easily broken, resulting in unstable behavior recognition results, which in turn affects the group behavior analysis results. Summary of the invention

[0003] The purpose of the present invention is to provide a smart playground group motion behavior analysis system with multi-perspective visual recognition fusion in order to solve the above problems.

[0004] The present invention provides a multi-view visual recognition fusion smart playground group movement behavior analysis system, comprising:

[0005] The playground data collection module is used to collect playground data, which includes area data and environmental data;

[0006] The data processing module is used to process the collected playground data to obtain playground group movement data; including calibration and synchronization of the collection equipment to ensure the accuracy and consistency of the data; three-dimensional reconstruction is achieved by analyzing the regional data to obtain the three-dimensional position information of the personnel on the playground; and group movement features are extracted at the same time, and the playground group movement data is obtained by combining the environmental data collected by the playground data collection module;

[0007] The analysis and decision-making module is used to conduct in-depth analysis and decision-making on playground group movement data.

[0008] Furthermore, the area includes a running track area, a football field area, a basketball court area, an equipment area, and a rest area.

[0009] Furthermore, regional data is obtained through high-definition cameras, and multiple high-definition cameras are installed around the playground to build a video acquisition network with no blind spots.

[0010] Furthermore, the intrinsic parameter matrix of each camera is defined as:

[0011] ;

[0012] in, and is the focal length parameter, and its value is selected according to the camera model and the required shooting range; and The coordinates of the principal point are determined according to the optical characteristics and installation position of the camera;

[0013] The extrinsic parameter matrix of each camera is expressed as ,in is the rotation matrix, which is used to describe the rotation relationship of the camera relative to the world coordinate system. is a translation vector that represents the position offset of the camera in space; by accurately measuring and setting the external parameter matrix, the accurate fusion of data collected by different cameras in the same coordinate system is achieved;

[0014] The distortion parameters of each camera use a five-parameter model:

[0015] ;

[0016] in represents the distortion parameter, , , is the radial distortion coefficient, , is the tangential distortion coefficient.

[0017] Furthermore, the video acquisition parameters include sampling frequency, video resolution, video encoding, and color space, where the sampling frequency Set to , video resolution Set to The video encoding adopts H.264 method, and the color space supports RGB / YUV dual-mode color space.

[0018] Furthermore, the group movement features extracted by the data processing module include density distribution features and kinematic features, wherein the density distribution features include the number of people in the area, the density heat map, and the flow direction vector, and the kinematic features include the velocity field and the acceleration field.

[0019] Furthermore, a multi-view detection result fusion method based on confidence weighting is used to deal with occlusion. The calculation method is as follows:

[0020] For viewing angle The confidence of the detection result in is defined as:

[0021] ;

[0022] in, For perspective The confidence value of is the target visibility, It is the detection quality score, which is calculated by the data processing module during the target detection process based on the accuracy and recall rate indicators of the detection algorithm. Depth reliability is determined by analyzing the target depth information obtained by 3D reconstruction and the depth relationship with other targets. , , is the weight coefficient;

[0023] The 3D position estimate after the result fusion is:

[0024] ;

[0025] in represents the fused three-dimensional position, From the perspective The three-dimensional position estimation of Indicates the number of viewing angles.

[0026] Furthermore, the analysis and decision module receives the three-dimensional reconstruction and target detection results provided by the data processing module, combines the depth information of each target, determines the occlusion relationship between the targets according to the depth sorting rules, and uses the timing information in the continuous video frames to predict the position of the occluded target, thereby establishing an occlusion map between the targets.

[0027] Furthermore, it also includes:

[0028] Based on the playground group movement data, the risk factor is calculated to assess the safety status of each area of ​​the playground. When the risk factor exceeds the preset threshold, the corresponding early warning mechanism is triggered according to different levels.

[0029] System defined risk factor for:

[0030] ;

[0031] in, The weights of each region are reasonably allocated according to the importance of different regions in terms of security, and higher weight values ​​are given; is the risk assessment function, combined with the population density of each area , Movement speed , acceleration and a collection of environmental factors , assess the risk level of each area;

[0032] It is a collection of environmental factors, including light intensity, weather conditions, temperature, and air quality index.

[0033] The present invention provides a computer storage medium for storing computer-readable instructions, which, when read, can execute the aforementioned multi-perspective visual recognition fusion smart playground group motion behavior analysis system.

[0034] The beneficial effects of the present invention are as follows: through the collaborative work of the playground data collection module, the data processing module and the analysis and decision-making module, the present invention can comprehensively and accurately monitor, analyze and evaluate the group movement behavior on the playground, and takes into account the occlusion problem in crowded scenes, thereby improving the accuracy of the results. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a system module diagram of the present invention. DETAILED DESCRIPTION

[0036] The subject matter described herein will now be discussed with reference to example embodiments. It should be understood that the discussion of these embodiments is only to enable those skilled in the art to better understand and implement the subject matter described herein, and the functions and arrangements of the elements discussed may be changed without departing from the scope of protection of the contents of this specification. Each example may omit, replace or add various processes or components as needed. In addition, the features described relative to some examples may also be combined in other examples.

[0037] At least one embodiment of the present invention discloses a multi-view visual recognition fusion smart playground group movement behavior analysis system, such as Figure 1 As shown, including:

[0038] A playground data collection module 101 is used to collect playground data, which includes area data and environment data;

[0039] In one embodiment of the present invention, the area includes a running track area, a football field area, a basketball court area, an equipment area, and a rest area;

[0040] In one embodiment of the present invention, the regional data is obtained through a high-definition camera, and multiple high-definition cameras are installed around the playground to build an all-round video acquisition network without blind spots;

[0041] The intrinsic parameter matrix of each camera is defined as:

[0042]

[0043] in, and The focal length parameter is selected according to the camera model and the required shooting range, aiming to obtain a clear video image with a suitable viewing angle; and The coordinates of the principal point are also determined based on the optical characteristics and installation position of the camera to ensure the accuracy of the image coordinate system.

[0044] The extrinsic parameter matrix of each camera is expressed as ,in is the rotation matrix, which is used to describe the rotation relationship of the camera relative to the world coordinate system. is a translation vector that represents the position offset of the camera in space. By accurately measuring and setting the external parameter matrix, the data collected by different cameras can be accurately fused in the same coordinate system;

[0045] The distortion parameters of each camera use a five-parameter model:

[0046]

[0047] in represents the distortion parameter, , , is the radial distortion coefficient, , is the tangential distortion coefficient.

[0048] After the camera is installed, professional calibration tools and methods are used to calibrate these distortion parameters to eliminate the impact of lens distortion on video image quality and ensure that the collected video data can truly reflect the playground scene and personnel situation.

[0049] In one embodiment of the present invention, the video acquisition parameters include sampling frequency, video resolution, video encoding, and color space, wherein the sampling frequency Set to , video resolution Set to The video encoding adopts H.264 method, and the color space supports RGB / YUV dual-mode color space. In the actual video acquisition process, the color mode can be flexibly switched according to different environmental lighting conditions, analysis requirements, and compatibility of subsequent data processing algorithms.

[0050] In one embodiment of the present invention, the environmental data includes light intensity, weather conditions, temperature, and air quality index;

[0051] Among them, the light intensity The weather conditions in different areas of the playground are obtained by real-time monitoring of the light sensor Obtained through weather sensors or data connection with external weather data sources, including sunny, cloudy, rainy, snowy and other conditions, temperature The air quality index is obtained by real-time monitoring of different areas of the playground through temperature sensors. , obtained by real-time monitoring of different areas of the playground through air quality monitoring equipment, including indicators such as PM2.5.

[0052] The data processing module 102 is used to process the collected playground data to obtain playground group motion data; specifically, it includes calibrating and synchronizing the acquisition equipment to ensure the accuracy and consistency of the data, realizing three-dimensional reconstruction by analyzing the regional data to obtain the three-dimensional position information of the personnel on the playground, and extracting the group motion characteristics, and combining the environmental data collected by the playground data collection module;

[0053] In one embodiment of the present invention, a standard checkerboard calibration plate is used to calibrate the acquisition device, and the internal parameter matrix is ​​solved by Zhang's calibration method. , and then use the PnP algorithm to solve the external parameter matrix ; Place the checkerboard calibration board at different locations on the playground, collect its image through the camera, and use Zhang's calibration method to calculate the internal parameter matrix based on the collected image feature points , determine the internal optical parameters of the camera; then, based on the known position of the calibration plate in the world coordinate system and the position of the calibration plate in the image coordinate system obtained through image recognition, the PnP algorithm is used to calculate the external parameter matrix , thereby determining the position and posture relationship of the camera relative to the world coordinate system.

[0054] In one embodiment of the present invention, multi-machine clock synchronization is achieved based on the NTP protocol, and timestamps are used to align multiple video streams to achieve a synchronous sampling rate of 30fps. Since multiple cameras in the playground data collection module work simultaneously, video synchronization processing is required to ensure the temporal consistency of video data from each perspective. The clock of the device where each camera is located is synchronized with the standard time source through the NTP protocol, and then the timestamp technology is used to align the video streams to ensure that the video data from different perspectives collected at the same time can accurately correspond.

[0055] In one embodiment of the present invention, SIFT features are used for cross-view matching; after receiving the synchronized video data, the data processing module extracts SIFT feature points from the video images of different viewpoints, and uses the SIFT feature's invariance to image scale and rotation to find corresponding feature points in the video images of different viewpoints; then, the RANSAC algorithm is used to eliminate mismatched points and establish point correspondences. In the specific operation, the similarity of SIFT feature points in video images of different viewing angles is calculated to find possible matching point pairs, and then these matching point pairs are input into the RANSAC algorithm. Through the iterative calculation of the algorithm, those erroneous matching points that do not conform to the geometric relationship are eliminated, and finally an accurate two-dimensional point correspondence relationship is established, laying the foundation for subsequent three-dimensional reconstruction.

[0056] In one embodiment of the present invention, three-dimensional coordinates are reconstructed based on the triangulation principle. ; Using the established point correspondence According to the principle of triangulation, the position coordinates of the target in three-dimensional space are determined by calculating the geometric relationship of corresponding feature points under different viewing angles; and, in order to improve the reconstruction accuracy, uncertainty estimation is introduced; in the process of calculating the three-dimensional coordinates, the influence of measurement errors, image noise and other factors that may affect the results is taken into account, and an uncertainty value is assigned to each calculated three-dimensional coordinate point by establishing an uncertainty model. The reconstruction result is adjusted and optimized according to this uncertainty value, and finally centimeter-level positioning accuracy is achieved, so that the three-dimensional position information of people on the playground can be obtained more accurately.

[0057] In one embodiment of the present invention, the group movement features extracted by the data processing module include density distribution features and kinematic features, wherein the density distribution features include the number of people in the area, the density heat map, and the flow direction vector, and the kinematic features include the velocity field and the acceleration field;

[0058] Regional population : By analyzing video data from different perspectives, the target detection algorithm is used to identify individuals in different areas, and then counting statistics are performed to obtain the number of people in each area at different times.

[0059] Density Heatmap :Based on the regional population statistics results and the division of the playground area, each area is divided into several small grids, and the population density in each small grid is calculated. A density heat map is drawn with different colors according to the level of population density, which intuitively reflects the degree of gathering and flow trend of people in different areas of the playground.

[0060] Flow direction vector : By analyzing the position changes of people in continuous video frames, the flow direction of people in each area is determined and quantified into flow direction vectors to further understand the flow patterns of people on the playground.

[0061] Velocity field: :By performing differential calculation on the changes in the positions of people in the video data at different times, we can obtain the The instantaneous velocities at the locations are combined to form a velocity field to understand the velocity distribution of the crowd moving on the playground.

[0062] Acceleration field: :Based on the velocity field, the velocity values ​​in the velocity field are differentially calculated again to obtain the velocity values ​​of personnel at different positions. The instantaneous accelerations at the points are combined to form an acceleration field, so as to gain an in-depth understanding of the speed changes and acceleration of the crowd moving on the playground, providing an important basis for the comprehensive analysis of group movement behavior.

[0063] The analysis and decision-making module 103 is used to conduct in-depth analysis and decision-making on the playground group movement data, so as to achieve effective control of group movement behavior and playground safety management; its main process includes: for the occlusion situation that may occur in the group movement, the detection results of different perspectives are integrated and processed through the multi-perspective detection result fusion method, the occlusion relationship is accurately judged and the movement trajectory of the occluded target is maintained, and the movement behavior analysis is ensured not to be affected by the occlusion; based on the playground group movement data, the risk factor is calculated to evaluate the safety status of each area of ​​the playground. When the risk factor exceeds the preset threshold, the corresponding early warning mechanism is triggered according to different levels, such as prompts, strengthening management measures or emergency evacuation operations; at the same time, various outputs are generated based on the processed and analyzed data, covering three-dimensional reconstruction models that intuitively display the group movement situation, comprehensive analysis reports, and site optimization suggestions and safety management suggestions that provide guidance for playground management.

[0064] In one embodiment of the present invention, a multi-view detection result fusion method based on confidence weighting is used to process the occlusion situation. The specific calculation method is as follows:

[0065] For viewing angle The confidence of the detection result in is defined as:

[0066]

[0067] in, For perspective The confidence value of The target visibility is determined by analyzing the clarity of the target in the video data of each perspective, whether it is partially blocked, etc. It reflects the visibility of the target at that perspective and directly affects the reliability of the detection result. It is the detection quality score, which is calculated by the data processing module during the target detection process based on the accuracy, recall rate and other indicators of the detection algorithm. It reflects the accuracy of the detection algorithm in target detection from this perspective. For depth reliability, considering the influence of the depth information of the target in three-dimensional space on the detection results, the depth information of the target obtained by three-dimensional reconstruction and the depth relationship with other targets are analyzed to determine the depth reliability. , , is the weight coefficient. The importance of different factors is adjusted according to the actual situation. For example, in the scenario where the target visibility has a greater impact on the result, the The value of .

[0068] The 3D position estimate after the result fusion is:

[0069]

[0070] in represents the fused three-dimensional position, From the perspective The three-dimensional position estimation of Indicates the number of viewing angles.

[0071] Through this confidence-weighted multi-view fusion strategy, the analysis and decision-making module can effectively integrate the detection results from different viewpoints, improve the accuracy of target detection and the precision of three-dimensional position estimation under occlusion, and thus better analyze the actual situation of group motion behavior under occlusion.

[0072] In one embodiment of the present invention, occlusion relationship reasoning is performed based on depth sorting, and the position of the occluded target is predicted using time series information to establish an occlusion graph between targets.

[0073] The analysis and decision module receives the 3D reconstruction and target detection results provided by the data processing module, combines the depth information of each target, and determines the occlusion relationship between targets according to the depth sorting rules. At the same time, it uses the timing information in continuous video frames to predict the position of the occluded target, thereby establishing an occlusion map between targets and providing a clear occlusion situation analysis for subsequent processing.

[0074] In one embodiment of the present invention, the ID consistency of the occluded target is maintained, the trajectory during the occlusion is predicted based on the motion model, and the trajectory association is performed after the occlusion is released;

[0075] When processing the motion trajectory of the occluded target, the analysis and decision-making module ensures that each occluded target maintains a unique ID throughout the entire process, and predicts the trajectory during the occlusion period based on the target's previous motion state and motion model. When the occlusion is released, the predicted trajectory is associated with the actual detected trajectory to ensure the consistency of the motion trajectory, so as to accurately analyze the role and influence of the occluded target in the entire group motion behavior.

[0076] In one embodiment of the present invention, the system defines the risk factor for:

[0077]

[0078] in, The weights of each region are reasonably allocated according to the importance of different regions in terms of security, and higher weight values ​​are given; is the risk assessment function, combined with the population density of each area , Movement speed , acceleration and a collection of environmental factors , assess the risk level of each area;

[0079] The analysis and decision-making module receives the group feature extraction results provided by the data processing module, including information such as crowd density, velocity field, acceleration field, and the set of environmental factors collected by the playground data collection module. , substituting these data into the risk assessment function Calculate the risk level of each area;

[0080] It is a collection of environmental factors, including the light intensity, weather conditions, temperature, air quality index, etc. mentioned above. These environmental factors will also affect the safety of people on the playground.

[0081] In one embodiment of the present invention, when Exceeding the preset threshold When an early warning is triggered, the analysis and decision-making module is responsible for executing the early warning mechanism:

[0082] Level 1 warning: , which indicates that there are certain safety hazards. The analysis and decision-making module will issue corresponding prompt information, such as displaying a yellow warning sign on the monitoring interface, and at the same time notify relevant management personnel to pay attention to the situation and take corresponding preventive measures, such as strengthening patrols.

[0083] Level 2 warning: In this case, the safety risk has increased. In addition to issuing stronger prompt information (such as displaying an orange warning sign on the monitoring interface), the analysis and decision-making module will also require relevant managers to further strengthen safety management measures, such as increasing patrol personnel and inspecting site facilities.

[0084] Level 3 warning: When it reaches the third level of warning, it means that the safety risk is already very high. The analysis and decision-making module will immediately trigger emergency measures, such as issuing evacuation orders to people on the playground through the broadcasting system, and notifying all relevant managers and security personnel to arrive at the scene quickly, organize people to evacuate in an orderly manner, and coordinate relevant resources such as medical first aid to make emergency preparations to ensure the safety of people on the playground.

[0085] Through the collaborative work of the playground data collection module, the data processing module and the analysis and decision-making module, the smart playground group movement behavior analysis system of the present invention can comprehensively and accurately monitor, analyze and evaluate the playground group movement behavior, and provide practical output results, providing strong support for the management of the playground and the safety of personnel.

[0086] In at least one embodiment of the present invention, a computer storage medium is provided for storing computer-readable instructions, which, when read, can execute the aforementioned multi-perspective visual recognition fusion smart playground group motion behavior analysis system.

[0087] The above describes an embodiment of the present invention, but this embodiment is not limited to the above-mentioned specific implementation mode. The above-mentioned specific implementation mode is merely illustrative and not restrictive. Under the guidance of this embodiment, ordinary technicians in this field can also make more forms of equivalent embodiments, all of which are within the protection of this embodiment.

Claims

1. A multi-view visual recognition fusion smart playground group movement behavior analysis system, characterized by: include: The playground data collection module is used to collect playground data, which includes area data and environmental data; The data processing module is used to process the collected playground data to obtain playground group movement data; including calibration and synchronization of the collection equipment to ensure the accuracy and consistency of the data; three-dimensional reconstruction is achieved by analyzing the regional data to obtain the three-dimensional position information of the personnel on the playground; and group movement features are extracted at the same time, and the playground group movement data is obtained by combining the environmental data collected by the playground data collection module; The analysis and decision-making module is used to conduct in-depth analysis and decision-making on playground group movement data.

2. According to the multi-view visual recognition fusion intelligent playground group movement behavior analysis system of claim 1, it is characterized by: The areas include running track area, football field area, basketball court area, equipment area and rest area.

3. According to the multi-view visual recognition fusion intelligent playground group movement behavior analysis system of claim 1, it is characterized in that: Regional data is obtained through high-definition cameras. Multiple high-definition cameras are installed around the playground to build a video acquisition network with no blind spots.

4. According to claim 3, a multi-view visual recognition fusion intelligent playground group movement behavior analysis system is characterized in that: The intrinsic parameter matrix of each camera is defined as: ； in, and is the focal length parameter, and its value is selected according to the camera model and the required shooting range; and The coordinates of the principal point are determined according to the optical characteristics and installation position of the camera; The extrinsic parameter matrix of each camera is expressed as ,in is the rotation matrix, which is used to describe the rotation relationship of the camera relative to the world coordinate system. is a translation vector that represents the position offset of the camera in space; by accurately measuring and setting the external parameter matrix, the accurate fusion of data collected by different cameras in the same coordinate system is achieved; The distortion parameters of each camera use a five-parameter model: ； in represents the distortion parameter, , , is the radial distortion coefficient, , is the tangential distortion coefficient.

5. According to the multi-view visual recognition fusion intelligent playground group movement behavior analysis system of claim 1, it is characterized in that: Video acquisition parameters include sampling frequency, video resolution, video encoding, and color space. Set to , video resolution Set to The video encoding adopts H.264 method, and the color space supports RGB / YUV dual-mode color space.

6. The multi-view visual recognition fusion intelligent playground group movement behavior analysis system according to claim 1 is characterized in that: The group motion features extracted by the data processing module include density distribution features and kinematic features. The density distribution features include the number of people in the area, density heat map, and flow direction vector, and the kinematic features include velocity field and acceleration field.

7. The multi-view visual recognition fusion intelligent playground group movement behavior analysis system according to claim 1 is characterized in that: The occlusion situation is handled by using a multi-view detection result fusion method based on confidence weighting. The calculation method is as follows: For viewing angle The confidence of the detection result in is defined as: ； in, For perspective The confidence value of is the target visibility, It is the detection quality score, which is calculated by the data processing module during the target detection process based on the accuracy and recall rate indicators of the detection algorithm. Depth reliability is determined by analyzing the target depth information obtained by 3D reconstruction and the depth relationship with other targets. , , is the weight coefficient; The 3D position estimate after the result fusion is: ； in represents the fused three-dimensional position, From the perspective The three-dimensional position estimation of Indicates the number of viewing angles.

8. The multi-view visual recognition fusion intelligent playground group movement behavior analysis system according to claim 7 is characterized in that: The analysis and decision module receives the 3D reconstruction and target detection results provided by the data processing module, combines the depth information of each target, determines the occlusion relationship between targets according to the depth sorting rules, and uses the timing information in continuous video frames to predict the position of the occluded target, thereby establishing an occlusion map between targets.

9. The multi-view visual recognition fusion intelligent playground group movement behavior analysis system according to claim 1 is characterized in that: Also includes: Based on the playground group movement data, the risk factor is calculated to assess the safety status of each area of ​​the playground. When the risk factor exceeds the preset threshold, the corresponding early warning mechanism is triggered according to different levels. System defined risk factor for: ； in, The weights of each region are reasonably allocated according to the importance of different regions in terms of security, and higher weight values ​​are given; is the risk assessment function, combined with the population density of each area , Movement speed , acceleration and a collection of environmental factors , assess the risk level of each area; It is a collection of environmental factors, including light intensity, weather conditions, temperature, and air quality index.

10. A computer storage medium, characterized in that: It is used to store computer-readable instructions, which, when read, can execute a multi-perspective visual recognition fusion smart playground group motion behavior analysis system as described in any one of claims 1-9.

Images