Crowd motion saliency detection method based on elastic collision and vector standard deviation

Abstract

The application discloses a large-scale crowd motion saliency detection method based on elastic collision and vector standard deviation. In the application, each vector in a crowd vector field is regarded as an elastic collision particle, and a collision force between the particles is obtained through an elastic collision theory in physics, and a crowd saliency feature is constructed with the aid of a crowd vector standard deviation. Firstly, a crowd vector field is extracted through an optical flow method, then the vector is regarded as an elastic collision particle, whether the particles can collide with each other is judged, and a collision force between the particles is obtained as a saliency feature through the aid of the elastic collision theory, the saliency feature map is optimized through the crowd vector standard deviation, and finally a crowd saliency region is detected through threshold segmentation.

Description

Technical Field

[0001] This invention relates to the field of video analysis and image processing, and in particular to a method for detecting the saliency of motion in large crowds based on elastic collision and vector standard deviation. Background Technology

[0002] With the development of computer technology, video saliency detection plays a vital role in daily life and has attracted great interest from researchers in related fields. However, saliency detection of crowd movement in videos remains a complex problem. Over the past few decades, with global population growth and the increase in various activity venues, incidents involving crowds have become increasingly frequent. Places such as train stations, sports fields, and temple fairs are particularly crowded, and relying solely on staff to monitor these areas in real time often cannot meet the needs. Furthermore, human attention spans decrease over time, leading to fatigue and a lack of supervisory engagement. Therefore, the demand for automated real-time monitoring systems is growing daily.

[0003] Specifically, in large-scale crowd behavior analysis, some researchers consider any deviation from generally observed events to constitute salient behavior; others consider rare or prominent events to be salient behaviors. Many existing saliency detection methods are almost exclusively designed for static images. However, for videos, the complexity of the scene and motion features are far greater. Therefore, saliency detection methods for dynamic motion scenes (especially dense crowd motion scenes) deserve more attention and research.

[0004] In large-scale crowd movement scenarios, due to the various motion states of the crowd, collisions are prone to occur in some local areas, leading to crowd safety incidents. To study the motion relationships between crowd particles more deeply, the crowd is regarded as elastically colliding particles. In the crowd area where particle collisions occur, the magnitude of the collision force on a particle from surrounding particles is extracted using the elastic collision theory of physics, and this is used as a salience detection feature. In areas with larger collision forces, the crowd movement is more chaotic, and the central particle exhibits a larger behavioral difference from the surrounding particles. The standard deviation of the particle vector is used to further characterize the magnitude of this difference. Finally, a salience detection map of crowd movement is constructed using these two features. Therefore, the elastic collision theory has become another useful tool for crowd behavior analysis. Summary of the Invention

[0005] To address the aforementioned technical problems in existing technologies, the purpose of this invention is to treat crowd movement as elastic particles, utilize perfectly elastic collision theory and standard deviation to construct a crowd saliency map, and then detect saliency regions of crowd movement. The specific technical solution is as follows:

[0006] A method for detecting the saliency of large-scale crowd motion based on elastic collision and vector standard deviation is proposed. This method treats vectors in the crowd vector field as elastic collision particles, calculates the collision force between particles using the theory of elasticity in physics, and uses this force as a feature to detect saliency of crowd motion. The collision force feature is then optimized using vector standard deviation to ultimately obtain the saliency region of crowd motion. The method includes the following steps:

[0007] Step 1: Process the input raw video frame sequence using the HS optical flow method, and extract the crowd vector field by consecutive adjacent frames;

[0008] Step 2: In the vector field of crowd motion, analyze and determine whether collisions will occur between crowd particles, and save the particles that can collide for subsequent calculation of the collision force between particles.

[0009] Step 3: In a group of particles that will collide, the movement of the crowd is regarded as elastic particles, and the collision force characteristics between particles are extracted by using the theory of elastic collision in physics.

[0010] Step 4: Calculate the standard deviation between the particle vector and the surrounding particles in the local area. Use the standard deviation to measure the degree of difference in the motion behavior between particles, and use the standard deviation to optimize the elastic collision force feature, thereby constructing a feature map for the saliency detection of the crowd.

[0011] The specific method described in step 1 is as follows:

[0012] Step 1.1 Assume there exists a point c0 in three-dimensional space, which is mapped to the corresponding point c in two-dimensional space. i If the optical flow at point (i,j) in two-dimensional space is (u,v), then combining (i,j,u,v) gives us the optical flow points. After optimization and filtering, we obtain the optical flow set (U,V), which is the optical flow vector field.

[0013] Step 1.2, the set of velocity vectors in the crowd velocity vector field is as follows: p is the total number of all vectors;

[0014] The specific method described in step 2 is as follows:

[0015] Step 2.1: Based on the crowd motion vector field obtained in Step 1, a local vector field region is extracted using a template; by traversing the video frames, a set of local regions of the same size is obtained: Q = (Q1, Q2, ..., Q...). t ), where t is the total number of local regions divided;

[0016] Step 2.2: In the local region defined in Step 2.1, find the particles that collide with the central particle within the time interval Δt, define the collision coefficient γ, and traverse the entire local region to obtain all particle groups that can collide; wherein, the ability of particles to collide is determined by formula (1):

[0017]

[0018] in,

[0019] Let j be the velocity of the particle at position j. Let be the velocity of the particle at position i, and Let k represent the direction vector from position j to position i; according to formula (1), the following judgment can be made: when k ji When k > 0, it means that the two particles located at i and j can collide within the time interval Δt; when k ji When <0, it means that the two particles cannot collide within the time interval Δt. Finally, the collision coefficient is determined by equation (3):

[0020]

[0021] The collision relationship between particles can be determined using formula (1);

[0022] The specific method described in step 3 is as follows:

[0023] The relationship between the impulse and energy change before and after a particle collision is shown in formula (6):

[0024]

[0025] in Let be the impact force acting on the particle, and Δt be the time interval before and after the collision. FR stands for video frame rate. Let the change in velocity of the particle before and after the collision be expressed. Substituting this into formula (6), we get:

[0026]

[0027] Assuming the particle's mass is m = 1; according to the theory of perfectly elastic collisions, two particles with equal mass will exchange velocities before and after the collision, i.e.:

[0028]

[0029] i and j represent two different positions in a video frame;

[0030] Next, substituting equation (8) into equation (7) and rearranging, we obtain the elastic collision force experienced by the particle as:

[0031]

[0032] Using the scalar of collision force To describe the saliency detection map of the crowd; finally, within each small region, the elastic collision force experienced by the central particle is obtained as follows:

[0033]

[0034] The specific method described in step 4 is as follows:

[0035] Step 4.1: For each local region obtained in Step 2.1, calculate the vector standard deviation between the central particle and surrounding particles to measure the degree of difference in motion between the particles; the vector standard deviation is as follows:

[0036]

[0037] in,

[0038]

[0039] λ = (x, y), Let n be the average vector of the local region, n be the total number of particles in the local region, and S be the standard deviation of the particle vector.

[0040] Step 4.2: Calculate the standard deviation of all particles using the method from Step 4.1 to obtain a standard deviation vector diagram of the population. In this vector diagram, normalize the standard deviations and divide them into four average levels. The division strategy is as follows:

[0041]

[0042] Where S′ represents the standard deviation level;

[0043] Step 4.2: Calculate two features of the crowd particles using formulas (11) and (14), and use the standard deviation to optimize the collision force saliency map. The optimization strategy is as follows:

[0044] T=S′×F′ (15)

[0045] Where T represents the optimized saliency map, and finally, the saliency region of crowd movement is detected by T.

[0046] The beneficial effect of this invention is that it proposes a novel method for detecting the saliency of crowd movement. This method treats crowd movement as elastic particles, extracting the elastic collision force experienced by each particle from surrounding particles during collisions using the theory of elastic collisions in physics. Then, the standard deviation is used to measure the degree of difference in movement between particles, and the two are fused to construct a crowd saliency detection map, thereby detecting salient regions. The method proposed in this invention is novel, visually appealing, and computationally simple, and its effectiveness has been proven through experiments. Attached Figure Description

[0047] Figure 1This is a flowchart of the method for detecting the saliency of large-scale crowd movement based on elastic collision and vector standard deviation according to the present invention.

[0048] Figure 2 This is a diagram showing the detection results of the present invention. Detailed Implementation

[0049] The invention will now be further described with reference to the accompanying drawings.

[0050] like Figure 1 As shown, the method for detecting the saliency of large-scale crowd movement based on elastic collision and vector standard deviation includes the following steps:

[0051] Step 1: Process the input original video frame sequence using the HS optical flow method, and extract the crowd vector field by consecutive adjacent frames.

[0052] Step 2: In the vector field of the crowd motion, analyze and determine whether collisions will occur between the particles in the crowd, and save the particles that can collide for subsequent calculation of the collision force between particles.

[0053] Step 3: In a group of particles that will collide, the movement of the crowd is regarded as elastic particles, and the collision force characteristics between particles are extracted by using the elastic collision theory of physics.

[0054] Step 4: Calculate the standard deviation between the particle vector and the surrounding particles in the local area. Use the standard deviation to measure the degree of difference in the motion behavior between particles, and use the standard deviation to optimize the elastic collision force feature, thereby constructing a feature map for the saliency detection of the crowd.

[0055] Furthermore, the specific method described in step 1 is as follows:

[0056] Step 1.1: Process the input raw video frame sequence using the HS optical flow method, and extract the crowd vector field by consecutive adjacent frames;

[0057] In large-scale crowd movement scenes, each particle in the crowd has a velocity magnitude and direction. The difference in the scene position of the same particle in two consecutive frames constitutes the motion field. Assume there is a point c0 in three-dimensional space, which maps to a corresponding point c in two-dimensional space. i If the optical flow at point (i,j) in two-dimensional space is (u,v), then combining these values ​​(i,j,u,v) gives us the optical flow point. After optimization and filtering, we can obtain an optical flow set (U,V), which is the optical flow vector field. We can objectively reflect the velocity of moving particles in a video through the optical flow vector field. The motion field we often talk about is actually a two-dimensional velocity field obtained through optical flow field transformation. The motion points of all objects in the image, after projection, form the motion vector field of the optical flow field. Therefore, it can be seen that the optical flow field contains rich vector information.

[0058] Step 1.2, the set of velocity vectors in the crowd velocity vector field is as follows: p is the total number of all vectors.

[0059] Furthermore, the specific method described in step 2 is as follows:

[0060] Step 2.1: Based on the crowd motion vector field obtained in Step 1, a local vector field region is extracted using a template of size n = 9 × 9, where n is the total number of vectors in the local region. By traversing the video frames, a set of local regions of the same size, Q = (Q1, Q2, ..., Q...), can be obtained. t ), where t is the total number of local regions divided.

[0061] Step 2.2: In the local region defined in Step 2.1, find the particles that can collide with the central particle within the time interval Δt, define a collision coefficient γ, and traverse the entire local region to obtain all the particle groups that can collide. Formula (1) can be used to determine whether a collision can occur between particles.

[0062]

[0063] in,

[0064] Let j be the velocity of the particle at position j. Let be the velocity of the particle at position i, and Let represent the direction vector from position j to position i. Through equation (1), we have the following condition: when k ji When k > 0, it means that the two particles located at i and j can collide within the time interval Δt; when k ji When <0, it means that the two particles cannot collide within the time interval Δt. Finally, the collision coefficient is determined by equation (3).

[0065]

[0066] The collision relationship between particles can be determined by formula (1).

[0067] Furthermore, the specific method described in step 3 is as follows:

[0068] Step 3.1: Based on steps 1 and 2, the motion ratio of the crowd is taken as an elastic particle, and the collision between the crowd is taken as a perfectly elastic collision. Through the energy relationship before and after the perfectly elastic collision of the particles, we can obtain the energy relationship before and after the collision between the particles as shown in (4).

[0069]

[0070] Where m represents the mass of the particle. This represents the velocity of the particle before the collision. The value represents the velocity of the particle after the collision, and i and j represent two different positions in the video frame.

[0071] Step 3.2: Based on steps 1 and 2, the motion ratio of the crowd is taken as an elastic particle, and the collision between the crowd is taken as a perfectly elastic collision. Through the momentum relationship before and after the perfectly elastic collision of the particles, we can obtain the momentum relationship before and after the collision between the particles as shown in (5).

[0072]

[0073] Step 3.3: Based on steps 1 and 2, the motion ratio of the crowd is taken as an elastic particle, and the collision between the crowd is taken as a perfectly elastic collision. By the relationship between the impulse and the change in energy before and after the perfectly elastic collision of the particles, we can obtain the relationship between the impulse and the change in energy before and after the collision between the particles as shown in (6).

[0074]

[0075] in Let be the impact force acting on the particle, and Δt be the time interval before and after the collision. (FR stands for video frame rate) Let the change in velocity of the particle before and after the collision be expressed. Substituting this into formula (6), we get:

[0076]

[0077] This paper assumes the particle's mass is m = 1. According to the theory of perfectly elastic collisions, two particles with equal mass will exchange velocities before and after the collision, i.e.:

[0078]

[0079] Next, substituting equation (8) into equation (7) and rearranging, we obtain the elastic collision force experienced by the particle as:

[0080]

[0081] The present invention will use formula (10) to solve for the collision force between particles.

[0082]

[0083] This invention uses a scalar of collision force This is used to describe the saliency detection map of the crowd. Finally, within each small region, the elastic impact force experienced by the central particle can be obtained as:

[0084]

[0085] Furthermore, the specific method described in step 4 is as follows:

[0086] Step 4.1: For each local region obtained in Step 2.1, calculate the vector standard deviation between the central particle and the surrounding particles to measure the degree of difference in motion between the particles. The vector standard deviation is as follows:

[0087]

[0088] in,

[0089]

[0090] λ = (x, y), Let n be the average vector of the local region, n be the total number of particles in the local region, and S be the standard deviation of the particle vector.

[0091] Step 4.2: Calculate the standard deviation of all particles using the method from Step 4.1 to obtain a standard deviation vector diagram of the population. In this vector diagram, normalize the standard deviations and divide them into four average levels. The division strategy is as follows:

[0092]

[0093] Where S′ represents the standard deviation level.

[0094] Step 4.2: Two features of the crowd particles are calculated using formulas (11) and (14). We use the standard deviation to optimize the collision force saliency map, and the optimization strategy is as follows:

[0095] T=S′×F′ (15)

[0096] Where T represents the optimized saliency map, and finally, the saliency region of crowd movement is detected by T.

[0097] The steps to calculate the significance value of the particle at position i are as follows:

[0098] (1) Determine whether particles can collide using formulas (1) and (3);

[0099] (2) Calculate the collision force between particles using the collision theory (11), and obtain the collision force vector field of the crowd by traversing all particles;

[0100] (3) Calculate the vector standard deviation of each particle (12) and normalize it to measure the difference in motion behavior between particles, and divide the difference into 4 levels (14);

[0101] (4) Use the standard deviation to optimize the collision force vector field (14), traverse all particles, and obtain the final crowd saliency detection map.

[0102] like Figure 2 The diagram shown is an effect diagram obtained from implementing the present invention. Figure 2 Therefore, the present invention can detect the significant areas of crowd movement in a scene.

Claims

1. A method for detecting the saliency of large-scale crowd motion based on elastic collision and vector standard deviation, characterized in that: The vectors in the crowd vector field are treated as elastic colliding particles. The collision force generated between particles is calculated using the elasticity theory of physics and used as a feature to detect significant crowd motion. The collision force feature is then optimized using the vector standard deviation to finally obtain the significant crowd motion region. The steps include: Step 1: Process the input raw video frame sequence using the HS optical flow method, and extract the crowd vector field by consecutive adjacent frames; Step 2: In the vector field of crowd motion, analyze and determine whether collisions will occur between crowd particles, and save the particles that can collide for subsequent calculation of the collision force between particles. Step 3: In a group of particles that will collide, the movement of the crowd is regarded as elastic particles, and the collision force characteristics between particles are extracted by using the theory of elastic collision in physics. Step 4: Calculate the standard deviation between the particle vector and the surrounding particles in the local area. Use the standard deviation to measure the degree of difference in the motion behavior between particles, and use the standard deviation to optimize the elastic collision force feature, thereby constructing a feature map for the saliency detection of the crowd. The specific method described in step 1 is as follows: Step 1.1 Assume there exists a point c0 in three-dimensional space, which is mapped to the corresponding point c in two-dimensional space. i If the optical flow at point (i,j) in two-dimensional space is (u,v), then combining (i,j,u,v) gives us the optical flow points. After optimization and filtering, we obtain the optical flow set (U,V), which is the optical flow vector field. Step 1.2, the set of velocity vectors in the crowd velocity vector field is as follows: p is the total number of all vectors; The specific method described in step 2 is as follows: Step 2.1: Based on the crowd motion vector field obtained in Step 1, a local vector field region is extracted using a template; by traversing the video frames, a set of local regions of the same size is obtained: Q = (Q1, Q2, ..., Q...). t ), where t is the total number of local regions divided; Step 2.2: In the local region defined in Step 2.1, find the particles that collide with the central particle within the time interval Δt, define the collision coefficient γ, and traverse the entire local region to obtain all particle groups that can collide; wherein, the ability of particles to collide is determined by formula (1): in, Let j be the velocity of the particle at position j. Let be the velocity of the particle at position i, and Let k represent the direction vector from position j to position i; according to formula (1), the following judgment can be made: when k ji When k > 0, it means that the two particles located at i and j can collide within the time interval Δt; when k ji When <0, it means that the two particles cannot collide within the time interval Δt. Finally, the collision coefficient is determined by equation (3): The collision relationship between particles can be determined using formula (1); The specific method described in step 3 is as follows: The relationship between impulse and energy change before and after a particle collision is shown in (6): in Let be the impact force acting on the particle, and Δt be the time interval before and after the collision. FR stands for video frame rate. Let the change in velocity of the particle before and after the collision be expressed. Substituting this into formula (6), we get: Assuming the particle's mass is m = 1; according to the theory of perfectly elastic collisions, two particles with equal mass will exchange velocities before and after the collision, i.e.: i and j represent two different positions in a video frame; Next, substituting equation (8) into equation (7) and rearranging, we obtain the elastic collision force experienced by the particle as: Using the scalar of collision force To describe the saliency detection map of the crowd; finally, within each small region, the elastic collision force experienced by the central particle is obtained as follows: The specific method described in step 4 is as follows: Step 4.1: For each local region obtained in Step 2.1, calculate the vector standard deviation between the central particle and surrounding particles to measure the degree of difference in motion between the particles; the vector standard deviation is as follows: in, λ = (x, y), Let n be the average vector of the local region, n be the total number of particles in the local region, and S be the standard deviation of the particle vector. Step 4.2: Calculate the standard deviation of all particles using the method from Step 4.1 to obtain a standard deviation vector diagram of the population. In this vector diagram, normalize the standard deviations and divide them into four average levels. The division strategy is as follows: Where S′ represents the standard deviation level; Step 4.2: Calculate two features of the crowd particles using formulas (11) and (14), and use the standard deviation to optimize the collision force saliency map. The optimization strategy is as follows: T=S′×F′ (15) Where T represents the optimized saliency map, and finally, the saliency region of crowd movement is detected by T.

Images