Openpose-based multi-view crowd stride frequency identification method and system, medium and equipment

Abstract

The invention discloses a multi-view crowd stride frequency identification method and system based on Openpose, a medium and equipment. The method comprises the following steps: S1, acquiring stride frequency observation data of a crowd and processing the stride frequency observation data; s2, constructing a crowd stride frequency visual identification network, inputting the processed data into the crowd stride frequency visual identification network, and performing identification to obtain stride frequency, the step S2 specifically comprising the following steps: S21, inputting the processed data into an Openpose network, and performing identification to obtain human body posture identification points; s22, utilizing a cosine similarity matching method to correct a human body posture matrix output by Openpose network identification; and S23, taking the time sequence information of the corrected human body posture recognition points as input, and performing crowd stride frequency recognition through a peak value recognition algorithm based on a foot pressure center CoP step-down model. According to the method, efficient and accurate identification of the instantaneous stride frequency of the crowd can be realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of crowd lateral gait recognition, and specifically, relates to a multi-view crowd gait recognition method, system, medium and equipment based on Openpose. Background Art

[0002] As modern urban transportation systems are increasingly evolving towards three-dimensional and networked directions, bridges, as key nodes of three-dimensional transportation networks, play an important role in improving transportation efficiency. At the same time, as the concept of green travel has become popular, walking, as an environmentally friendly and healthy way of travel, has gradually become the first choice for people to travel short distances. This change has brought new challenges and requirements to the construction and management of urban pedestrian infrastructure. In order to meet the growing demand for pedestrian traffic, the number of large-span pedestrian bridges has continued to increase, and their structural complexity has also increased. However, the common characteristics of such bridges, such as low fundamental frequency and low damping, have greatly increased their dynamic sensitivity, which has led to frequent lateral vibration accidents, which will not only seriously threaten the lives of pedestrians, but also have an adverse impact on the traffic order and image of the city.

[0003] After in-depth research, although some scholars have preliminarily explored and verified the causes of human-induced lateral vibration, the academic community has not yet reached a full and authoritative consensus on its core essence and its mechanism of action. At present, research in this field still lacks a widely recognized theoretical model, which to a certain extent restricts the in-depth understanding and effective response to the phenomenon of human-induced lateral vibration. When studying the problem of human-induced lateral vibration on pedestrian bridges, accurately measuring the characteristics of pedestrian action is a crucial link. However, this measurement technology not only requires the use of high-precision equipment to ensure the accuracy of the measurement results, but also requires extensive pedestrian tests to collect sufficient data to support the research. There are shortcomings such as high research costs and difficulty in implementation.

[0004] With the rapid development of science and technology, human posture visual recognition technology has shown wide application potential in many fields. This technology has high-precision algorithms and advanced image processing capabilities, and the cost is relatively low. Therefore, it is of great significance to study a crowd step frequency recognition method that introduces visual recognition technology into the research field of human-induced lateral vibration of pedestrian bridges. Summary of the invention

[0005] The first purpose of the present invention is to overcome the shortcomings and deficiencies in the prior art and provide a multi-perspective crowd cadence recognition method based on Openpose, which realizes efficient and accurate recognition of the instantaneous cadence of the crowd.

[0006] The second objective of the present invention is to provide a multi-view crowd cadence recognition system based on Openpose.

[0007] The third object of the present invention is to provide a storage medium.

[0008] A fourth object of the present invention is to provide a computing device.

[0009] The purpose of the present invention is achieved through the following technical solution: a multi-view crowd cadence recognition method based on Openpose, comprising the steps of:

[0010] S1, obtaining and processing the cadence observation data of the crowd;

[0011] S2, constructing a crowd cadence visual recognition network, inputting the processed data into the crowd cadence visual recognition network, and identifying the cadence, specifically comprising the steps of:

[0012] S21, input the processed data into the Openpose network to identify the human body posture recognition points;

[0013] S22, using the cosine similarity matching method to correct the human posture matrix output by the Openpose network recognition;

[0014] S23, taking the time series information of the corrected human posture recognition points as input, and performing crowd step frequency recognition through a peak recognition algorithm based on a stepping model of the foot pressure center CoP.

[0015] Preferably, the stepping model of the foot pressure center CoP specifically includes:

[0016] The single-leg walking steps are decomposed into four periods, including the front foot lifting period, the landing period, the back foot lifting period and the forward stepping period. A coordinate system is established with the walking time as the horizontal axis and the height of the foot pressure center as the vertical axis. The vertical axis setting of the foot pressure center CoP in the landing period remains unchanged, and the vertical axis interval of the foot pressure center CoP in the front foot lifting period and the back foot lifting period is set as the CoP peak area.

[0017] Preferably, the peak identification algorithm specifically comprises the steps of:

[0018] S231, extract the foot pressure center CoP ordinate data frame by frame from the processed step frequency observation data, and output it as the displacement time course P y (t), P y represents the CoP ordinate of the foot center of pressure;

[0019] S232, identifying and separating the CoP peak area: presetting the first slope threshold K1 of the CoP displacement time course curve, using the pointer to traverse each discrete CoP ordinate data and calculate the current slope value K n , determine all slope values ​​K n Is it greater than the first slope threshold K1, that is: K n>K1,

[0020] If so, the corresponding CoP coordinate points constitute the CoP peak area, and the remaining part after separating the CoP peak area is set as the CoP approximate stable period;

[0021] S233, separating the CoP fluctuation period: presetting the second slope threshold value K2 of the CoP displacement time course curve, and K2<K1, judging all the slope values ​​K in the CoP approximate stable period n Is it greater than the second slope threshold K2, that is: K n >K2,

[0022] If yes, the corresponding CoP coordinate points constitute the CoP fluctuation period, and the remaining part after separating the CoP fluctuation period is set as the CoP stable period;

[0023] S234, take the median time of the CoP stable period as the landing time t gait ;

[0024] S235, according to all landing time t gait The distribution of step frequency is calculated.

[0025] Preferably, the method for obtaining the cadence observation data of a group of people in step S1 includes:

[0026] Cameras are respectively set up at multiple camera positions on the road section to be tested, including at least one main observation camera position and one auxiliary observation camera position; the main observation camera position is set up at the entrance of the road section to be tested, and the shooting direction of the corresponding camera is parallel to the walking direction of pedestrians; the auxiliary observation camera position is set up on the roadside of the road section to be tested, and the shooting direction of the corresponding camera is perpendicular to the walking direction of pedestrians.

[0027] Preferably, the processing in step S1 includes using the GrabCut algorithm of OpenCV to perform background removal on the pedestrian's gait observation video material file to improve the posture recognition rate of the Openpose network.

[0028] Preferably, step S21 specifically includes: obtaining 25 human body posture key points of the pedestrian through Openpose network recognition, including foot key points.

[0029] Preferably, step S22 specifically includes the steps of:

[0030] S221. Define the cosine vector of the foot pressure center CoP coordinates of pedestrian a in the i-th frame as:

[0031]

[0032] in, represents the cosine vector of the CoP coordinates of pedestrian a in the i-th frame, is the pixel position of the CoP recognition point of the human body posture matrix image of the i-th frame in the image, x i is the CoP horizontal coordinate of pedestrian a in the i-th frame, y i is the CoP ordinate of pedestrian a in the i-th frame;

[0033] S222. Calculate the cosine similarity of the previous and next frames of the CoP coordinates of each pedestrian:

[0034]

[0035] in, is the cosine similarity of the previous and next frames of pedestrian a, and Respectively represent the human body posture matrix of the i-th frame and the i+1-th frame, The closer it is to 1, the more it represents the human body posture matrix and The closer, the order of the human pose matrices of the same pedestrian is determined according to the sorting of all calculated cosine similarities.

[0036] The multi-view crowd cadence recognition system based on Openpose includes:

[0037] The data acquisition module is used to acquire and process the cadence observation data of the crowd;

[0038] A data processing module is used to construct a crowd cadence visual recognition network, input the processed data into the crowd cadence visual recognition network, and identify the cadence;

[0039] The data processing module specifically includes:

[0040] The posture recognition unit is used to input the processed data into the Openpose network to identify the human body posture recognition points;

[0041] A correction unit, used to correct the human body posture matrix output by the Openpose network recognition by using the cosine similarity matching method;

[0042] The cadence recognition unit is used to take the time series information of the corrected human posture recognition points as input, and recognize the cadence of the crowd through a peak recognition algorithm based on the stepping model of the foot pressure center CoP.

[0043] A storage medium stores a program, and when the program is executed by a processor, the multi-view crowd cadence recognition method based on Openpose described above is implemented.

[0044] A computing device includes a processor and a memory for storing a program executable by the processor. When the processor executes the program stored in the memory, the multi-view crowd cadence recognition method based on Openpose described above is implemented.

[0045] Compared with the prior art, the present invention has the following advantages and effects:

[0046] (1) The measurement is efficient, accurate and low-cost. The multi-perspective crowd cadence recognition method based on Openpose provided by the present invention first recognizes the crowd's cadence observation data by using the OpenPose human posture recognition network to achieve accurate recognition of the crowd's posture; then, the time series information of the human posture recognition point is used as input, and based on the constructed foot pressure center CoP step model, the crowd posture information is converted into crowd gait information through the peak recognition algorithm, thereby achieving efficient and accurate recognition of the crowd's instantaneous cadence, and then statistically calculating the cadence distribution, greatly reducing the measurement cost. Compared with the prior art, this method overcomes the shortcomings of needing to use high-precision equipment, high research cost, high implementation difficulty and low recognition result accuracy.

[0047] (2) The present invention constructs a foot pressure center CoP landing model based on the law of the vertical height of the pedestrian's foot pressure center CoP changing with time period. Based on the landing model, the coordinate time history corresponding to the foot key point of the human posture recognition point is input into the peak recognition algorithm. By setting the first tilt threshold and the second tilt threshold, the CoP peak area and the CoP fluctuation period are separated respectively, and finally the CoP stable period is obtained, and then the step frequency is calculated according to the step time of the CoP stable period. This method can quickly and accurately identify the landing time of each step of the pedestrian, and determine the pedestrian's instantaneous step frequency and instantaneous step width and other information.

[0048] (3) Non-contact measurement to improve accuracy. The present invention is based on visual recognition technology, including using multiple cameras to capture and record the movement images of the crowd from multiple angles, and then combining it with a crowd cadence visual recognition network to achieve non-contact measurement and recognition of the crowd cadence. Compared with traditional contact sensor measurement, this method does not require the deployment of complex equipment on pedestrians or on the ground. The equipment is easy to deploy and flexible to operate, and has little effect on the natural walking behavior of the crowd. It can more realistically reflect the walking characteristics of pedestrians and improve the accuracy of the measurement results.

[0049] (4) The method of the present invention obtains cadence observation data by setting up cameras at multiple positions, thereby ensuring the comprehensiveness and accuracy of the data. The scheme can be applied to various real bridge crowd test scenarios, and by adjusting the installation position and configuration parameters of the camera, it can effectively adapt to different crowd densities and movement speeds, ensuring the reliability of the data, and has wide applicability and strong flexibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a flowchart of the Openpose-based multi-view crowd cadence recognition method of the present invention;

[0051] Figure 2 is a human posture recognition point of the Openpose network in Embodiment 1 of the present invention;

[0052] Figure 3 The four footstep periods of the pedestrian's walking process in Embodiment 1 of the present invention;

[0053] Figure 4 Schematic diagram of the CoP peak area in Example 1 of the present invention;

[0054] Figure 5 A schematic diagram of the flow of the peak recognition algorithm of the present invention;

[0055] Figure 6 A top view of the camera setup for crowd observation in Example 1 of the present invention;

[0056] Figure 7 Schematic diagram of crowd observation and recognition effect in Example 1 of the present invention, wherein (a) is the perspective of the main observation camera, and (b) is the perspective of the auxiliary observation camera;

[0057] Figure 8 The instantaneous step frequency change diagram of the crowd in a certain period of time corresponding to the residential bridge in Example 1 of the present invention;

[0058] Fig. 9 The instantaneous step frequency distribution statistics of the crowd in the residential area and the bridge in Example 1 of the present invention;

[0059] Fig.10 The instantaneous step frequency distribution statistics of the crowd at the university bridge in embodiment 1 of the present invention;

[0060] Fig.11 This is a statistical diagram of the instantaneous cadence distribution of a crowd on an expressway bridge in Example 1 of the present invention. DETAILED DESCRIPTION

[0061] The present invention will be further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0062] Example 1

[0063] like Figure 1 The figure shows a flow chart of a multi-view crowd cadence recognition method based on Openpose, including the following steps:

[0064] S1, obtaining and processing the cadence observation data of the crowd;

[0065] S2, constructing a crowd cadence visual recognition network, inputting the processed data into the crowd cadence visual recognition network, and identifying the cadence, specifically comprising the steps of:

[0066] S21, input the processed data into the Openpose network to identify the human body posture recognition points;

[0067] S22, using the cosine similarity matching method to correct the human posture matrix output by the Openpose network recognition;

[0068] S23, taking the time series information of the corrected human posture recognition points as input, and performing crowd step frequency recognition through a peak recognition algorithm based on a stepping model of the foot pressure center CoP.

[0069] Specifically, the present invention adopts a human posture recognition convolutional neural network based on deep learning, namely the Openpose network, to construct a crowd cadence visual recognition network. The multi-perspective crowd cadence recognition method based on Openpose provided by the present invention first recognizes the crowd's cadence observation data by utilizing the OpenPose human posture recognition network to achieve accurate recognition of the crowd's posture; then, the time series information of the human posture recognition point is used as input, and based on the constructed foot pressure center CoP step model, the crowd posture information is converted into crowd gait information through the peak recognition algorithm, thereby achieving efficient and accurate recognition of the crowd's instantaneous cadence, and then the cadence distribution is statistically calculated, greatly reducing the measurement cost. Compared with the prior art, this method overcomes the shortcomings of the need to use high-precision equipment, high research cost, high implementation difficulty, and low recognition result accuracy.

[0070] The stepping model of the foot pressure center CoP specifically includes:

[0071] The single-leg walking steps are decomposed into four periods, including the front foot lifting period, the landing period, the back foot lifting period and the forward stepping period. A coordinate system is established with the walking time as the horizontal axis and the height of the foot pressure center as the vertical axis. The vertical axis setting of the foot pressure center CoP in the landing period remains unchanged, and the vertical axis interval of the foot pressure center CoP in the front foot lifting period and the back foot lifting period is set as the CoP peak area.

[0072] Specifically, Figure 3The figure shows the four footstep periods of the pedestrian walking process of the present invention. During the walking process, the vertical height of the pedestrian's CoP changes with the time period. A step of the same foot is decomposed into four periods, namely: the front foot lifting period, the landing period, the back foot lifting period and the forward stepping period. The front foot lifting period is the period from when the pedestrian lifts one foot and steps forward to when the foot touches the ground. Once it touches the ground, it enters the landing period; the back foot lifting period is the period when the foot at the back is lifted before stepping forward; when in the landing period, the pedestrian's foot is in contact with the ground, supporting the whole body, while the other foot enters the back foot lifting period, then enters the forward stepping period, and finally enters the front foot lifting period. The above process is carried out alternately by two feet. In the four periods, the CoP coordinates of the landing period remain basically unchanged, while the CoPs of the front foot lifting period, the back foot lifting period and the forward stepping period are separated from the ground, and when in the front foot lifting period and the back foot lifting period, the pedestrian's CoP is located at the highest position from the ground, that is, the CoP vertical coordinate P y Reaching a local peak is called the CoP peak area, such as Figure 4 As shown in the figure, for a single foot, the period between the two CoP peak areas when the CoP coordinate changes are relatively stable is the step-down period.

[0073] The peak identification algorithm specifically comprises the steps of:

[0074] S231, extract the foot pressure center CoP ordinate data frame by frame from the processed step frequency observation data, and output it as the displacement time course P y (t), P y represents the CoP ordinate of the foot center of pressure;

[0075] S232, identifying and separating the CoP peak area: presetting the first slope threshold K1 of the CoP displacement time course curve, using the pointer to traverse each discrete CoP ordinate data and calculate the current slope value K n , determine all slope values ​​K n Is it greater than the first slope threshold K1, that is: K n >K1,

[0076] If so, the corresponding CoP coordinate points constitute the CoP peak area, and the remaining part after separating the CoP peak area is set as the CoP approximate stable period;

[0077] S233, separating the CoP fluctuation period: presetting the second slope threshold value K2 of the CoP displacement time course curve, and K2<K1, judging all the slope values ​​K in the CoP approximate stable period n Is it greater than the second slope threshold K2, that is: K n >K2,

[0078] If yes, the corresponding CoP coordinate points constitute the CoP fluctuation period, and the remaining part after separating the CoP fluctuation period is set as the CoP stable period;

[0079] S234, take the median time of the CoP stable period as the landing time t gait ;

[0080] S235, according to all landing time t gait The distribution of step frequency is calculated.

[0081] Specifically, based on the height variation characteristics of the stepping point during pedestrian walking, that is, the vertical coordinate variation law of the foot pressure center CoP (Centre of Pressure), the present invention proposes a peak recognition algorithm to achieve accurate capture of the pedestrian's stepping position and time.

[0082] like Figure 5 The flowchart of the peak recognition algorithm of the present invention is shown. The key to pedestrian frequency recognition is to determine the pedestrian's landing point, that is, the CoP (Centre of Pressure) landing point. In the entire time series of pedestrian walking, the CoP ordinate data is first extracted through frame-by-frame analysis, and the CoP displacement time-history curve is constructed. In order to identify the CoP peak area, the first slope threshold that reflects the change in the slope of the time-history curve is set in the algorithm, and the pointer mechanism in the algorithm is used to traverse each discrete CoP ordinate point and calculate its current slope value. By comparing the calculated slope value with the preset first slope threshold, the coordinate points with slope values ​​higher than the first slope threshold can be separated, and these coordinate points correspond to the CoP peak area.

[0083] Then, according to the same principle, a second slope threshold is set that is slightly smaller than the first slope threshold, and the forward stepping period with certain fluctuations in the CoP coordinates is separated again. The time period that is not identified as the CoP peak area and the CoP fluctuation period is the pedestrian's landing period, thereby obtaining the pedestrian's landing time point each time, and then calculating its landing point position (CoP position). In this embodiment, the median time of the landing period, i.e., the CoP stable period, is defined as the landing time t gait This algorithm effectively improves the accuracy of pedestrian stepping position and time detection. In this embodiment, in order to obtain the time of the stepping point, select Figure 2 The coordinate time history of the 11th and 14th identification points in the midfoot key points is input into the peak recognition algorithm to locate the moment when the pedestrian steps down each time. Figures 9 to 11 The corresponding normal distribution statistical diagram of pedestrian cadence is obtained by statistically analyzing the discrete pedestrian cadence values ​​through data processing software. Cadence is the most important pedestrian action characteristic. The method of the present invention can identify the cadence of the crowd, thereby providing reliable data support for the study of pedestrian-induced lateral vibration of pedestrian bridges.

[0084] By using a peak recognition algorithm, the present invention can quickly and accurately identify the landing time of each step of a pedestrian and determine information such as the pedestrian's instantaneous step frequency and instantaneous step width.

[0085] The method for obtaining the cadence observation data of a group of people in step S1 includes:

[0086] Cameras are respectively set up at multiple camera positions on the road section to be tested, including at least one main observation camera position and one auxiliary observation camera position; the main observation camera position is set up at the entrance of the road section to be tested, and the shooting direction of the corresponding camera is parallel to the walking direction of pedestrians; the auxiliary observation camera position is set up on the roadside of the road section to be tested, and the shooting direction of the corresponding camera is perpendicular to the walking direction of pedestrians.

[0087] Specifically, in order to improve the efficiency of the test, the test chose to conduct the cadence observation test during the peak traffic hours of each pedestrian bridge. On the pedestrian bridge, pedestrians cross the bridge in different ways, most of which walk alone, while a small number of people (such as groups of relatives and friends) walk side by side. People walking side by side are often accompanied by behaviors such as chatting and sightseeing during the walking process, which leads to greater variability in their pace. At the same time, walking side by side can easily block the pedestrians behind, thereby affecting the camera's capture of pedestrian images, which is not conducive to the accurate recognition of the crowd's cadence. To solve this problem, the present invention adopts a multi-camera position arrangement scheme to identify the crowd's cadence from multiple angles.

[0088] Based on comprehensive consideration of the test cost and feasibility, this embodiment chooses to set up three cameras to identify the walking frequency of the crowd. Figure 6 As shown, a top view of the three-camera setup for the crowd observation test is shown. The three cameras consist of a main observation camera and two auxiliary observation cameras. The main observation camera is set up at the entrance end of the pedestrian bridge. The camera is 1.3m above the ground and faces the other entrance of the bridge. The collected crowd images are used as the main step frequency recognition material. The auxiliary camera is located on one side of the bridge guardrail, with a setup height of 0.2m. A wide-angle lens is used for shooting, and the shooting direction is perpendicular to the direction of pedestrian movement. The auxiliary observation camera can supplement the perspective when the main observation camera cannot take a full portrait (obstruction caused by pedestrians walking side by side). In practical applications, by adjusting the camera setup position and configuration parameters, the present invention can effectively adapt to different crowd densities and movement speeds to ensure the accuracy and reliability of the data.

[0089] like Figure 6 The imaging conditions and human body recognition effects of the main observation camera and the auxiliary observation camera are shown. (a) is the main observation camera perspective, from which we can see that the pedestrian is photographed from the front, his posture can be clearly displayed, and the posture recognition point is accurately located by the visual recognition network; (b) is the auxiliary camera perspective, under the side shooting of the auxiliary camera, the pedestrian's posture can also be well recognized.

[0090] The present invention verifies the method of the present invention by applying a crowd gait capture test on a real bridge. In order to ensure the representativeness and extensiveness of the crowd gait recognition data, this experiment selected pedestrian bridges around three different places as observation test sites. The selected pedestrian bridges are: a pedestrian overpass in a residential area (residential area bridge), a pedestrian bridge across campuses of a university (university bridge), and a pedestrian bridge above a fast road (fast road bridge). In order to improve the efficiency of the experiment, the experiment chooses to conduct cadence observation tests during the peak traffic period of each bridge. At the same time, the crowd observation video samples in the experiment are stored by mobile devices and edited (excluding the unmanned passage segments in the video). Figure 8 to Figure 11 The instantaneous cadence recognition results of the crowd measured on different pedestrian bridges are shown. The above recognition results reflect that the setting scheme of the present invention can effectively capture the cadence of pedestrians within an effective observation distance of 15m to 20m, ensuring the comprehensiveness and accuracy of the data.

[0091] In addition, the present invention has wide applicability and flexibility, and can cope with challenges in various environments and scenarios, including but not limited to indoor and outdoor environments, as well as high-density and low-density crowds. The present invention is scalable and integrable, and can be integrated with other sensors and systems (such as positioning systems, health monitoring systems, etc.) to achieve richer data collection and analysis. By combining visual technology with these systems, a more complete and intelligent crowd monitoring and analysis platform can be built.

[0092] The processing in step S1 includes using the GrabCut algorithm of OpenCV to remove the background of the pedestrian's cadence observation video material file to improve the posture recognition rate of the Openpose network.

[0093] Specifically, the specific steps of the above background elimination include:

[0094] (1) Load crowd video material and install necessary libraries: OpenCv (Python);

[0095] (2) Initialization parameters: Set a rectangular box (Rect) to identify the foreground object area that needs to be retained in the image. The area outside this rectangular box will be considered as the background; create a mask matrix (mask) of the same size as the source image to identify whether each pixel in the image belongs to the foreground (FGD), background (BGD), or unknown (may be foreground PR_FGD or background PR_BGD). Initialize the background model (bgdModel) and foreground model (fgdModel). If these two parameters are null, OpenCV's internal functions will automatically create models for them.

[0096] (3) Apply the GrabCut algorithm: Call the grabCut function of OpenCV and pass in parameters such as the crowd material video, mask matrix, rectangular frame, background model, foreground model, and number of iterations. The GrabCut function can use the Gaussian mixture model (GMM) to model the foreground and background based on the provided rectangular frame and initialized mask matrix, and iteratively optimize the segmentation of the foreground and background through the graph cut algorithm.

[0097] (4) Iterative optimization: If the initial segmentation result is not ideal, the segmentation result can be optimized by increasing the number of iterations, adjusting the position or size of the rectangular box, and providing additional user interaction (such as manually marking foreground and background pixels).

[0098] Step S21 specifically includes: obtaining 25 human body posture key points of the pedestrian through Openpose network recognition, including foot key points.

[0099] Specifically, the present invention uses the Openpose human posture recognition network to detect pedestrian postures under multi-person conditions. Figure 2 The 25 human posture key points that Openpose can recognize are shown. The Openpose network can analyze images or videos frame by frame and accurately output human posture key points, which play a vital role in human posture estimation. The algorithm strategy in the Openpose network first detects all human joints (i.e., identification points) based on images or videos through a deep learning model, and then accurately assigns these identification points to each corresponding human object, making the Openpose network extremely robust and able to accurately identify multi-person postures in complex scenes.

[0100] Step S22 specifically includes the following steps:

[0101] S221. Define the cosine vector of the foot pressure center CoP coordinates of pedestrian a in the i-th frame as:

[0102]

[0103] in, represents the cosine vector of the CoP coordinates of pedestrian a in the i-th frame, is the pixel position of the CoP recognition point of the human body posture matrix image of the i-th frame in the image, x i is the CoP horizontal coordinate of pedestrian a in the i-th frame, y i is the CoP ordinate of pedestrian a in the i-th frame;

[0104] S222. Calculate the cosine similarity of the previous and next frames of the CoP coordinates of each pedestrian:

[0105]

[0106] in, is the cosine similarity of the previous and next frames of pedestrian a, and Respectively represent the human body posture matrix of the i-th frame and the i+1-th frame, The closer it is to 1, the more it represents the human body posture matrix and The closer, the order of the human pose matrices of the same pedestrian is determined according to the sorting of all calculated cosine similarities.

[0107] Specifically, in this embodiment, in order to make the application principle of the cosine similarity matching method clearer, an example is given as follows: when there are three pedestrians in the picture at the same time, namely pedestrian a, pedestrian b and pedestrian c. If the cosine similarity calculation results of the CoP coordinates of pedestrian a in the i-th frame and all pedestrians in the i+1-th frame before and after the frame are respectively: It can be seen that 0.9>0.7, 0.9>0.6, that is and The cosine similarity is greater than and and The cosine similarity of and is the CoP coordinate of the same pedestrian (pedestrian a) in the previous and next frames. Similarly, it can be confirmed that and and For the same people.

[0108] Example 2

[0109] The multi-view crowd cadence recognition system based on Openpose includes:

[0110] The data acquisition module is used to acquire and process the cadence observation data of the crowd;

[0111] A data processing module is used to construct a crowd cadence visual recognition network, input the processed data into the crowd cadence visual recognition network, and identify the cadence;

[0112] The data processing module specifically includes:

[0113] The posture recognition unit is used to input the processed data into the Openpose network to identify the human body posture recognition points;

[0114] A correction unit, used to correct the human body posture matrix output by the Openpose network recognition by using the cosine similarity matching method;

[0115] The cadence recognition unit is used to take the time series information of the corrected human posture recognition points as input, and recognize the cadence of the crowd through a peak recognition algorithm based on the stepping model of the foot pressure center CoP.

[0116] Specifically, during walking, the vertical height of the pedestrian's CoP changes over time. In this embodiment, the pedestrian's single-foot walking steps are decomposed into four periods, including the front foot lifting period, the landing period, the rear foot lifting period and the forward step period. A coordinate system is established with the walking time as the horizontal coordinate and the height of the foot pressure center as the vertical coordinate. The vertical coordinate setting of the foot pressure center CoP in the landing period remains unchanged, and the vertical coordinate interval of the foot pressure center CoP in the front foot lifting period and the rear foot lifting period is set as the CoP peak area, so as to establish the landing model of the foot pressure center CoP; then, based on the above-derived CoP vertical coordinate change law, the present invention proposes a peak recognition algorithm, and the peak recognition algorithm specifically includes:

[0117] The data extraction module is used to extract the CoP ordinate data of the foot pressure center frame by frame from the processed step frequency observation data, and output it as the displacement time course P y (t), P y represents the CoP ordinate of the foot center of pressure;

[0118] The module for identifying and separating the CoP peak area is used to preset the first slope threshold K1 of the CoP displacement time curve, use the pointer to traverse each discrete CoP ordinate data and calculate the current slope value K n , determine all slope values ​​K n Is it greater than the first slope threshold K1, that is: K n >K1,

[0119] If so, the corresponding CoP coordinate points constitute the CoP peak area, and the remaining part after separating the CoP peak area is set as the CoP approximate stable period;

[0120] Separate the CoP fluctuation period module, which is used to preset the second slope threshold K2 of the CoP displacement time curve, and K2<K1, and judge all the slope values ​​K in the CoP approximate stable period n Is it greater than the second slope threshold K2, that is: K n >K2,

[0121] If yes, the corresponding CoP coordinate points constitute the CoP fluctuation period, and the remaining part after separating the CoP fluctuation period is set as the CoP stable period;

[0122] The first calculation module is used to take the median time of the CoP stable period as the landing time t gait ;

[0123] The second calculation module is used to calculate the time according to all the landing moments t gaitThe distribution of step frequency is calculated.

[0124] The present invention provides a multi-view crowd cadence recognition system based on Openpose, which makes full use of the significant advantages of the human posture recognition network OpenPose in the field of multi-person posture recognition, realizes accurate recognition of crowd postures, and successfully captures the instantaneous cadence of the crowd in combination with the peak recognition algorithm. It not only has wide applicability and can be applied to various real bridge crowd test scenarios, but also shows the advantages of efficient, accurate and convenient measurement under limited scientific research conditions.

[0125] Example 3

[0126] A storage medium stores a program, and when the program is executed by a processor, the multi-view crowd cadence recognition method based on Openpose described in Example 1 is implemented as follows:

[0127] S1, obtaining and processing the cadence observation data of the crowd;

[0128] S2, constructing a crowd cadence visual recognition network, inputting the processed data into the crowd cadence visual recognition network, and identifying the cadence, specifically comprising the steps of:

[0129] S21, input the processed data into the Openpose network to identify the human body posture recognition points;

[0130] S22, using the cosine similarity matching method to correct the human posture matrix output by the Openpose network recognition;

[0131] S23, taking the time series information of the corrected human posture recognition points as input, and performing crowd step frequency recognition through a peak recognition algorithm based on a stepping model of the foot pressure center CoP.

[0132] In the above process, the specific processing process is as described in Example 1 and will not be repeated here.

[0133] In this embodiment, the storage medium may be a magnetic disk, an optical disk, a computer memory, a read-only memory, a random access memory, a USB flash drive, a mobile hard disk, or the like.

[0134] Example 4

[0135] A computing device includes a processor and a memory for storing a program executable by the processor. When the processor executes the program stored in the memory, the multi-view crowd cadence recognition method based on Openpose described in Example 1 is implemented as follows:

[0136] S1, obtaining and processing the cadence observation data of the crowd;

[0137] S2, constructing a crowd cadence visual recognition network, inputting the processed data into the crowd cadence visual recognition network, and identifying the cadence, specifically comprising the steps of:

[0138] S21, input the processed data into the Openpose network to identify the human body posture recognition points;

[0139] S22, using the cosine similarity matching method to correct the human posture matrix output by the Openpose network recognition;

[0140] S23, taking the time series information of the corrected human posture recognition points as input, and performing crowd step frequency recognition through a peak recognition algorithm based on a stepping model of the foot pressure center CoP.

[0141] In the above process, the specific processing process is as described in Example 1 and will not be repeated here.

[0142] In this embodiment, the computing device may be a terminal device such as a desktop computer, a laptop computer, a PDA handheld terminal, a tablet computer, etc.

[0143] The above embodiments are preferred implementations of the present invention and are not intended to limit the present invention. Any other changes or other equivalent replacement methods that do not deviate from the technical solutions of the present invention are included in the protection scope of the present invention.

Claims

1. The multi-view crowd cadence recognition method based on Openpose is characterized by: Includes steps: S1, obtaining and processing the cadence observation data of the crowd; S2, constructing a crowd cadence visual recognition network, inputting the processed data into the crowd cadence visual recognition network, and identifying the cadence, specifically comprising the steps of: S21, input the processed data into the Openpose network to identify the human body posture recognition points; S22, using the cosine similarity matching method to correct the human posture matrix output by the Openpose network recognition; S23, taking the time series information of the corrected human posture recognition points as input, and performing crowd step frequency recognition through a peak recognition algorithm based on a stepping model of the foot pressure center CoP.

2. The multi-view crowd cadence recognition method based on Openpose according to claim 1 is characterized in that: The stepping model of the foot pressure center CoP specifically includes: The single-leg walking steps are decomposed into four periods, including the front foot lifting period, the landing period, the back foot lifting period and the forward stepping period. A coordinate system is established with the walking time as the horizontal axis and the height of the foot pressure center as the vertical axis. The vertical axis setting of the foot pressure center CoP in the landing period remains unchanged, and the vertical axis interval of the foot pressure center CoP in the front foot lifting period and the back foot lifting period is set as the CoP peak area.

3. The multi-view crowd cadence recognition method based on Openpose according to claim 2 is characterized in that: The peak identification algorithm specifically comprises the steps of: S231, extract the foot pressure center CoP ordinate data frame by frame from the processed step frequency observation data, and output it as the displacement time course P y (t), P y represents the CoP ordinate of the foot center of pressure; S232, identifying and separating the CoP peak area: presetting the first slope threshold K1 of the CoP displacement time course curve, using the pointer to traverse each discrete CoP ordinate data and calculate the current slope value K n , determine all slope values ​​K n Is it greater than the first slope threshold K1, that is: K n >K1, If so, the corresponding CoP coordinate points constitute the CoP peak area, and the remaining part after separating the CoP peak area is set as the CoP approximate stable period; S233, separating the CoP fluctuation period: presetting the second slope threshold value K2 of the CoP displacement time course curve, and K2<K1, judging all the slope values ​​K in the CoP approximate stable period n Is it greater than the second slope threshold K2, that is: K n >K2, If yes, the corresponding CoP coordinate points constitute the CoP fluctuation period, and the remaining part after separating the CoP fluctuation period is set as the CoP stable period; S234, take the median time of the CoP stable period as the landing time t gait ; S235, according to all landing time t gait The distribution of step frequency is calculated.

4. The multi-view crowd cadence recognition method based on Openpose according to claim 1 is characterized in that: The method for obtaining the cadence observation data of a group of people in step S1 includes: Cameras are respectively set up at multiple camera positions on the road section to be tested, including at least one main observation camera position and one auxiliary observation camera position; the main observation camera position is set up at the entrance of the road section to be tested, and the shooting direction of the corresponding camera is parallel to the walking direction of pedestrians; the auxiliary observation camera position is set up on the roadside of the road section to be tested, and the shooting direction of the corresponding camera is perpendicular to the walking direction of pedestrians.

5. The multi-view crowd cadence recognition method based on Openpose according to claim 1 is characterized in that: The processing in step S1 includes using the GrabCut algorithm of OpenCV to remove the background of the pedestrian's cadence observation video material file to improve the posture recognition rate of the Openpose network.

6. The multi-view crowd cadence recognition method based on Openpose according to claim 1 is characterized in that: Step S21 specifically includes: obtaining 25 human body posture key points of the pedestrian through Openpose network recognition, including foot key points.

7. The multi-view crowd cadence recognition method based on Openpose according to claim 1 is characterized in that: Step S22 specifically includes the following steps: S221. Define the cosine vector of the foot pressure center CoP coordinates of pedestrian a in the i-th frame as: in, represents the cosine vector of the CoP coordinates of pedestrian a in the i-th frame, is the pixel position of the CoP recognition point of the human body posture matrix image of the i-th frame in the image, x i is the CoP horizontal coordinate of pedestrian a in the i-th frame, y i is the CoP ordinate of pedestrian a in the i-th frame; S222. Calculate the cosine similarity of the previous and next frames of the CoP coordinates of each pedestrian: in, is the cosine similarity of the previous and next frames of pedestrian a, and Respectively represent the human body posture matrix of the i-th frame and the i+1-th frame, The closer it is to 1, the more it represents the human body posture matrix and The closer, the order of the human pose matrices of the same pedestrian is determined according to the sorting of all calculated cosine similarities.

8. The multi-view crowd cadence recognition system based on Openpose is characterized by: include: The data acquisition module is used to obtain and process the cadence observation data of the crowd; A data processing module is used to construct a crowd cadence visual recognition network, input the processed data into the crowd cadence visual recognition network, and identify the cadence; The data processing module specifically includes: The posture recognition unit is used to input the processed data into the Openpose network to identify the human body posture recognition points; A correction unit, used to correct the human body posture matrix output by the Openpose network recognition by using the cosine similarity matching method; The cadence recognition unit is used to take the time series information of the corrected human posture recognition points as input, and recognize the cadence of the crowd through a peak recognition algorithm based on the stepping model of the foot pressure center CoP.

9. A storage medium, characterized in that: A program is stored, and when the program is executed by a processor, the multi-view crowd cadence recognition method based on Openpose described in any one of claims 1 to 7 is implemented.

10. A computing device, characterized in that It includes a processor and a memory for storing a program executable by the processor. When the processor executes the program stored in the memory, the multi-view crowd cadence recognition method based on Openpose as described in any one of claims 1 to 7 is implemented.

Images