Pedestrian crossing motion pattern classification method, device, computer equipment, and medium

Abstract

The present application relates to a method, device, computer equipment, storage medium and computer program product for classifying pedestrian crossing motion patterns, wherein the method includes: obtaining motion data of pedestrians crossing the street; analyzing the speed and displacement of the motion data to obtain pedestrian crossing data samples; extracting characteristic variables of pedestrian crossing motion patterns according to the pedestrian crossing data samples, and combining the characteristic variables of each pedestrian's crossing motion pattern to obtain a pedestrian sample set; clustering the pedestrian sample set based on a preset number of cluster centers using the K-means algorithm to obtain the categories of crossing patterns to which different pedestrians belong. In the whole process, the pedestrian crossing motion data is analyzed and processed to determine the characteristic variables of the pedestrian crossing motion pattern, and then cluster analysis is performed based on the K-means algorithm and the characteristic variables of the pedestrian crossing motion pattern, so as to obtain accurate pedestrian crossing motion pattern classification results.

Description

Technical Field

[0001] The present application relates to the field of intelligent recognition technology, and in particular to a method, device, computer equipment, storage medium and computer program product for classifying pedestrian crossing motion patterns. Background Art

[0002] Self-driving cars are one of the most eye-catching areas in today's science and technology. With the continuous advancement of technology, self-driving cars are gradually becoming the mainstream of future transportation.

[0003] However, pedestrian recognition technology for autonomous vehicles still faces many challenges and development opportunities. Among them, the development of pedestrian recognition and prediction technology is crucial to the safety of autonomous vehicles. Autonomous vehicles use sensors or cameras to perceive the trajectory, movement, posture and other characteristics of pedestrians on the roadside, and combine the autonomous vehicle's own dynamic information (such as speed, acceleration, etc.) to recognize the intention of specific pedestrians to cross the street and predict their trajectory, so that autonomous vehicles can better avoid pedestrians and thus improve road safety.

[0004] However, in actual applications, pedestrians' shapes, movement postures, and movement trajectories are varied. How to enable self-driving cars to accurately classify and identify pedestrians' crossing movement patterns is a technical problem that urgently needs to be solved. Summary of the invention

[0005] Based on this, it is necessary to provide an accurate pedestrian crossing motion pattern classification method, device, computer equipment, storage medium and computer program product to address the above technical problems.

[0006] In a first aspect, the present application provides a method for classifying pedestrian crossing motion patterns. The method comprises:

[0007] Obtain movement data of pedestrians crossing the street;

[0008] Analyzing the speed and displacement of the motion data to obtain pedestrian crossing data samples;

[0009] Extracting characteristic variables of pedestrian crossing motion patterns according to the pedestrian crossing data samples, and combining the characteristic variables of the pedestrian crossing motion patterns respectively to obtain a pedestrian sample set;

[0010] Based on the preset number of cluster centers, the K-means algorithm is used to cluster the pedestrian sample set to obtain the street crossing mode categories to which different pedestrians belong.

[0011] In one embodiment, obtaining the motion data of pedestrians crossing the street includes:

[0012] The human motion capture acquisition equipment is used to collect real-time data on the forward speed and forward displacement of pedestrians during the entire process of crossing the street.

[0013] In one embodiment, obtaining the motion data of pedestrians crossing the street includes:

[0014] Collect video data of the entire process of pedestrians crossing the street;

[0015] The video data is analyzed using a deep learning human posture estimation algorithm to obtain the forward speed and forward displacement data of the pedestrians throughout the entire process of crossing the street.

[0016] In one embodiment, the performing speed and displacement analysis on the motion data to obtain pedestrian crossing data samples comprises:

[0017] Extracting the x-direction speed and x-direction position data of the center of mass of the human body in each event from the motion data, and obtaining the speed and displacement data of the pedestrian in each event;

[0018] Based on the speed and displacement data of pedestrians in each event, draw a time sequence diagram of the pedestrians' crossing speed;

[0019] Generate a speed curve based on the street crossing speed sequence diagram, and extract the peak and trough positions of the speed curve by using an extreme value method;

[0020] Find the corresponding positions of the second trough and the second to last trough in the speed curve sorted by time;

[0021] The data before the second trough and the data after the second to last trough are deleted to obtain initial pedestrian crossing speed and displacement data;

[0022] The initial pedestrian crossing speed and displacement data are smoothed to obtain pedestrian crossing data samples.

[0023] In one embodiment, the characteristic variables include the average speed of pedestrians crossing the street, the maximum speed of pedestrians crossing the street, the minimum speed of pedestrians crossing the street, the speed difference of pedestrians crossing the street, the maximum acceleration of pedestrians crossing the street, the distance of pedestrians retreating, and the length of time pedestrians pause when crossing the street.

[0024] In one embodiment, the street crossing mode categories include normal walking, fast walking, slow walking, trotting, first slow then fast, walk-slow-walk, walk-stop-walk, and walk-back-walk.

[0025] In one embodiment, the characteristic variable includes the pedestrian retreat distance;

[0026] The process of extracting the pedestrian retreat distance includes:

[0027] Extracting a plurality of two adjacent frames of images from the pedestrian crossing data sample;

[0028] Calculating the displacement difference of the pedestrian in the moving direction in the extracted two adjacent frames of images;

[0029] Extract the data whose displacement difference is a negative number to form a backward data set;

[0030] The values ​​in the retreat data set are summed and the absolute values ​​are taken to obtain the pedestrian retreat distance.

[0031] In one embodiment, the characteristic variables include the duration of a pedestrian's pause when crossing the street;

[0032] The process of extracting the duration of pedestrian crossing pause includes:

[0033] Extracting pedestrian crossing speed data from the pedestrian crossing data sample;

[0034] Based on the extracted pedestrian crossing speed data, data with speeds within the range of [-0.1, 0.1] in the pedestrian crossing speed data are obtained to obtain a valid speed data set;

[0035] Counting the number of data m included in the valid speed data set;

[0036] The duration of a pedestrian's pause while crossing the street is calculated based on the number of data m and the constant t, wherein the constant t is the duration corresponding to one frame.

[0037] In a second aspect, the present application also provides a pedestrian crossing motion pattern classification device. The device comprises:

[0038] A data acquisition module, used to acquire movement data of pedestrians crossing the street;

[0039] An analysis module is used to analyze the speed and displacement of the motion data to obtain pedestrian crossing data samples;

[0040] A feature extraction module is used to extract the characteristic variables of the pedestrian crossing motion mode according to the pedestrian crossing data samples, and to combine the characteristic variables of the pedestrian crossing motion mode respectively to obtain a pedestrian sample set;

[0041] The clustering module is used to cluster the pedestrian sample set based on a preset number of cluster centers using a K-means algorithm to obtain the street crossing mode categories to which different pedestrians belong.

[0042] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0043] Obtain movement data of pedestrians crossing the street;

[0044] Analyzing the speed and displacement of the motion data to obtain pedestrian crossing data samples;

[0045] Extracting characteristic variables of pedestrian crossing motion patterns according to the pedestrian crossing data samples, and combining the characteristic variables of the pedestrian crossing motion patterns respectively to obtain a pedestrian sample set;

[0046] Based on the preset number of cluster centers, the K-means algorithm is used to cluster the pedestrian sample set to obtain the street crossing mode categories to which different pedestrians belong.

[0047] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0048] Obtain movement data of pedestrians crossing the street;

[0049] Analyzing the speed and displacement of the motion data to obtain pedestrian crossing data samples;

[0050] Extracting characteristic variables of pedestrian crossing motion patterns according to the pedestrian crossing data samples, and combining the characteristic variables of the pedestrian crossing motion patterns respectively to obtain a pedestrian sample set;

[0051] Based on the preset number of cluster centers, the K-means algorithm is used to cluster the pedestrian sample set to obtain the street crossing mode categories to which different pedestrians belong.

[0052] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0053] Obtain movement data of pedestrians crossing the street;

[0054] Analyzing the speed and displacement of the motion data to obtain pedestrian crossing data samples;

[0055] Extracting characteristic variables of pedestrian crossing motion patterns according to the pedestrian crossing data samples, and combining the characteristic variables of the pedestrian crossing motion patterns respectively to obtain a pedestrian sample set;

[0056] Based on the preset number of cluster centers, the K-means algorithm is used to cluster the pedestrian sample set to obtain the street crossing mode categories to which different pedestrians belong.

[0057] The above-mentioned pedestrian crossing motion pattern classification method, device, computer equipment, storage medium and computer program product obtain the motion data of pedestrians crossing the street; analyze the speed and displacement of the motion data to obtain pedestrian crossing data samples; extract the characteristic variables of the pedestrian crossing motion pattern according to the pedestrian crossing data samples, and combine the characteristic variables of the crossing motion pattern of each pedestrian to obtain a pedestrian sample set; based on the preset number of cluster centers, the K-means algorithm is used to cluster the pedestrian sample set to obtain the crossing mode category to which different pedestrians belong. In the whole process, the pedestrian crossing motion data is analyzed and processed to determine the characteristic variables of the pedestrian crossing motion pattern, and then cluster analysis is performed based on the K-means algorithm and the characteristic variables of the pedestrian crossing motion pattern, so as to obtain an accurate pedestrian crossing motion pattern classification result. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 A diagram of an application environment of a method for classifying movement patterns of pedestrians crossing a street according to an embodiment;

[0059] Figure 2 A schematic diagram of a flow chart of a method for classifying pedestrian crossing motion patterns according to an embodiment of the present invention;

[0060] Figure 3 A schematic diagram of a flow chart of a method for classifying movement patterns of pedestrians crossing the street according to another embodiment;

[0061] Figure 4 This is the time sequence diagram of pedestrian crossing speed;

[0062] Figure 5 This is the time series diagram of pedestrian crossing speed after smoothing;

[0063] Figure 6 It is a structural block diagram of a pedestrian crossing motion pattern classification device in one embodiment;

[0064] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0066] The pedestrian crossing motion pattern classification method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown, the terminal 102 communicates with the server 104 through a network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104, or placed on the cloud or other network servers. The terminal 102 sends a request for classification of the motion pattern of pedestrians crossing the street to the server 104, and the server 104 responds to the request. The server 104 obtains the motion data of the pedestrians crossing the street. The motion data of the pedestrians crossing the street can be collected by an external collection device and then cached in a database, and the server 104 obtains it from the database, or the terminal 102 uploads the motion data of the pedestrians crossing the street to the server 104 when sending the request, so as to request the server 104 to classify the motion pattern of the pedestrians crossing the street; after obtaining the motion data of the pedestrians crossing the street, the server 104 analyzes the speed and displacement of the motion data to obtain a pedestrian crossing data sample; according to the pedestrian crossing data sample, the characteristic variables of the pedestrian crossing motion pattern are extracted, and the characteristic variables of the crossing motion pattern of each pedestrian are respectively combined to obtain a pedestrian sample set; based on the preset number of cluster centers, the K-means algorithm is used to cluster the pedestrian sample set to obtain the crossing pattern categories to which different pedestrians belong. The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented by an independent server or a server cluster consisting of multiple servers. It should be noted that the above-mentioned pedestrian crossing motion mode classification method can also be directly applied to the terminal, that is, the terminal completes the collection of motion data and the classification of motion modes. The specific implementation process is similar to the above-mentioned process and will not be repeated here.

[0067] In one embodiment, Figure 2 As shown in FIG, a pedestrian crossing motion pattern classification method is provided, and the method is applied to Figure 1 Taking the server 104 in the example as an example, the following steps are included:

[0068] S200: Obtain movement data of pedestrians crossing the street.

[0069] The motion data of pedestrians crossing the street can be obtained by collecting the motion data of pedestrians in the pedestrian crossing scene. The pedestrian crossing scene can specifically be a city road section without traffic light control, no crosswalks, high frequency of pedestrians crossing the road, and large traffic volume. It is understandable that other typical paths can also be used, such as pedestrian crossing sections without traffic light control, no crosswalk sections with traffic light control, pedestrian crossing sections with traffic light control, rural road sections, sections within the community, school area sections, etc. (Different types of pedestrians will appear in different sections and their motion characteristics are also different. For example, there are more students in school sections, and the movement patterns of students crossing the road are very different from those of adults). When collecting data, it can be achieved specifically through human motion capture collection equipment, and then the collected data is uploaded to the server, and the server obtains the motion data of pedestrians crossing the street.

[0070] S400: Analyze the speed and displacement of the motion data to obtain pedestrian crossing data samples.

[0071] The motion data obtained in S200 is analyzed for speed and displacement. Here, the speed and displacement analysis specifically includes analyzing the x-direction (forward direction) speed and x-direction (forward direction) displacement data of the entire process of pedestrians crossing the street, and obtaining pedestrian crossing data samples. Furthermore, in the process of speed and displacement analysis, some abnormal data can be eliminated, and the speed data can be smoothed to eliminate speed fluctuations caused by the steps of pedestrians, and the processed pedestrian crossing data samples can be obtained.

[0072] S600: Extracting characteristic variables of pedestrian crossing motion patterns according to pedestrian crossing data samples, and combining the characteristic variables of the crossing motion patterns of each pedestrian respectively to obtain a pedestrian sample set.

[0073] The characteristic variables of pedestrian crossing motion patterns include the average speed of pedestrians crossing, the maximum speed of pedestrians crossing, the minimum speed of pedestrians crossing, the speed difference of pedestrians crossing, the maximum acceleration of pedestrians crossing, the distance of pedestrians retreating, and the duration of pedestrians crossing. These characteristic variables are extracted from the pedestrian crossing data samples, and then for each pedestrian, the corresponding characteristic variables of the crossing motion pattern are combined, that is, pedestrian samples corresponding to different pedestrians are obtained, and these samples are combined to form a complete pedestrian sample set.

[0074] Specifically, here, the average speed of pedestrians crossing the street, the maximum speed of pedestrians crossing the street, the minimum speed of pedestrians crossing the street, the speed difference of pedestrians crossing the street, the maximum acceleration of pedestrians crossing the street, the pedestrian retreat distance, and the length of time that pedestrians pause when crossing the street are taken as characteristic variables to characterize the pedestrian crossing motion pattern, and the crossing motion pattern characteristic variables of each pedestrian are combined into a crossing motion pattern characteristic variable vector, and all pedestrian crossing motion pattern characteristic variables are organized into a pedestrian sample set.

[0075] S800: Based on the preset number of cluster centers, a K-means algorithm is used to cluster the pedestrian sample set to obtain the crossing mode categories to which different pedestrians belong.

[0076] The preset number of cluster centers is preset, which characterizes the number of types of pedestrian crossing motion patterns, and can be set according to actual needs, for example, based on the number of required classification categories. When clustering, based on the preset number of cluster centers, the K-means algorithm is used to cluster the pedestrian sample set to obtain the crossing mode categories to which different pedestrians belong. The K-means algorithm is a widely used unsupervised learning algorithm for dividing data points into K different clusters. Here, the K-means algorithm can be applied to the pedestrian sample set to identify different crossing mode categories based on the preset number of cluster centers (i.e., the K value).

[0077] Specifically, we can first set the number of cluster centers of the K-means algorithm, which represents the various pedestrian crossing movement modes, and use the K-means algorithm to classify the pedestrian sample set S p Clustering is performed, and the pedestrian sample set S p All pedestrian samples in are divided into a corresponding number of street crossing motion mode categories, realizing the unlabeled pedestrian sample set S p Classification of pedestrian crossing movement patterns in .

[0078] After obtaining the pedestrian crossing motion pattern classification results, the pedestrian sample set S p Each pedestrian sample in is assigned a label of the street crossing mode to which it belongs, and a set of labeled pedestrian samples is obtained. , the mathematical expression of this set is as follows:

[0079]

[0080] in, Representative number is The pedestrian sample belongs to the street crossing mode category.

[0081] In actual applications, the above-mentioned street crossing mode categories include normal walking, fast walking, slow walking, trotting, first slow then fast, walk-slow-walk, walk-stop-walk, and walk-back-walk.

[0082] The above-mentioned pedestrian crossing motion pattern classification method obtains the motion data of pedestrians crossing the street; analyzes the speed and displacement of the motion data to obtain pedestrian crossing data samples; extracts the characteristic variables of the pedestrian crossing motion pattern according to the pedestrian crossing data samples, and combines the characteristic variables of the crossing motion pattern of each pedestrian to obtain a pedestrian sample set; based on the preset number of cluster centers, the K-means algorithm is used to cluster the pedestrian sample set to obtain the crossing pattern categories to which different pedestrians belong. In the whole process, the pedestrian crossing motion data is analyzed and processed to determine the characteristic variables of the pedestrian crossing motion pattern, and then cluster analysis is performed based on the K-means algorithm and the characteristic variables of the pedestrian crossing motion pattern, so as to obtain accurate pedestrian crossing motion pattern classification results.

[0083] In one embodiment, obtaining the motion data of pedestrians crossing the street includes: collecting the forward speed and forward displacement data of the pedestrians crossing the street in real time through a human motion capture device.

[0084] After selecting a pedestrian crossing scene, more than 100 pedestrians can be randomly recruited, and each pedestrian will wear a uniform human motion capture acquisition device; then, each pedestrian will independently choose the time to cross the road within the selected road section according to his or her own style and habits and the traffic conditions at the time, completing the three steps of pedestrian starting, crossing the road, and ending and stopping; finally, the x-direction (forward direction) speed and x-direction (forward direction) displacement data of the entire pedestrian crossing process are collected in real time through the human motion capture acquisition device.

[0085] Specifically, the human motion capture equipment here can include two categories: optical motion capture equipment and inertial motion capture equipment. Optical motion capture equipment uses optical lenses to capture the position information of markers fixed on the human body, and then completes the motion posture capture. Optical motion capture systems include passive and active types. The passive type uses the light board of the lens itself to emit infrared light to the marker, and the infrared light reflected by the marker is used to capture its motion trajectory; while the active type is that the marker actively emits light, and even has its own light-emitting ID code, so that the lens can directly observe and capture its movement. Inertial motion capture equipment uses inertial navigation sensors (such as IMU) to measure the acceleration, direction and tilt angle of the human body, and then calculate the speed and displacement data. According to the selected human motion capture equipment, install the corresponding marker or sensor on the pedestrian. For optical motion capture, it is necessary to ensure that the position of the marker on the human body can accurately reflect the movement of the pedestrian; for inertial motion capture, it is necessary to wear and calibrate the sensor correctly. Configure the motion capture system, including setting the camera position, adjusting the lens parameters, calibrating the sensor, etc.

[0086] In one embodiment, obtaining the motion data of pedestrians crossing the street includes: collecting video data of the entire process of pedestrians crossing the street; using a deep learning human posture estimation algorithm to analyze the video data to obtain the forward speed and forward displacement data of the entire process of pedestrians crossing the street.

[0087] Specifically, a video camera (such as a surveillance camera, a smartphone, or a professional camera) records video in the area where pedestrians cross the street, ensuring that the camera's position, angle, and resolution can clearly capture the entire process of pedestrians crossing the street. If necessary, multiple cameras can be set up to obtain a more comprehensive perspective. The collected video data is preprocessed as necessary, such as cropping, denoising, and contrast enhancement to improve the accuracy of subsequent analysis. The video data can be segmented into multiple frames or segments for batch processing and analysis. Select a deep learning algorithm suitable for human pose estimation, such as OpenPose, DeepLabCut, etc. These algorithms can usually identify the key points of the human body (such as joints) in the video frame and estimate the human body's pose. Use the deep learning human pose estimation algorithm to analyze the preprocessed video data. For each frame or segment in the video, the algorithm will output the position information of the key points of the human body. By analyzing the changes in the positions of the key points in consecutive frames, the forward speed and displacement of the pedestrian crossing the street can be estimated.

[0088] like Figure 3 As shown, in one embodiment, S400 includes:

[0089] S410: Extracting the x-direction speed and x-direction position data of the center of mass of the human body in each event from the motion data, and obtaining the speed and displacement data of the pedestrian in each event.

[0090] The motion data obtained in S200 is screened to remove the pedestrian crossing motion data with data packet loss or beyond the limited road section range. For each segment of pedestrian crossing motion data, each segment of data is divided as an independent pedestrian traffic behavior event and numbered. For example, the pedestrian traffic behavior event numbering rule can be PY001. In practical applications, motion data can be obtained through a human motion capture acquisition device. The human motion capture acquisition device can provide motion flow data described in motion anatomical terms, including characteristic information of 59 joints of the human body, including quaternions, angular velocity, acceleration, speed, position, direction, etc. The acquisition frequency is 60Hz, that is, the length of one frame is 16.6 milliseconds, and the data is output by frame, and one line of data is output for each frame. The x-direction speed and x-direction position data of the center of mass position of the human body in each event are extracted from the data output by the human motion capture acquisition device as the speed and displacement data of the pedestrian in each event.

[0091] S420: Draw a time series diagram of the crossing speed of each pedestrian based on the speed and displacement data of the pedestrian in each event.

[0092] Drawing software can be used to draw a time sequence diagram of pedestrian crossing speed based on the speed and displacement data of pedestrians in each event. Specifically, Python software can be used to draw a time sequence diagram of pedestrian crossing speed. The specific pedestrian crossing speed time sequence diagram is as follows: Figure 4 shown.

[0093] S430: Generate a speed curve based on the street crossing speed sequence diagram, and extract the peak and trough positions of the speed curve by using an extreme value method.

[0094] After obtaining the pedestrian crossing speed sequence diagram in S420, a speed curve is further generated, and the peak and trough positions of the speed curve are extracted by using an extreme value method. Each peak and trough cycle represents a step forward taken by the pedestrian.

[0095] S440: Find corresponding positions of the second trough and the second to last trough in the speed curve sorted by time.

[0096] S450: Delete the data before the second trough and the data after the second to last trough to obtain initial pedestrian crossing speed and displacement data.

[0097] Generally speaking, the movement speed of pedestrians at the start and end of the two stages is relatively slow. In order to prevent it from interfering with the calculation of the subsequent characteristic variables of the street crossing movement pattern, especially the accuracy of the average speed and the minimum speed of pedestrians crossing the street, the movement data of the first two steps and the last two steps of pedestrians crossing the street should be eliminated. Find the corresponding positions of the second trough and the second to last trough in the speed curve, and delete the data before the second trough and the data after the second to last trough to obtain the initial pedestrian crossing speed and displacement data.

[0098] S460: Smoothing the initial pedestrian crossing speed and displacement data to obtain pedestrian crossing data samples.

[0099] Smoothing is performed on the initial pedestrian crossing speed and displacement data to eliminate the speed fluctuation caused by the pedestrian's steps. The specific pedestrian crossing speed time series diagram after smoothing is as follows: Figure 5 As shown in the figure, data processing is performed for each pedestrian traffic behavior event one by one, and each processed data is used as a pedestrian crossing data sample and numbered. The pedestrian crossing data sample numbering rule is PE001.

[0100] In one embodiment, the characteristic variables include the average speed of pedestrians crossing the street, the maximum speed of pedestrians crossing the street, the minimum speed of pedestrians crossing the street, the speed difference of pedestrians crossing the street, the maximum acceleration of pedestrians crossing the street, the distance of pedestrians retreating, and the duration of pedestrians stopping when crossing the street.

[0101] In one embodiment, the characteristic variables include pedestrian retreat distance;

[0102] The process of extracting the pedestrian retreat distance includes: extracting multiple adjacent frames of images from the pedestrian crossing data sample; calculating the displacement difference of the pedestrian in the moving direction in the extracted two adjacent frames of images; extracting data with negative displacement differences to form a retreat data set; summing the values ​​in the retreat data set and taking the absolute value to obtain the pedestrian retreat distance.

[0103] In one embodiment, the characteristic variables include the length of time a pedestrian pauses when crossing the street;

[0104] The process of extracting the pause time of pedestrians crossing the street includes: extracting pedestrian crossing speed data from pedestrian crossing data samples; based on the extracted pedestrian crossing speed data, obtaining data with speeds in the range of [-0.1, 0.1] in the pedestrian crossing speed data to obtain a valid speed data set; counting the number of data m contained in the valid speed data set; calculating the pause time of pedestrians crossing the street according to the number of data m and the constant t, where the constant t is the duration corresponding to one frame.

[0105] The following will introduce the specific meaning and calculation process of the above-mentioned characteristic variables.

[0106] 1. Average speed of pedestrians crossing the street : Refers to the pedestrian crossing the street The average speed of movement in the forward direction is used to characterize the speed of pedestrians crossing the street. The calculation unit is m / s and the calculation formula is:

[0107] in, is the first frame Direction speed value, is the total number of frames of the pedestrian crossing data sample; is a positive number, indicating that the pedestrian is moving in the forward direction, i.e. direction; A negative number indicates that the pedestrian is moving in the backward direction, that is, direction.

[0108] 2. Maximum speed for pedestrians crossing the street : Refers to the pedestrian crossing the road The maximum speed of movement in the forward direction is used to characterize the speed of pedestrians crossing the street. The calculation unit is m / s and the calculation formula is:

[0109]

[0110] 3. Minimum speed for pedestrians crossing the street : Refers to the pedestrian crossing the road The minimum speed of movement in the forward direction is used to characterize the speed of pedestrians crossing the street. The calculation unit is m / s and the calculation formula is:

[0111]

[0112] 4. Pedestrians crossing the street at different speeds : Refers to the difference between the maximum speed and the minimum speed of the pedestrian when crossing the road. It is used to characterize the amplitude of the change in the speed of pedestrians crossing the street. The calculation unit is m / s. The calculation formula is:

[0113]

[0114] 5. Maximum acceleration of pedestrians crossing the street : Refers to the maximum acceleration generated by the pedestrian when crossing the road. It is used to characterize the amplitude of the change in the pedestrian's speed when crossing the street. The calculation unit is m / s². The calculation formula is:

[0115]

[0116] in, It is a constant of 0.0166, which means that one frame lasts 0.0166 seconds.

[0117] 6. Pedestrian retreat distance : Refers to the pedestrian's backward movement to avoid vehicles while crossing the road. Refers to pedestrians walking backwards ( The distance moved in the direction of the pedestrian is used to indicate whether the pedestrian has backward movement characteristics. The calculation unit is m. The calculation steps include:

[0118] 6.1. Calculate the displacement difference between two adjacent frames :

[0119]

[0120] in, is the first frame Direction displacement value, Refers to the displacement difference between two adjacent frames.

[0121] 6.2 Extraction The median is negative, forming a new data set ;

[0122] 6.3、Dataset Sum the values ​​in and take the absolute value to get the pedestrian retreat distance .

[0123]

[0124] 7. How long do pedestrians stop when crossing the street? :Pedestrians stop and wait to avoid vehicles while crossing the street. Refers to the cumulative length of time that pedestrians stop and wait, used to characterize whether pedestrians have the characteristic of stopping midway. The calculation unit is m, and the calculation steps include:

[0125] 7.1. Extracting pedestrian crossing speed data The values ​​of ;

[0126] 7.2. Calculate the data set The number of data in ;

[0127] 7.3. Calculate the duration of pedestrians stopping when crossing the street ,Right now Multiply by a constant .

[0128]

[0129] in, It is a constant of 0.0166, which means that one frame corresponds to 0.0166 seconds.

[0130] According to the above calculation formula, the above 7 street crossing motion pattern characteristic variables of each pedestrian crossing data sample are calculated, and the street crossing motion pattern characteristic variables of each sample pedestrian are combined into a street crossing motion pattern characteristic variable vector. Each street crossing motion pattern characteristic variable vector represents a pedestrian crossing motion sample. By combining all pedestrian samples, the pedestrian sample set can be obtained. , the mathematical expression of this set is as follows,

[0131]

[0132] in, represents a pedestrian crossing motion pattern characteristic variable vector composed of 7 pedestrian crossing motion pattern characteristic variables, i.e., a pedestrian crossing motion sample. represents the number of pedestrian crossing samples, that is, 306 pedestrian crossing data samples, It represents a mathematical operation, namely, transposing a matrix.

[0133] Based on the above characteristic variables, the K-means algorithm is used to cluster the pedestrian sample set based on the preset number of cluster centers to obtain the street crossing mode categories to which different pedestrians belong. Specifically, the pedestrian sample set is obtained. Then, the K-means algorithm is used to classify the pedestrian sample set Clustering is performed and the number of cluster centers of the K-means algorithm is set. This number represents the various pedestrian crossing movement modes. All pedestrian samples in the dataset are divided into a corresponding number of street crossing motion mode categories, realizing the classification of unlabeled pedestrian sample sets In the specific application example, the number of cluster centers is set to 8, and the pedestrian sample set is clustered by K-means algorithm. The 306 pedestrian crossing movements are divided into 8 categories, namely normal walking, fast walking, slow walking, trotting, slow first then fast, walk-slow-walk, walk-stop-walk, and walk-back-walk.

[0134] In general, in the above embodiments, the pedestrian crossing motion pattern classification method of the present application has the following significant effects:

[0135] 1. This application uses data from real roads and real pedestrians, which can more realistically reflect the behavior of pedestrians in real road environments and their first reactions when traffic conflicts occur.

[0136] 2. The pedestrian crossing motion pattern classification method of the present application can be embedded in the pedestrian intention recognition algorithm of an autonomous driving vehicle to provide the autonomous driving vehicle with pedestrian crossing motion category information and improve the accuracy of pedestrian intention recognition of the autonomous driving vehicle.

[0137] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0138] Based on the same inventive concept, the embodiment of the present application also provides a pedestrian crossing motion pattern classification device for implementing the above-mentioned pedestrian crossing motion pattern classification method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in one or more embodiments of the pedestrian crossing motion pattern classification device provided below can refer to the limitations of the pedestrian crossing motion pattern classification method above, and will not be repeated here.

[0139] In one embodiment, Figure 6 As shown, a pedestrian crossing motion pattern classification device is provided, comprising:

[0140] The data acquisition module 200 is used to acquire the motion data of pedestrians crossing the street;

[0141] The analysis module 400 is used to analyze the speed and displacement of the motion data to obtain pedestrian crossing data samples;

[0142] The feature extraction module 600 is used to extract the feature variables of the pedestrian crossing motion mode according to the pedestrian crossing data sample, and to combine the feature variables of the pedestrian crossing motion mode respectively to obtain a pedestrian sample set;

[0143] The clustering module 800 is used to cluster the pedestrian sample set based on a preset number of cluster centers using a K-means algorithm to obtain the crossing mode categories to which different pedestrians belong.

[0144] In one embodiment, the data acquisition module 200 is also used to collect the forward speed and forward displacement data of pedestrians in the whole process of crossing the street in real time through human motion capture and collection equipment.

[0145] In one of the embodiments, the data acquisition module 200 is also used to collect video data of the entire process of pedestrians crossing the street; the video data is analyzed using a deep learning human posture estimation algorithm to obtain the forward speed and forward displacement data of the entire process of pedestrians crossing the street.

[0146] In one embodiment, the analysis module 400 is also used to extract the x-direction speed and x-direction position data of the center of mass of the human body in each event from the motion data, and obtain the speed and displacement data of the pedestrian in each event; draw a time series diagram of the crossing speed of each pedestrian based on the speed and displacement data of the pedestrian in each event; generate a speed curve based on the crossing speed time series diagram, and use the extreme value method to extract the peak and trough positions of the speed curve; find the corresponding positions of the second trough and the second to last trough in the speed curve sorted by time; delete the data before the second trough and the data after the second to last trough to obtain the initial pedestrian crossing speed and displacement data; smooth the initial pedestrian crossing speed and displacement data to obtain a pedestrian crossing data sample.

[0147] In one embodiment, the characteristic variables include the average speed of pedestrians crossing the street, the maximum speed of pedestrians crossing the street, the minimum speed of pedestrians crossing the street, the speed difference of pedestrians crossing the street, the maximum acceleration of pedestrians crossing the street, the distance of pedestrians retreating, and the duration of pedestrians stopping when crossing the street.

[0148] In one embodiment, the street crossing mode categories include normal walking, fast walking, slow walking, trotting, first slow then fast, walk-slow-walk, walk-stop-walk, and walk-back-walk.

[0149] In one embodiment, the feature variable includes the pedestrian retreat distance; the feature extraction module 600 is also used to extract multiple adjacent frames of images from the pedestrian crossing data sample; calculate the displacement difference of the pedestrian in the direction of travel in the extracted two adjacent frames of images; extract data with negative displacement differences to form a retreat data set; sum the values ​​in the retreat data set and take the absolute value to obtain the pedestrian retreat distance.

[0150] In one embodiment, the feature variables include the duration of a pedestrian pause when crossing the street; the feature extraction module 600 is further used to extract pedestrian crossing speed data from the pedestrian crossing data sample; based on the extracted pedestrian crossing speed data, data with speeds in the range of [-0.1, 0.1] in the pedestrian crossing speed data are obtained to obtain a valid speed data set; the number of data m contained in the valid speed data set is counted; based on the number of data m and the constant t, the duration of a pedestrian pause when crossing the street is calculated, wherein the constant t is the duration corresponding to one frame.

[0151] Each module in the above-mentioned pedestrian crossing motion pattern classification device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0152] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store preset data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a pedestrian crossing motion pattern classification method is implemented.

[0153] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0154] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the above-mentioned pedestrian crossing motion pattern classification method when executing the computer program.

[0155] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned pedestrian crossing motion pattern classification method is implemented.

[0156] In one embodiment, a computer program product is provided, including a computer program, which implements the above-mentioned pedestrian crossing motion pattern classification method when executed by a processor.

[0157] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0158] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0159] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for classifying pedestrian crossing motion patterns, characterized in that: The method comprises: Obtain movement data of pedestrians crossing the street; Analyzing the speed and displacement of the motion data to obtain pedestrian crossing data samples; Extracting characteristic variables of pedestrian crossing motion patterns according to the pedestrian crossing data samples, and combining the characteristic variables of each pedestrian's crossing motion pattern to obtain a pedestrian sample set; the characteristic variables include pedestrian crossing average speed, pedestrian crossing maximum speed, pedestrian crossing minimum speed, pedestrian crossing speed difference, pedestrian crossing maximum acceleration, pedestrian retreat distance, and pedestrian crossing pause duration; Based on the preset number of cluster centers, the K-means algorithm is used to cluster the pedestrian sample set to obtain the street crossing mode categories to which different pedestrians belong; The speed and displacement analysis of the motion data to obtain pedestrian crossing data samples includes: extracting the x-direction speed and x-direction position data of the center of mass of the human body in each event from the motion data to obtain the speed and displacement data of the pedestrian in each event; drawing a crossing speed time series diagram of each pedestrian according to the speed and displacement data of the pedestrian in each event; generating a speed curve based on the crossing speed time series diagram, and extracting the peak and trough positions of the speed curve by using an extreme value method; searching for the corresponding positions of the second trough and the second to last trough in the speed curve sorted by time; deleting the data before the second trough and the data after the second to last trough to obtain initial pedestrian crossing speed and displacement data; and smoothing the initial pedestrian crossing speed and displacement data to obtain pedestrian crossing data samples.

2. The method according to claim 1, characterized in that The obtaining of movement data of pedestrians crossing the street comprises: The human motion capture acquisition equipment is used to collect real-time data on the forward speed and forward displacement of pedestrians during the entire process of crossing the street.

3. The method according to claim 1, characterized in that: The obtaining of movement data of pedestrians crossing the street comprises: Collect video data of the entire process of pedestrians crossing the street; The video data is analyzed using a deep learning human posture estimation algorithm to obtain the forward speed and forward displacement data of the pedestrians throughout the entire process of crossing the street.

4. The method according to claim 1, characterized in that: The characteristic variables include the average speed of pedestrians crossing the street, the maximum speed of pedestrians crossing the street, the minimum speed of pedestrians crossing the street, the speed difference of pedestrians crossing the street, the maximum acceleration of pedestrians crossing the street, the distance of pedestrians retreating, and the length of time pedestrians pause when crossing the street.

5. The method according to claim 1, characterized in that The street crossing mode categories include normal walking, fast walking, slow walking, trotting, first slow then fast, walk-slow-walk, walk-stop-walk, and walk-back-walk.

6. The method according to claim 1, characterized in that The process of extracting the pedestrian retreat distance includes: Extracting a plurality of two adjacent frames of images from the pedestrian crossing data sample; Calculating the displacement difference of the pedestrian in the moving direction in the extracted two adjacent frames of images; Extract the data whose displacement difference is a negative number to form a backward data set; The values ​​in the retreat data set are summed and the absolute values ​​are taken to obtain the pedestrian retreat distance.

7. The method according to claim 1, characterized in that The process of extracting the duration of pedestrian crossing pause includes: Extracting pedestrian crossing speed data from the pedestrian crossing data sample; Based on the extracted pedestrian crossing speed data, data with speeds within the range of [-0.1, 0.1] in the pedestrian crossing speed data are obtained to obtain a valid speed data set; Counting the number of data m included in the valid speed data set; The duration of a pedestrian's pause while crossing the street is calculated based on the number of data m and the constant t, wherein the constant t is the duration corresponding to one frame.

8. A pedestrian crossing motion pattern classification device, characterized in that: The device comprises: A data acquisition module, used to acquire movement data of pedestrians crossing the street; An analysis module is used to analyze the speed and displacement of the motion data to obtain pedestrian crossing data samples; A feature extraction module is used to extract the characteristic variables of the pedestrian crossing motion mode according to the pedestrian crossing data samples, and to combine the characteristic variables of the pedestrian crossing motion mode to obtain a pedestrian sample set; the average speed of pedestrians crossing, the maximum speed of pedestrians crossing, the minimum speed of pedestrians crossing, the speed difference of pedestrians crossing, the maximum acceleration of pedestrians crossing, the distance of pedestrians retreating, and the duration of pedestrians crossing pauses; A clustering module, for clustering the pedestrian sample set using a K-means algorithm based on a preset number of cluster centers to obtain the street crossing mode categories to which different pedestrians belong; The analysis module is also used to extract the x-direction speed and x-direction position data of the human body's center of mass position in each event from the motion data, and obtain the speed and displacement data of the pedestrian in each event; draw a time series diagram of the crossing speed of each pedestrian based on the speed and displacement data of the pedestrian in each event; generate a speed curve based on the crossing speed time series diagram, and use the extreme value method to extract the peak and trough positions of the speed curve; find the corresponding positions of the second trough and the second to last trough in the speed curve sorted by time; delete the data before the second trough and the data after the second to last trough to obtain initial pedestrian crossing speed and displacement data; smooth the initial pedestrian crossing speed and displacement data to obtain pedestrian crossing data samples.

9. The device according to claim 8, characterized in that The data acquisition module is also used to collect the forward speed and forward displacement data of pedestrians in the whole process of crossing the street in real time through human motion capture collection equipment.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

Images