Pedestrian motion state detection and recognition method and device, electronic equipment, storage medium and program product

Abstract

The invention relates to a pedestrian motion state detection and recognition method and device, electronic equipment, a storage medium and a program product. The method comprises the following steps: acquiring first inertial data of a target user; filtering the first inertial data to obtain filtered first inertial data; and inputting the filtered first inertial data into a target pedestrian motion state detection and recognition model corresponding to the target user, and performing motion state detection and recognition on the target user through the target pedestrian motion state detection and recognition model to obtain a first motion state prediction result of the target user, the target pedestrian motion state detection and recognition model adopts a long-short term memory network.

Description

Technical Field

[0001] The present disclosure relates to the field of pedestrian navigation technology, and in particular to a method for detecting and identifying a pedestrian's motion state, a device for detecting and identifying a pedestrian's motion state, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] Research on pedestrian navigation technology began in the late 1990s, and it can provide positioning that meets application requirements in indoor and jungle environments where satellite signals are weak. Pedestrians have different step length parameters in different motion states such as walking and running. Therefore, in order to achieve high-precision pedestrian navigation, it is necessary to accurately detect and identify the motion state of pedestrians. At the same time, with the introduction of concepts such as human-computer interaction and smart home, the recognition of users' motion states helps robots make correct responses and provide more complete services.

[0003] In the field of pedestrian navigation, motion state detection and recognition is centered on the inertial measurement unit (IMU), supplemented by information such as barometers and magnetometers, to accurately judge the motion state of pedestrians. The motion state recognition results can provide a reference for the step size parameters in the commonly used pedestrian dead reckoning (PDR) technology, effectively improving the navigation accuracy and robustness of the pedestrian dead reckoning algorithm.

[0004] Some scholars use the wavelet transform method to analyze pedestrian inertial data in the frequency domain and classify the motion state according to the time-frequency characteristics of the inertial data. This method has a high recognition accuracy, but it is computationally intensive and complex, making it difficult to deploy on hardware and to achieve real-time output of motion states on hardware. Using machine learning methods, such as neural networks and support vector machines, the network is trained with inertial data, and the lightweight network is deployed on a single-chip microcomputer to achieve real-time gait recognition. However, this method needs to solve the problem of network generalization and is easily affected by sensor noise and abnormal pedestrian movement.

[0005] In summary, pedestrian motion state detection and recognition methods can improve pedestrian navigation accuracy and have good application prospects in various application scenarios. However, related technologies have problems such as large computational complexity and poor generalization, which make it difficult to meet the usage needs of different users and difficult to achieve real-time output. Summary of the invention

[0006] The present disclosure provides a technical solution for detecting and identifying the motion status of pedestrians.

[0007] According to one aspect of the present disclosure, a method for detecting and identifying a pedestrian's motion state is provided, comprising:

[0008] Obtain the first inertial data of the target user;

[0009] filtering the first inertial data to obtain filtered first inertial data;

[0010] The filtered first inertial data is input into a target pedestrian motion state detection and recognition model corresponding to the target user, and the motion state of the target user is detected and recognized by the target pedestrian motion state detection and recognition model to obtain a first motion state prediction result of the target user, wherein the target pedestrian motion state detection and recognition model adopts a long short-term memory network.

[0011] In a possible implementation manner, after obtaining the first motion state prediction result of the target user, the method further includes:

[0012] Performing at least one post-processing on the first motion state prediction result to obtain a corrected first motion state prediction result;

[0013] Wherein, the post-processing includes at least one of the following: mean filtering, threshold judgment processing, and continuity judgment processing.

[0014] In a possible implementation manner, obtaining the first inertial data of the target user includes:

[0015] The first inertial data of the target user is collected through an inertial measurement unit worn on the waist of the target user.

[0016] In a possible implementation manner, before obtaining the first inertial data of the target user, the method further includes:

[0017] obtaining second inertial data of the target user;

[0018] Based on the second inertial data, a target pedestrian motion state detection and recognition model is determined.

[0019] In a possible implementation manner, obtaining the second inertial data of the target user includes:

[0020] Outputting first prompt information corresponding to a first specified action set; wherein the first specified action set includes at least one specified action; the first prompt information is used to prompt: the type of the specified action in the first specified action set, and the duration and / or number of the specified action in the first specified action set;

[0021] In a process in which the target user performs a designated action in the first designated action set based on the first prompt information, second inertia data of the target user is obtained.

[0022] In a possible implementation, determining the target pedestrian motion state detection and recognition model based on the second inertial data includes:

[0023] Inputting the second inertial data into a pre-trained user identity recognition model, and identifying the identity information of the target user through the user identity recognition model;

[0024] In response to identifying the identity information of the target user, a target pedestrian motion state detection and recognition model corresponding to the target user is determined based on the identity information of the target user.

[0025] In a possible implementation, determining the target pedestrian motion state detection and recognition model based on the second inertial data includes:

[0026] In response to the target user being a new user, the target pedestrian motion state detection and recognition model is determined from a plurality of pedestrian motion state detection and recognition models corresponding to a plurality of body features based on the second inertial data.

[0027] In a possible implementation, before determining the target pedestrian motion state detection and recognition model from a plurality of pedestrian motion state detection and recognition models corresponding to a plurality of body features, the method further includes:

[0028] Pre-train multiple pedestrian motion state detection and recognition models corresponding to multiple body posture features;

[0029] Among them, for a pedestrian motion state detection and recognition model corresponding to any body feature, an inertial data training set corresponding to the body feature is used to train the pedestrian motion state detection and recognition model corresponding to the body feature.

[0030] In a possible implementation, after determining the target pedestrian motion state detection and recognition model, the method further includes:

[0031] In response to satisfying a preset model adjustment condition, outputting second prompt information corresponding to a second specified action set; wherein the second specified action set includes at least two specified actions; the second prompt information is used to prompt: the type of the specified action in the second specified action set, and the duration and / or number of the specified action in the second specified action set;

[0032] In a process in which the target user performs a designated action in the second designated action set based on the second prompt information, obtaining third inertia data of the target user;

[0033] The third inertial data is used to train the target pedestrian motion state detection and recognition model.

[0034] In a possible implementation, the preset model adjustment condition includes at least one of the following:

[0035] The target user is a new user;

[0036] Receiving a motion status type configuration request from the target user;

[0037] A sensor arrangement change request is received from the target user.

[0038] According to one aspect of the present disclosure, a device for detecting and identifying a pedestrian's motion state is provided, comprising:

[0039] A first obtaining module, used to obtain first inertia data of a target user;

[0040] A filtering module, used for filtering the first inertial data to obtain filtered first inertial data;

[0041] A motion state detection and recognition module is used to input the filtered first inertial data into a target pedestrian motion state detection and recognition model corresponding to the target user, detect and recognize the motion state of the target user through the target pedestrian motion state detection and recognition model, and obtain a first motion state prediction result of the target user, wherein the target pedestrian motion state detection and recognition model adopts a long short-term memory network.

[0042] In a possible implementation manner, the device further includes:

[0043] A post-processing module, configured to perform at least one post-processing on the first motion state prediction result to obtain a corrected first motion state prediction result;

[0044] Wherein, the post-processing includes at least one of the following: mean filtering, threshold judgment processing, and continuity judgment processing.

[0045] In a possible implementation manner, the first obtaining module is used to:

[0046] The first inertial data of the target user is collected through an inertial measurement unit worn on the waist of the target user.

[0047] In a possible implementation manner, the device further includes:

[0048] A second obtaining module, used to obtain second inertial data of the target user;

[0049] A determination module is used to determine the target pedestrian motion state detection and recognition model based on the second inertial data.

[0050] In a possible implementation manner, the second obtaining module is used to:

[0051] Outputting first prompt information corresponding to a first specified action set; wherein the first specified action set includes at least one specified action; the first prompt information is used to prompt: the type of the specified action in the first specified action set, and the duration and / or number of the specified action in the first specified action set;

[0052] In a process in which the target user performs a designated action in the first designated action set based on the first prompt information, second inertia data of the target user is obtained.

[0053] In a possible implementation manner, the determining module is used to:

[0054] Inputting the second inertial data into a pre-trained user identity recognition model, and identifying the identity information of the target user through the user identity recognition model;

[0055] In response to identifying the identity information of the target user, a target pedestrian motion state detection and recognition model corresponding to the target user is determined based on the identity information of the target user.

[0056] In a possible implementation manner, the determining module is used to:

[0057] In response to the target user being a new user, the target pedestrian motion state detection and recognition model is determined from a plurality of pedestrian motion state detection and recognition models corresponding to a plurality of body features based on the second inertial data.

[0058] In a possible implementation manner, the device further includes:

[0059] A pre-training module, used to pre-train multiple pedestrian motion state detection and recognition models corresponding to multiple body features;

[0060] Among them, for a pedestrian motion state detection and recognition model corresponding to any body feature, an inertial data training set corresponding to the body feature is used to train the pedestrian motion state detection and recognition model corresponding to the body feature.

[0061] In a possible implementation manner, the device further includes:

[0062] An output module, configured to output second prompt information corresponding to a second specified action set in response to satisfying a preset model adjustment condition; wherein the second specified action set includes at least two specified actions; and the second prompt information is used to prompt: the type of the specified action in the second specified action set, and the duration and / or number of the specified action in the second specified action set;

[0063] a third obtaining module, configured to obtain third inertia data of the target user during a process in which the target user performs a designated action in the second designated action set based on the second prompt information;

[0064] A training module is used to train the target pedestrian motion state detection and recognition model using the third inertial data.

[0065] In a possible implementation, the preset model adjustment condition includes at least one of the following:

[0066] The target user is a new user;

[0067] Receiving a motion status type configuration request from the target user;

[0068] A sensor arrangement change request is received from the target user.

[0069] According to one aspect of the present disclosure, an electronic device is provided, comprising: one or more processors; a memory for storing executable instructions; wherein the one or more processors are configured to call the executable instructions stored in the memory to execute the above method.

[0070] According to one aspect of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the above method is implemented.

[0071] According to one aspect of the present disclosure, a computer program product is provided, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in an electronic device, a processor in the electronic device executes the above method.

[0072] In the embodiment of the present disclosure, the first inertial data of the target user is obtained, the first inertial data is filtered to obtain the filtered first inertial data, and the filtered first inertial data is input into the target pedestrian motion state detection and recognition model corresponding to the target user, and the motion state of the target user is detected and recognized by the target pedestrian motion state detection and recognition model to obtain the first motion state prediction result of the target user, wherein the target pedestrian motion state detection and recognition model adopts a long short-term memory network, thereby reducing the influence of factors such as sensor noise and abnormal motion state of pedestrians, and improving the accuracy of detection and recognition of pedestrian motion state. The embodiment of the present disclosure adopts a long short-term memory network to realize the detection and recognition of pedestrian motion state, and the detection and recognition method of pedestrian motion state provided by the embodiment of the present disclosure does not need to perform wavelet transform on the user's inertial data, and the amount of calculation is small, so that the user's motion state can be provided in real time, and support can be provided for pedestrian navigation, human-computer interaction and other technologies.

[0073] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

[0074] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used to illustrate the technical solutions of the present disclosure together with the specification.

[0076] Figure 1 A flow chart of a method for detecting and identifying the motion status of a pedestrian provided by an embodiment of the present disclosure is shown.

[0077] Figure 2 A schematic diagram of the structure of a pedestrian motion state detection and recognition model provided by an embodiment of the present disclosure is shown.

[0078] Figure 3 A schematic diagram showing a confusion matrix obtained from a test set in a method for detecting and identifying the motion state of a pedestrian provided in an embodiment of the present disclosure.

[0079] Figure 4 A schematic diagram showing the first inertial data before and after filtering in the method for detecting and identifying the motion state of a pedestrian provided by an embodiment of the present disclosure.

[0080] Figure 5a A schematic diagram showing a motion state prediction result before post-processing in the method for detecting and identifying the motion state of a pedestrian provided in an embodiment of the present disclosure.

[0081] Figure 5bA schematic diagram showing a post-processed motion state prediction result in the method for detecting and identifying the motion state of a pedestrian provided in an embodiment of the present disclosure.

[0082] Figure 6a A schematic diagram showing the real-time output probability of each motion state in the method for detecting and identifying the motion state of a pedestrian provided in an embodiment of the present disclosure.

[0083] Figure 6b A schematic diagram showing a motion state judgment result after statistics are taken on the probability of each motion state in the method for detecting and identifying the motion state of a pedestrian provided in an embodiment of the present disclosure.

[0084] Figure 7 A schematic diagram showing a method for detecting and identifying the motion state of a pedestrian provided by an embodiment of the present disclosure.

[0085] Figure 8 A block diagram of a device for detecting and identifying the motion status of a pedestrian provided by an embodiment of the present disclosure is shown.

[0086] Fig. 9 A block diagram of an electronic device 1900 provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0087] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0088] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0089] The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

[0090] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0091] The disclosed embodiment provides a method for detecting and identifying the motion state of a pedestrian, by obtaining the first inertial data of a target user, filtering the first inertial data to obtain the filtered first inertial data, and inputting the filtered first inertial data into a target pedestrian motion state detection and identification model corresponding to the target user, detecting and identifying the motion state of the target user through the target pedestrian motion state detection and identification model, and obtaining a first motion state prediction result of the target user, wherein the target pedestrian motion state detection and identification model adopts a long short-term memory network, thereby reducing the influence of factors such as sensor noise and abnormal motion state of pedestrians, and improving the accuracy of detection and identification of the motion state of pedestrians. The disclosed embodiment adopts a long short-term memory network to realize the detection and identification of the motion state of pedestrians, and the detection and identification method of the motion state of pedestrians provided by the disclosed embodiment does not need to perform wavelet transform on the inertial data of the user, and the amount of calculation is small, so that the user's motion state can be provided in real time, and support can be provided for technologies such as pedestrian navigation and human-computer interaction.

[0092] The following is a detailed description of the method for detecting and identifying the motion status of pedestrians provided by the embodiments of the present disclosure in conjunction with the accompanying drawings.

[0093] Figure 1 A flow chart of a method for detecting and identifying the motion state of a pedestrian provided in an embodiment of the present disclosure is shown. In one possible implementation, the execution subject of the method for detecting and identifying the motion state of a pedestrian may be a device for detecting and identifying the motion state of a pedestrian. For example, the method for detecting and identifying the motion state of a pedestrian may be executed by a terminal device or a server or other electronic device. The terminal device may be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device or a wearable device. In some possible implementations, the method for detecting and identifying the motion state of a pedestrian may be implemented by a processor calling computer-readable instructions stored in a memory. For example, Figure 1 As shown, the method for detecting and identifying the motion state of a pedestrian includes steps S11 to S13.

[0094] In step S11 , first inertial data of a target user is obtained.

[0095] In step S12, the first inertial data is filtered to obtain filtered first inertial data.

[0096] In step S13, the filtered first inertial data is input into the target pedestrian motion state detection and recognition model corresponding to the target user, and the motion state of the target user is detected and recognized by the target pedestrian motion state detection and recognition model to obtain a first motion state prediction result of the target user, wherein the target pedestrian motion state detection and recognition model adopts a long short-term memory network.

[0097] In the embodiment of the present disclosure, the target user may represent an object detected and identified in the pedestrian motion state detection and identification system.

[0098] In the disclosed embodiment, the pedestrian motion state detection and recognition model may adopt a long short-term memory network (Long Short-Term Memory, LSTM). Figure 2 The schematic diagram of the structure of the pedestrian motion state detection and recognition model provided by the embodiment of the present disclosure is shown. By adopting the long short-term memory network as the pedestrian motion state detection and recognition model, it can effectively process time series data, capture the long-term dependency of motion state, provide high-precision motion state recognition, and have good generalization ability, and can adapt to the motion characteristics of different users.

[0099] In a possible implementation, before obtaining the first inertial data of the target user, the method further includes: obtaining second inertial data of the target user; and determining the target pedestrian motion state detection and recognition model based on the second inertial data.

[0100] In this implementation, using the collected second inertial data, a suitable pedestrian motion state detection and recognition model (ie, target pedestrian motion state detection and recognition model) can be determined or selected to detect and recognize the future motion state of the target user.

[0101] As an example of this implementation method, after obtaining the second inertial data of the target user, the second inertial data can be stored as a basis for user identity recognition. When the user uses the system again, the user identity can be quickly recognized and the corresponding pedestrian motion state detection and recognition model can be called.

[0102] For example, when a user uses the pedestrian motion state detection and recognition system for the first time, the system will ask the user to perform a series of actions, such as walking, running, etc. The inertial data (second inertial data) of these actions will be collected and used to train or select a pedestrian motion state detection and recognition model suitable for the user. Once the pedestrian motion state detection and recognition model is determined, it is stored in the system together with the user's second inertial data. When the user uses the system again, the system quickly recognizes the user by analyzing the stored data, and immediately uses the stored pedestrian motion state detection and recognition model to detect and recognize the user's motion state without the need to perform complex settings or model training processes again.

[0103] In a possible implementation, obtaining the second inertial data of the target user includes: outputting first prompt information corresponding to a first specified action set; wherein the first specified action set includes at least one specified action; the first prompt information is used to prompt: the type of the specified action in the first specified action set, and the duration and / or number of the specified action in the first specified action set; and obtaining the second inertial data of the target user during the process in which the target user performs the specified action in the first specified action set based on the first prompt information.

[0104] In this implementation, the system may define a set of specified actions, namely, a first specified action set. The first prompt information corresponding to the first specified action set may include prompt information corresponding to each specified action in the first specified action set. The first prompt information may be used to guide the target user to perform the specified action. The prompt content of the first prompt information may include the type of the specified action (e.g., walking, running, jumping, etc.), and the execution time and / or number of each specified action to ensure that the target user accurately performs it as required.

[0105] The target user can perform corresponding actions according to the received first prompt information. The execution of these actions will be captured by the inertial measurement unit to collect second inertial data. The second inertial data reflects the motion characteristics of the target user when performing the specified action.

[0106] In this implementation, the second inertial data is collected by having the target user perform a series of specified actions, and the second inertial data collected thereby helps the system to more accurately determine the target pedestrian motion state detection and recognition model corresponding to the target user.

[0107] In a possible implementation, determining the target pedestrian motion state detection and recognition model based on the second inertial data includes: inputting the second inertial data into a pre-trained user identity recognition model, and identifying the identity information of the target user through the user identity recognition model; in response to identifying the identity information of the target user, determining the target pedestrian motion state detection and recognition model corresponding to the target user based on the identity information of the target user.

[0108] In this implementation, the user identity recognition model can be used to recognize the user's identity information based on the user's inertial data, thereby determining the corresponding pedestrian motion state detection and recognition model.

[0109] As an example of this implementation, the user identity recognition model may be a lightweight network.

[0110] As an example of this implementation, the user identity recognition model can be set in the cloud.

[0111] As an example of this implementation method, if the confidence of the identity information output by the user identity recognition model is greater than or equal to the confidence threshold, it can be determined that the identity information of the target user is identified; if the confidence of the identity information output by the user identity recognition model is less than the confidence threshold, it can be determined that the identity information of the target user is not identified.

[0112] In this implementation, after identifying the identity information of the target user, the target pedestrian motion state detection and recognition model corresponding to the target user stored in the system can be directly called without re-evaluating or training the pedestrian motion state detection and recognition model.

[0113] This implementation method selects the most appropriate pedestrian motion state detection and recognition model through user identity recognition, thereby improving the accuracy and efficiency of pedestrian motion state detection and recognition. This implementation method can reduce system resource consumption and speed up response time, especially when users use the system multiple times, and can provide a more personalized and consistent experience.

[0114] In a possible implementation, the determining the target pedestrian motion state detection and recognition model based on the second inertial data includes: in response to the target user being a new user, determining the target pedestrian motion state detection and recognition model from a plurality of pedestrian motion state detection and recognition models corresponding to a plurality of body features based on the second inertial data.

[0115] In this implementation, when the target user starts to use the pedestrian motion state detection and recognition system as a new user, the second inertial data of the target user can be used to test multiple pedestrian motion state detection and recognition models corresponding to multiple body features. By comparing the prediction results of the second inertial data of the target user by multiple pedestrian motion state detection and recognition models corresponding to multiple body features, the accuracy of pedestrian motion state detection and recognition for the target user by the pedestrian motion state detection and recognition model corresponding to each body feature can be evaluated. The pedestrian motion state detection and recognition model with the highest accuracy during the evaluation process can be selected as the target pedestrian motion state detection and recognition model corresponding to the target user.

[0116] In this implementation, by selecting the pedestrian motion state detection and recognition model that best suits the body characteristics of the target user, the accuracy of pedestrian motion state detection and recognition for the target user can be improved. The system can adapt to the body characteristics of different users and provide more personalized services. In addition, compared with retraining a pedestrian motion state detection and recognition model for each user, it is more efficient to select and adjust the pedestrian motion state detection and recognition model from multiple pre-trained pedestrian motion state detection and recognition models. New users can quickly start using the pedestrian motion state detection and recognition system without a long personalized training process.

[0117] In a possible implementation, before determining the target pedestrian motion state detection and recognition model from a plurality of pedestrian motion state detection and recognition models corresponding to a plurality of body features, the method further includes: pre-training a plurality of pedestrian motion state detection and recognition models corresponding to a plurality of body features; wherein, for a pedestrian motion state detection and recognition model corresponding to any one of the body features, an inertial data training set corresponding to the body feature is used to train the pedestrian motion state detection and recognition model corresponding to the body feature.

[0118] In this implementation, multiple different pedestrian motion state detection and recognition models can be trained according to different body characteristics (such as height, weight, body shape, etc.) to improve the accuracy and generalization ability of the pedestrian motion state detection and recognition model in detecting and recognizing the motion state of users with different body characteristics.

[0119] In this implementation, inertial data of pedestrians with different body characteristics can be collected, for example, inertial data of pedestrians of different heights, weights and body shapes in various motion states (such as walking, running, climbing stairs, etc.). The collected inertial data can be labeled, that is, each motion state can be classified and labeled. The collected and labeled inertial data training set can be used to train multiple different pedestrian motion state detection and recognition models. Each pedestrian motion state detection and recognition model can be optimized for a user group with specific body characteristics. Among them, each pedestrian motion state detection and recognition model can be a long short-term memory network.

[0120] As an example of this implementation, during the training phase of the pedestrian motion state detection and recognition model, the three-axis angular velocity and three-axis acceleration collected by the inertial measurement unit can be combined into 6-dimensional data as training data.

[0121] As an example of this implementation, the inertial data in the inertial data training set may be inertial data that has been subjected to Gaussian filtering.

[0122] As an example of this implementation, after multiple pedestrian motion state detection and recognition models for users with different body features are trained, these pedestrian motion state detection and recognition models can be stored to form a pedestrian motion state detection and recognition model library.

[0123] In this implementation, by training specialized pedestrian motion state detection and recognition models for users with different body characteristics, the prediction accuracy of the pedestrian motion state detection and recognition models for specific user groups can be improved. The existence of multiple pedestrian motion state detection and recognition models enables the system to adapt to a wider range of user groups and enhances the generalization ability of the system. In addition, if the user's body characteristics change (such as weight gain or loss), the system can adapt to these changes by retraining or selecting new pedestrian motion state detection and recognition models.

[0124] In this implementation, the inertial data training set and the inertial data test set corresponding to any pedestrian motion state detection and recognition model can be randomly obtained from the collected inertial data, and the two are not repeated.

[0125] Figure 3 The schematic diagram of the confusion matrix obtained from the test set in the method for detecting and identifying the motion state of pedestrians provided by the embodiment of the present disclosure is shown. It can be calculated that the prediction accuracy of the detection and identification model of the motion state of pedestrians in the test set reaches 99.82%.

[0126] In a possible implementation, after determining the target pedestrian motion state detection and recognition model, the method also includes: in response to satisfying a preset model adjustment condition, outputting second prompt information corresponding to a second specified action set; wherein the second specified action set includes at least two specified actions; the second prompt information is used to prompt: the type of the specified action in the second specified action set, and the duration and / or number of the specified actions in the second specified action set; in the process of the target user performing the specified action in the second specified action set based on the second prompt information, obtaining third inertial data of the target user; and using the third inertial data to train the target pedestrian motion state detection and recognition model.

[0127] For example, the second prompt information may be used to prompt the target user to perform the following actions in sequence: stand for 1 minute, jump for 30 seconds, walk for 1 minute, walk backward for 1 minute, crawl for 20 seconds, and run for 30 seconds.

[0128] In this implementation, in the process of the target user performing a designated action in the second designated action set based on the second prompt information, multiple third inertial data of the target user can be obtained, and the multiple third inertial data can be used to train the target pedestrian motion state detection and recognition model.

[0129] In this implementation, when the target user performs a designated action in the second designated action set based on the second prompt information, the motion state label of the obtained third inertial data can be automatically determined according to the type of the designated action currently prompted by the second prompt information. Alternatively, the motion state label manually input by the target user can be obtained.

[0130] As an example of this implementation method, the target pedestrian motion state detection and recognition model can be trained based on the third inertial data through the cloud.

[0131] In a possible implementation, the preset model adjustment condition includes at least one of the following: the target user is a new user; a motion state type configuration request is received from the target user; a sensor arrangement change request is received from the target user.

[0132] As an example of this implementation method, in response to the target user being a new user, second prompt information corresponding to the second specified action set can be output, and in the process of the target user performing a specified action in the second specified action set based on the second prompt information, third inertial data of the target user is obtained, so as to use the third inertial data to train the target pedestrian motion state detection and recognition model.

[0133] As another example of this implementation method, in response to receiving a motion state type configuration request from a target user, second prompt information corresponding to a second specified action set can be output, and while the target user performs a specified action in the second specified action set based on the second prompt information, third inertial data of the target user can be obtained, so as to use the third inertial data to train the target pedestrian motion state detection and recognition model.

[0134] In this example, when the target user wishes to change the type and number of motion states detected and recognized by the target pedestrian motion state detection and recognition model, the second designated action set may be determined according to the type of motion state selected by the target user.

[0135] This example can configure the type and number of motion states detected and identified by the target pedestrian motion state detection and recognition model according to user needs, and can adjust the target pedestrian motion state detection and recognition model online to meet the diverse needs of users and realize user-defined network design.

[0136] As another example of this implementation method, in response to receiving a sensor layout change request from a target user, second prompt information corresponding to a second specified action set can be output, and while the target user performs a specified action in the second specified action set based on the second prompt information, third inertial data of the target user can be obtained, so as to use the third inertial data to train a target pedestrian motion state detection and recognition model.

[0137] Among them, the sensor arrangement change request can be used to request to change the waist deployment to the shoulder deployment, or to change the shoulder deployment to the waist deployment, and so on.

[0138] This example can change the sensors corresponding to the target pedestrian motion state detection and recognition model (such as the location of the sensor) according to user needs, and can adjust the target pedestrian motion state detection and recognition model online to meet the diverse needs of users and realize user-defined network design.

[0139] In one possible implementation, while the target user is using the target pedestrian motion state detection and recognition model, the system can continue to collect feedback and new inertial data from the target user, and fine-tune the target pedestrian motion state detection and recognition model to further improve the accuracy of the target pedestrian motion state detection and recognition model in detecting and identifying the target user's motion state.

[0140] In the embodiment of the present disclosure, in the process of detecting and identifying the motion state of the target user using the target pedestrian motion state detection and identification model, the first inertial data of the target user can be obtained. In the embodiment of the present disclosure, the first inertial data of the target user can be obtained by an inertial measurement unit worn on a specific part of the target user's body.

[0141] In a possible implementation, the raw inertial data collected by the inertial measurement unit worn on a specific part of the target user's body can be windowed to obtain the first inertial data. For example, the output frequency of the raw inertial data is 100 Hz, the window size is 100, and the inertial data of the target user for 1 second is collected within the window to obtain the first inertial data. The setting of the window size is sufficient to extract the pedestrian's pace information, and at the same time, a faster recognition speed can be achieved in the pace conversion stage.

[0142] In a possible implementation, the first inertial data may include: a three-axis angular velocity and a three-axis acceleration of a specific part of the target user's body. The three-axis angular velocity may refer to the speed at which the specific part of the target user's body rotates on three perpendicular axes (usually X, Y, and Z axes). The three-axis acceleration may refer to the linear acceleration of the specific part of the target user's body on three perpendicular axes, which may reflect the change in the moving speed and direction of the specific part of the target user's body.

[0143] In a possible implementation manner, obtaining the first inertial data of the target user includes: collecting the first inertial data of the target user through an inertial measurement unit worn on the waist of the target user.

[0144] In this implementation, by detecting and identifying the motion state of pedestrians based on the first inertial data of the target user's waist, the stability and accuracy of the first inertial data can be improved, the interference caused by the movement of clothing or other parts of the body can be reduced, a more comprehensive motion feature reflection can be provided, and integration and wearing can be facilitated. It is also conducive to realizing the personalization and generalization capabilities of the system, thereby providing users with accurate and real-time motion state monitoring in a variety of application scenarios.

[0145] As an example of this implementation, the first inertial data includes: a three-axis angular velocity of the waist of the target user, and a three-axis acceleration of the waist of the target user.

[0146] In another possible implementation manner, obtaining the first inertial data of the target user includes: collecting the first inertial data of the target user through an inertial measurement unit worn on a shoulder of the target user.

[0147] As an example of this implementation, the first inertial data includes: a three-axis angular velocity of a shoulder of the target user, and a three-axis acceleration of a shoulder of the target user.

[0148] In a possible implementation, the input of the target pedestrian motion state detection and recognition model may include raw inertial data collected by an inertial measurement unit, such as three-axis angular velocity and three-axis acceleration.

[0149] As an example of this implementation, the three-axis angular velocity and the three-axis acceleration can be combined into 6-dimensional data as input data for the target pedestrian motion state detection and recognition model.

[0150] In the embodiment of the present disclosure, after the first inertial data of the target user is obtained, the first inertial data may be filtered to obtain filtered first inertial data.

[0151] In a possible implementation, the first inertial data may be subjected to Gaussian filtering to obtain filtered first inertial data. The Gaussian filtering performs weighted processing on information before and after the data according to Gaussian distribution, and performs filtering processing on the data according to the Gaussian function:

[0152]

[0153] Through Gaussian filtering, Gaussian noise can be suppressed, and detailed information in the first inertial data can be retained. At the same time, the peak and valley values ​​in the first inertial data are retained, thereby better protecting the original information.

[0154] Figure 4 The schematic diagram of the first inertial data before and after filtering in the method for detecting and identifying the motion state of a pedestrian provided by an embodiment of the present disclosure is shown. Figure 4 In the example, a Gaussian filter with sigma = 1.0 is selected. Figure 4 It can be seen that Gaussian filtering can maintain the overall waveform and can well preserve high-frequency changing signals, and is suitable for use in pedestrian motion state recognition.

[0155] In the embodiment of the present disclosure, after obtaining the filtered first inertial data, the filtered first inertial data can be input into a target pedestrian motion state detection and recognition model, and the motion state of the target user is detected and recognized through the target pedestrian motion state detection and recognition model to obtain a first motion state prediction result of the target user.

[0156] For example, the target pedestrian motion state detection and recognition model can predict 8 motion states: stationary, walking, backward, running, jumping, going upstairs, going downstairs, and crawling. The target pedestrian motion state detection and recognition model can be a probability vector containing 8 elements, each element corresponding to the predicted probability of a motion state. For example, if the target pedestrian motion state detection and recognition model predicts that the current data point has the highest probability of belonging to the "walking" state, then the element value corresponding to "walking" will be the largest in this vector.

[0157] In a possible implementation, offline detection and recognition of the motion state of a pedestrian may be achieved by using the target pedestrian motion state detection and recognition model.

[0158] In a possible implementation, after obtaining the first motion state prediction result of the target user, the method further includes: performing at least one post-processing on the first motion state prediction result to obtain a corrected first motion state prediction result; wherein the post-processing includes at least one of the following: mean filtering, threshold judgment processing, and continuity judgment processing.

[0159] In this implementation, in order to improve the robustness of the motion state prediction, the first motion state prediction result may be post-processed.

[0160] As an example of this implementation, the first motion state prediction result can be mean filtered. In the pedestrian motion state detection and recognition system, when pedestrians switch between different motion states (such as walking, running, jumping), the data collected by the inertial measurement unit may change suddenly, which may cause the pedestrian motion state detection and recognition model to make mistakes in judging the current motion state. In order to reduce the misjudgment in this step conversion stage, the first motion state prediction result can be mean filtered, and the first motion state prediction result and the motion state prediction results before and after it are averaged, thereby reducing the misjudgment caused by sudden changes in pace.

[0161] As an example of this implementation, the first motion state prediction result can be subjected to a threshold judgment process. In this example, a preset threshold can be determined, and only when the prediction probability of a certain motion state by the target pedestrian motion state detection and recognition model exceeds the preset threshold, it is regarded as a reliable prediction. In this example, the highest prediction probability output by the target pedestrian motion state detection and recognition model can be compared with the threshold. If the highest prediction probability is greater than or equal to the threshold, it can be considered that the prediction of the motion state by the target pedestrian motion state detection and recognition model is reliable. If the highest prediction probability is less than the threshold, it can be considered that the motion state is not obvious, which may be due to noise, motion state conversion or unknown motion state. Among them, the unknown motion state can represent the motion state that the target pedestrian motion state detection and recognition model has not learned during the training stage.

[0162] As an example of this implementation, after the threshold judgment processing is performed on the first motion state prediction result, a continuity judgment can be performed. Among them, the continuity judgment is based on such a realistic law: in daily life, people's action states (such as walking, running, etc.) are usually stable over a period of time, and will not change frequently or suddenly. The continuity judgment assumes that once pedestrians enter a certain motion state, such as walking, they will continue to maintain this motion state for a period of time before there is a clear transition signal. If a sudden change in the motion state (such as a sudden change from walking to running) is detected in the output of the target pedestrian motion state detection and recognition model, and this change does not conform to the actual physical laws or living habits (for example, people usually do not suddenly change from walking to running at full speed without any reason), then this mutation may be wrong.

[0163] For example, suppose a person starts walking from a "stationary" state. The system will first detect the "walking" state. If, in the next few seconds, the system continues to receive data consistent with "walking" from the sensor, it will confirm the "walking" state. If data consistent with "running" is suddenly detected in the next second, the system will not change the state immediately, but will check whether this transition complies with the continuity principle. If historical data shows that the user has been "walking" and there is no sign of acceleration, the system may ignore this "running" prediction and maintain the judgment of the "walking" state.

[0164] Continuous judgment helps filter out inaccurate predictions that may be caused by sensor errors or temporary fluctuations, thereby improving the performance and reliability of the entire system.

[0165] Figure 5a A schematic diagram showing a motion state prediction result before post-processing in the method for detecting and identifying the motion state of a pedestrian provided in an embodiment of the present disclosure. Figure 5bA schematic diagram showing a post-processed motion state prediction result in the method for detecting and identifying the motion state of a pedestrian provided by an embodiment of the present disclosure. Figure 5a and Figure 5b It can be seen that post-processing can correct the erroneous judgment results in situations such as motion state transition and abnormal points.

[0166] In a possible implementation, the corrected first motion state prediction result may be written into an Excel table and output as a graph.

[0167] Figure 6a A schematic diagram showing the real-time output probability of each motion state in the method for detecting and identifying the motion state of a pedestrian provided in an embodiment of the present disclosure. Figure 6b The following is a schematic diagram showing the motion state judgment result after the probability of each motion state is statistically analyzed in the method for detecting and identifying the motion state of a pedestrian provided by an embodiment of the present disclosure. Figure 6b As shown, the probability of each motion state can be subsequently counted and the motion state judgment result can be output as a graph.

[0168] By adopting this implementation method, the influence of factors such as sensor noise and abnormal movement state of pedestrians can be further reduced, and the accuracy of detection and recognition of pedestrian movement state can be improved.

[0169] The following describes the method for detecting and identifying the motion status of pedestrians provided by the embodiment of the present disclosure through a specific application scenario.

[0170] In this application scenario, firstly, multiple pedestrian motion state detection and recognition models corresponding to multiple body shape features can be trained to form a pedestrian motion state detection and recognition model library.

[0171] When the target user needs to use the pedestrian motion state detection and recognition system, the first prompt information corresponding to the first specified action set can be output. The first specified action set can include multiple specified actions, and the first prompt information is used to prompt: the type of the specified action in the first specified action set, and the duration and / or number of the specified action in the first specified action set. In the process of the target user performing the specified action in the first specified action set based on the first prompt information, the second inertial data of the target user can be obtained.

[0172] The second inertial data can be input into a pre-trained user identity recognition model, and the identity information of the target user can be recognized through the user identity recognition model. If the confidence of the identity information output by the user identity recognition model is greater than or equal to the confidence threshold, it can be determined that the identity information of the target user is recognized; if the confidence of the identity information output by the user identity recognition model is less than the confidence threshold, it can be determined that the identity information of the target user is not recognized.

[0173] When the identity information of the target user is identified, the target pedestrian motion state detection and recognition model corresponding to the target user stored in the system can be directly called without re-evaluating or training the pedestrian motion state detection and recognition model.

[0174] In the case where the identity information of the target user is not identified (i.e., the target user is a new user), the second inertial data can be used to test multiple pedestrian motion state detection and recognition models corresponding to multiple body features in the pedestrian motion state detection and recognition model library. By comparing the prediction results of the multiple pedestrian motion state detection and recognition models corresponding to multiple body features on the second inertial data of the target user, the accuracy of the pedestrian motion state detection and recognition model corresponding to each body feature for detecting and recognizing the pedestrian motion state for the target user can be evaluated. The pedestrian motion state detection and recognition model with the highest accuracy in the evaluation process can be selected as the target pedestrian motion state detection and recognition model corresponding to the target user. In the case where the target user is a new user, the second prompt information corresponding to the second specified action set can also be output. Among them, the second specified action set can include multiple specified actions, and the second prompt information can be used to prompt: the type of the specified action in the second specified action set, and the duration and / or number of the specified action in the second specified action set. In the process of the target user performing the specified action in the second specified action set based on the second prompt information, the third inertial data of the target user can be obtained. The third inertial data can be used to train the target pedestrian motion state detection and recognition model.

[0175] In this application scenario, the target pedestrian motion state detection and recognition model can be deployed on the specified hardware. For example, the specified hardware can be STM32H743VIT6. The target pedestrian motion state detection and recognition model can be embedded and deployed through CubeMX software.

[0176] Figure 7 A schematic diagram showing a method for detecting and identifying the motion state of a pedestrian provided by an embodiment of the present disclosure is shown. Figure 7As shown, the raw inertial data collected by the inertial measurement unit worn on a specific part of the target user's body can be windowed to obtain the first inertial data. The first inertial data can be Gaussian filtered to obtain the filtered first inertial data. The filtered first inertial data can be input into the target pedestrian motion state detection and recognition model, and the motion state of the target user can be detected and identified through the target pedestrian motion state detection and recognition model to obtain the first motion state prediction result of the target user. The first motion state prediction result can be processed by outlier elimination (such as mean filtering, threshold judgment, continuity judgment) to obtain the corrected first motion state prediction result. The corrected first motion state prediction result can be written into an Excel table and output as a drawing.

[0177] In this application scenario, the target pedestrian motion state detection and recognition model can output the target user's motion state in real time at a frequency of 1 Hz.

[0178] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, the present disclosure will not repeat them. It can be understood by those skilled in the art that in the above-mentioned method of the specific implementation method, the specific execution order of each step should be determined according to its function and possible internal logic.

[0179] In addition, the present disclosure also provides a pedestrian motion state detection and identification device, an electronic device, a computer-readable storage medium, and a computer program product, all of which can be used to implement any pedestrian motion state detection and identification method provided by the present disclosure. The corresponding technical solutions and technical effects can be found in the corresponding records in the method part and will not be repeated here.

[0180] Figure 8 FIG. 1 is a block diagram of a device for detecting and identifying the motion state of a pedestrian provided by an embodiment of the present disclosure. Figure 8 As shown, the pedestrian motion state detection and identification device includes:

[0181] A first obtaining module 81, used to obtain first inertial data of a target user;

[0182] A filtering module 82, configured to filter the first inertial data to obtain filtered first inertial data;

[0183] The motion state detection and recognition module 83 is used to input the filtered first inertial data into the target pedestrian motion state detection and recognition model corresponding to the target user, detect and recognize the motion state of the target user through the target pedestrian motion state detection and recognition model, and obtain the first motion state prediction result of the target user, wherein the target pedestrian motion state detection and recognition model adopts a long short-term memory network.

[0184] In a possible implementation manner, the device further includes:

[0185] A post-processing module, configured to perform at least one post-processing on the first motion state prediction result to obtain a corrected first motion state prediction result;

[0186] Wherein, the post-processing includes at least one of the following: mean filtering, threshold judgment processing, and continuity judgment processing.

[0187] In a possible implementation, the first obtaining module 81 is used to:

[0188] The first inertial data of the target user is collected through an inertial measurement unit worn on the waist of the target user.

[0189] In a possible implementation manner, the device further includes:

[0190] A second obtaining module, used to obtain second inertial data of the target user;

[0191] A determination module is used to determine the target pedestrian motion state detection and recognition model based on the second inertial data.

[0192] In a possible implementation manner, the second obtaining module is used to:

[0193] Outputting first prompt information corresponding to a first specified action set; wherein the first specified action set includes at least one specified action; the first prompt information is used to prompt: the type of the specified action in the first specified action set, and the duration and / or number of the specified action in the first specified action set;

[0194] In a process in which the target user performs a designated action in the first designated action set based on the first prompt information, second inertia data of the target user is obtained.

[0195] In a possible implementation manner, the determining module is used to:

[0196] Inputting the second inertial data into a pre-trained user identity recognition model, and identifying the identity information of the target user through the user identity recognition model;

[0197] In response to identifying the identity information of the target user, a target pedestrian motion state detection and recognition model corresponding to the target user is determined based on the identity information of the target user.

[0198] In a possible implementation manner, the determining module is used to:

[0199] In response to the target user being a new user, the target pedestrian motion state detection and recognition model is determined from a plurality of pedestrian motion state detection and recognition models corresponding to a plurality of body features based on the second inertial data.

[0200] In a possible implementation manner, the device further includes:

[0201] A pre-training module, used to pre-train multiple pedestrian motion state detection and recognition models corresponding to multiple body features;

[0202] Among them, for a pedestrian motion state detection and recognition model corresponding to any body feature, an inertial data training set corresponding to the body feature is used to train the pedestrian motion state detection and recognition model corresponding to the body feature.

[0203] In a possible implementation manner, the device further includes:

[0204] An output module, configured to output second prompt information corresponding to a second specified action set in response to satisfying a preset model adjustment condition; wherein the second specified action set includes at least two specified actions; and the second prompt information is used to prompt: the type of the specified action in the second specified action set, and the duration and / or number of the specified action in the second specified action set;

[0205] a third obtaining module, configured to obtain third inertia data of the target user during a process in which the target user performs a designated action in the second designated action set based on the second prompt information;

[0206] A training module is used to train the target pedestrian motion state detection and recognition model using the third inertial data.

[0207] In a possible implementation, the preset model adjustment condition includes at least one of the following:

[0208] The target user is a new user;

[0209] Receiving a motion status type configuration request from the target user;

[0210] A sensor arrangement change request is received from the target user.

[0211] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. Its specific implementation and technical effects can refer to the description of the above method embodiments. For the sake of brevity, they will not be repeated here.

[0212] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium may be a non-volatile computer-readable storage medium, or may be a volatile computer-readable storage medium.

[0213] The embodiment of the present disclosure further provides a computer program, including a computer-readable code. When the computer-readable code is executed in an electronic device, a processor in the electronic device executes the above method.

[0214] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in an electronic device, a processor in the electronic device executes the above method.

[0215] An embodiment of the present disclosure also provides an electronic device, comprising: one or more processors; and a memory for storing executable instructions; wherein the one or more processors are configured to call the executable instructions stored in the memory to execute the above method.

[0216] The electronic device may be provided as a terminal, a server, or a device in other forms.

[0217] Fig. 9 1 is a block diagram of an electronic device 1900 provided in an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server. Fig. 9 , the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0218] The electronic device 1900 may also include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output interface 1958 (I / O interface). The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as a Microsoft Server operating system (Windows Server 2003). TM ), a graphical user interface operating system launched by Apple (MacOS X TM ), a multi-user, multi-process computer operating system (Unix TM), a free and open source Unix-like operating system (Linux TM ), an open source Unix-like operating system (FreeBSD TM ) or similar.

[0219] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.

[0220] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0221] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0222] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0223] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0224] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0225] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0226] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0227] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0228] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK) and the like.

[0229] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0230] If the technical solution of the embodiments of the present disclosure involves personal information, the product using the technical solution of the embodiments of the present disclosure has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of the embodiments of the present disclosure involves sensitive personal information, the product using the technical solution of the embodiments of the present disclosure has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the requirement of "explicit consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to collect his or her personal information; or on the device for processing personal information, when the personal information processing rules are notified by obvious signs / information, the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0231] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for detecting and identifying the motion state of a pedestrian, characterized in that: include: Obtain the first inertial data of the target user; filtering the first inertial data to obtain filtered first inertial data; The filtered first inertial data is input into a target pedestrian motion state detection and recognition model corresponding to the target user, and the motion state of the target user is detected and recognized by the target pedestrian motion state detection and recognition model to obtain a first motion state prediction result of the target user, wherein the target pedestrian motion state detection and recognition model adopts a long short-term memory network.

2. The method according to claim 1, characterized in that: After obtaining the first motion state prediction result of the target user, the method further includes: Performing at least one post-processing on the first motion state prediction result to obtain a corrected first motion state prediction result; Wherein, the post-processing includes at least one of the following: mean filtering, threshold judgment processing, and continuity judgment processing.

3. The method according to claim 1, characterized in that The obtaining of first inertial data of the target user includes: The first inertial data of the target user is collected through an inertial measurement unit worn on the waist of the target user.

4. The method according to claim 1, characterized in that: Before obtaining the first inertial data of the target user, the method further includes: obtaining second inertial data of the target user; Based on the second inertial data, a target pedestrian motion state detection and recognition model is determined.

5. The method according to claim 4, characterized in that The obtaining the second inertial data of the target user includes: Outputting first prompt information corresponding to a first specified action set; wherein the first specified action set includes at least one specified action; the first prompt information is used to prompt: the type of the specified action in the first specified action set, and the duration and / or number of the specified action in the first specified action set; In a process in which the target user performs a designated action in the first designated action set based on the first prompt information, second inertia data of the target user is obtained.

6. The method according to claim 4, characterized in that The step of determining the target pedestrian motion state detection and recognition model based on the second inertial data includes: Inputting the second inertial data into a pre-trained user identity recognition model, and identifying the identity information of the target user through the user identity recognition model; In response to identifying the identity information of the target user, a target pedestrian motion state detection and recognition model corresponding to the target user is determined based on the identity information of the target user.

7. The method according to claim 4, characterized in that The step of determining the target pedestrian motion state detection and recognition model based on the second inertial data includes: In response to the target user being a new user, the target pedestrian motion state detection and recognition model is determined from a plurality of pedestrian motion state detection and recognition models corresponding to a plurality of body features based on the second inertial data.

8. The method according to claim 7, characterized in that Before determining the target pedestrian motion state detection and recognition model from the plurality of pedestrian motion state detection and recognition models corresponding to the plurality of body posture features, the method further comprises: Pre-train multiple pedestrian motion state detection and recognition models corresponding to multiple body posture features; Among them, for a pedestrian motion state detection and recognition model corresponding to any body feature, an inertial data training set corresponding to the body feature is used to train the pedestrian motion state detection and recognition model corresponding to the body feature.

9. The method according to claim 4, characterized in that After determining the target pedestrian motion state detection and recognition model, the method further includes: In response to satisfying a preset model adjustment condition, outputting second prompt information corresponding to a second specified action set; wherein the second specified action set includes at least two specified actions; the second prompt information is used to prompt: the type of the specified action in the second specified action set, and the duration and / or number of the specified action in the second specified action set; In a process in which the target user performs a designated action in the second designated action set based on the second prompt information, obtaining third inertia data of the target user; The third inertial data is used to train the target pedestrian motion state detection and recognition model.

10. The method according to claim 9, characterized in that The preset model adjustment condition includes at least one of the following: The target user is a new user; Receiving a motion status type configuration request from the target user; A sensor arrangement change request is received from the target user.

11. A device for detecting and identifying the motion state of pedestrians, characterized in that: include: A first obtaining module, used to obtain first inertia data of a target user; A filtering module, used for filtering the first inertial data to obtain filtered first inertial data; A motion state detection and recognition module is used to input the filtered first inertial data into a target pedestrian motion state detection and recognition model corresponding to the target user, detect and recognize the motion state of the target user through the target pedestrian motion state detection and recognition model, and obtain a first motion state prediction result of the target user, wherein the target pedestrian motion state detection and recognition model adopts a long short-term memory network.

12. An electronic device, characterized in that: include: one or more processors; a memory for storing executable instructions; The one or more processors are configured to call the executable instructions stored in the memory to execute the method according to any one of claims 1 to 10.

13. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 10 is implemented.

14. A computer program product comprising computer readable code, or a non-computer program product carrying computer readable code A volatile computer-readable storage medium, characterized in that When the computer readable code is executed in an electronic device, The processor in the electronic device executes the method according to any one of claims 1 to 10.

Images