Method and apparatus for person recognition based on 2d lidar and inertial sensing data

Abstract

The application discloses a kind of based on 2D laser radar and inertial sensing data figure recognition method and equipment, utilize 2D laser radar real-time acquisition surrounding environment's point cloud data;By to 2D point cloud data is removed, clustering and curve fitting obtains figure dot matrix, and is positioned to figure coordinate;Utilize the identity information and inertial sensing data of figure that figure is worn by wearing type device;From point cloud data and inertial sensing data respectively extract out the behavior feature sequence of person;Two groups of behavior feature sequence are normalized, and based on the similarity of two groups of behavior feature sequence is short-term pairing;Utilize historical information to correct short-term pairing deviation, obtain stable long-term pairing and figure recognition result.The application combines the advantage that 2D laser radar is in local positioning and wearing type device is in mobile perception, realizes figure recognition in multiple scenes such as no light source, no image, privacy protection.

Description

Technical Field

[0001] This invention relates to multi-sensor information fusion, and in particular to a method and device for person recognition based on 2D lidar and inertial sensing data. Background Technology

[0002] Person recognition has wide applications and is a hot research topic in the field of artificial intelligence. It is a fundamental technology for many applications such as smart cities, smart transportation, and intelligent monitoring systems. Devices used for person recognition are generally divided into two categories: contact and contactless. Contact devices typically include wearable devices such as smartwatches, smart bracelets, chip-embedded ID cards, and smartphones, while contactless devices include cameras, LiDAR, RFID, and Wi-Fi. Among them, camera-based person recognition technology extracts facial features or other biometric characteristics by collecting images or videos, which has problems such as privacy leaks and strong dependence on lighting conditions. LiDAR, RFID, and Wi-Fi have received a lot of attention from researchers in recent years due to their low invasiveness, independence from ambient light, and ease of use. However, RFID and Wi-Fi can only identify coarse-grained human behaviors such as standing, sitting, squatting, walking, and lying down, and their robustness is low in real-world scenarios where many human behaviors are similar.

[0003] A person recognition method integrating 2D LiDAR and wearable devices can solve the above problems. First, 2D LiDAR is suitable for detection tasks under any lighting conditions. Multi-channel LiDAR can generate dense point cloud images, enabling real-time person target detection without compromising facial privacy. In target detection tasks, since laser scanning inherently includes the spatial coordinates of point clouds, LiDAR has an advantage in acquiring the pose and shape of detected persons. However, due to the lack of specific personalized information about the person, it is difficult for LiDAR to identify their identity.

[0004] Secondly, wearable devices carried by individuals, such as smartwatches, smart bracelets, work IDs, and smartphones, typically store personal identification information and are equipped with inertial sensors such as accelerometers, gyroscopes, and magnetometers. Personal information and inertial sensing data from these devices can be acquired via wireless communication, and the motion characteristics and activity trajectory of the individual can be further calculated based on the inertial sensing data. However, it is impossible to obtain information about the individual's environment or high-precision indoor location information. Therefore, it is necessary to effectively correlate 2D LiDAR data with the inertial data of the wearable device and design a stable method for extracting and matching personal features in order to locate the individual in the point cloud image scanned by LiDAR and correctly label their identity information. Compared to single-sensor-based person identification methods, multi-sensor-based methods acquire richer target information, offer a wider observation perspective, and are more robust. However, person identification methods that fuse 2D LiDAR data and inertial sensing data face the following challenges:

[0005] 1. Compared with visual image data, point cloud data based on 2D LiDAR and inertial sensing data from wearable devices cover a lower dimension of information, making it more difficult to extract human features;

[0006] 2. Occlusion of people can lead to the loss of some point cloud data, resulting in deviations in the extracted features. It is quite difficult to achieve a highly fault-tolerant and stable and reliable association pairing based on features extracted from 2D LiDAR data and inertial sensing data. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to provide a person recognition method and device based on 2D LiDAR and inertial sensing data to overcome the problems of low accuracy and poor robustness of using a single sensor for person recognition, and to ensure accurate person recognition to the greatest extent possible in various scenarios such as no light source, no image, and privacy protection.

[0008] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a person recognition method based on 2D lidar and inertial sensing data, comprising the following steps:

[0009] S1. Utilize 2D LiDAR to acquire 2D point cloud data of the environment in real time; acquire the identity information of the person and inertial sensing data;

[0010] S2. Preprocess the 2D point cloud data;

[0011] S3. Extract human behavioral feature sequences from the preprocessed 2D point cloud data and inertial sensing data respectively;

[0012] S4. Normalize the two sets of behavioral feature sequences obtained in S3, and perform short-term pairing based on the similarity between the two sets of behavioral feature sequences.

[0013] S5. Use historical information to correct short-term matching biases and obtain stable long-term matching results and person identification results.

[0014] This invention integrates 2D LiDAR and wearable devices to acquire real-time 2D point cloud data and inertial sensing data of the environment. It performs short-term pairing based on the two types of data and uses historical information to correct short-term pairing errors, thereby obtaining certain long-term pairing results and person recognition results. This invention overcomes the problems of low accuracy and poor robustness of using a single sensor for person recognition, and maximizes the protection of person recognition in various scenarios such as no light source, no image, and privacy protection.

[0015] To further improve recognition accuracy, the specific implementation process of step S2 of this invention includes:

[0016] 2.1 Delete static background data from the 2D point cloud data;

[0017] 2.2 Cluster the 2D point cloud data processed in 2.1 to identify all character point sets at time t;

[0018] 2.3 Perform curve fitting on each set of points for each character and delete the set of points that do not conform to the shape of the human body;

[0019] 2.4. For the character point set processed in 2.3, connect the point sets belonging to the same person in consecutive frames to obtain the preprocessed 2D point cloud data.

[0020] The specific implementation process of step 2.2 includes:

[0021] Select scan point P i (x i ,y i Let ) be the cluster center, and set the cluster area radius T. c Iterate through each scan point, when scan point P j coordinates (x) j ,y j When within the cluster region, i.e. The scan point P j Join the dot set When scanning point P j coordinates (x) j ,y j When the scan point P is outside the cluster area, the scan point P will be... j Add candidate points to the set until all scan points have been traversed; T c The set boundary threshold;

[0022] Select a scan point from the candidate point set as the cluster center of the next new point set, and then determine whether the remaining scan points in the candidate point set are within the cluster area of ​​the new point set.

[0023] Until all scan points are assigned to point sets, the set C of all character point sets at time t is obtained. t ;

[0024] The 2D lidar is installed in a rectangular coordinate system with the x-axis pointing east, the y-axis pointing north, and the z-axis pointing upwards. i and y i These represent the distances the scan point has translated in the x and y directions, respectively.

[0025] The specific implementation process of step 2.3 includes: using 2D LiDAR to scan the upper body of the human body, obtaining a set of human body points, and statistically calculating the range of the length of the human body point set [L]. min ,L max ], L min L is the minimum length of the human body point set. max The maximum length of the human body point set; for set C t The i-th person point set like The length of the point set is in [L] min ,L max Within the range, then Considered as a human body, select a set of points The center point is taken as the centroid of the human body, and the coordinates of this centroid represent the position of the human body, denoted as . Otherwise From C t Delete it.

[0026] The specific implementation process of step 2.4 includes: knowing the set of points C at frame t-1 and frame t. t-1 and C t For C t-1 point set in The trajectory of this point set within frame t-1 is known. Predicting point sets using a Kalman filter At the position of the next frame, i.e., the predicted point set The centroid at frame t is denoted as For C t-1 and C t Pair up all points in the set, based on and The cost matrix is ​​constructed using the Euclidean distance; based on the cost matrix, the Hungarian algorithm is used to calculate C. t-1 and C t Correlation between midpoint sets; if and The one with the highest correlation will be Add the trajectory to obtain the time point set of frame t. trajectory in,

[0027] The specific implementation process of step S3 includes:

[0028] The nearest trajectory of a person is obtained from the preprocessed 2D point cloud data, and the first behavioral feature sequence of the person is extracted. This first behavioral feature sequence includes: movement features, orientation features, acceleration features, and turning features. The nearest trajectory is then extracted. The acquisition process includes: setting For the time point set of frame t The set of trajectories, if If the number of midpoint sets is not less than m, then select the m point sets closest to the trajectory to form a sequence. Otherwise, form a sequence from all points in the trajectory. And fill in the empty point set with empty coordinates until... The number of midpoint sets is m;

[0029] The inertial sensing data of a person is obtained from the wearable device, and the second behavioral feature sequence of the person is extracted from the inertial sensing data. The second behavioral feature sequence includes: movement feature, orientation feature, acceleration feature, and turning feature.

[0030] In the first behavioral feature sequence:

[0031] The process of obtaining movement and orientation features includes: using the point set at time t-1 A square region is defined with the midpoint as the location. This square region consists of 9 nodes with sides of length T. s The data is composed of squares, with the center of the central square as the starting point, and the direction from that starting point toward the midpoints of the other 8 squares as the orientation. A value is assigned to each square; the point set in the preprocessed 2D point cloud data is then determined. For the square in which the point set is located, set the movement and orientation features of the point set to the corresponding values ​​of the square.

[0032] The process of obtaining steering features includes: setting For the time point set of frame t movement vector With the time point set of frame t-1 movement vector The included angle, that is Then the time point set of frame t The turning characteristic value is the point set The cumulative change in the angle between all adjacent motion vectors up to frame t, i.e. in

[0033] In the second behavioral feature sequence:

[0034] The motion feature determination process includes: acquiring wearable devices The data from the accelerometer, gyroscope, and magnetometer are respectively denoted as W.s3; If it does not meet the peak detection standard, then Setting it to 0 indicates the character is in a stopped state; otherwise, the gyroscope is used. and magnetometer Calculate quaternions, calculate The azimuth angle is calculated, and the average azimuth angle of m azimuth angles is used as the azimuth angle value at time t. Based on the square region, the direction of movement is determined. Set to a value from 1 to 8; where the peak detection standard is: SW represents the set sliding window size;

[0035] The orientation feature determination process includes: setting the orientation feature as the direction of the square region whose azimuth value at time t is closest to;

[0036] The process of determining acceleration characteristics includes: [determining the acceleration characteristics...] Normalization is divided into 4 levels, represented by 0, 1, 2, and 3 respectively, where 0 represents the smallest eigenvalue;

[0037] The steering feature determination process includes: in wearable devices A movement vector is obtained between every two adjacent data points. Let... Wearable device at frame t movement vector With the (t-1)th frame movement vector The included angle, that is Then at frame t The steering characteristic value is The cumulative change in the angle between all adjacent motion vectors at the time of frame arrival, i.e. in

[0038] The feature extraction method designed in this invention can extract relatively fine-grained behavioral features, covering the movement, orientation, acceleration, and turning features of a person that change over time. By forming a fine-grained sequence of person behavior, it can help improve the accuracy of person identification.

[0039] The implementation process of step S4 includes:

[0040] The similarity between two sets of behavioral feature sequences is calculated using the dynamic time warping method. These represent the two extracted sets of behavioral feature sequences;

[0041] A similarity matrix S[1:I,1:J] is obtained for all character dot matrix and wearable device, where I and J represent the similarity between the dot matrix and wearable device at time interval T. s The number of people detected from laser and inertial data is... For each wearable device The confidence level of the device pairing is calculated using the following variance formula. Where S[1:I,J] represents the character dot matrix. and The similarity sequence, i.e., S[1:I,J]=[S[i,J]],i∈[1,I];

[0042] Select the wearable device with the highest confidence level. In the similarity matrix S[1:I,1:J], select the row with the largest value in column j, and select the set of points represented by that row to form a pair. Add the pairing result to the short-term pairing result. Then, delete row i and column j from the similarity matrix S[1:I,1:J], and select the next wearable device with the highest confidence level for pairing in sequence until all pairings are completed, obtaining all short-term pairing results P. short (t).

[0043] The short-term pairing method designed in this invention mainly uses data from wearable devices with stable signals. By calculating confidence, it selects the human point array with the highest similarity, which can effectively overcome the problem of missing point cloud data caused by human occlusion.

[0044] The implementation process of step S5 includes: letting the long-term pairing result within the previous time interval t-1 be... Among them, group Represents a point set With wearable devices Pairing, and their long-term pairing similarity score is For the following three cases, calculate the long-term pairing result P within the current time interval t. long (t): Represents a point set With wearable devices Similarity;

[0045] When the results of short-term pairing and long-term pairing are consistent, i.e. In short-term pairing P short In (t), In long-term pairing P long In (t-1), and Then Add to long-term paired results P iong In (t), where The calculation formula is: α is the weighting parameter; Represents two sets of behavioral feature sequences and Similarity;

[0046] When the results of short-term pairing and long-term pairing are inconsistent, i.e., if In short-term pairing P short In (t), In long-term pairing P long In (t-1), when hour, like Then Add to P long (t) in; like Then Add to P long (t) in;

[0047] When a new pairing occurs in a short-term pairing, i.e., if Appearing in short-term pairing P short (t) in, but It did not appear in long-term paired P long In (t-1), then Add to P long In (t),

[0048] This invention calculates the similarity between a person's dot matrix and the behavioral feature sequence of a wearable device, and combines this with confidence measurement to perform long-term and short-term comprehensive pairing. This effectively overcomes the pairing jitter caused by person occlusion, data loss, or data errors in the person identification process, and achieves highly fault-tolerant and stable and reliable association pairing.

[0049] The present invention also provides a terminal device comprising a processor and a memory; the memory stores computer programs / instructions; the processor executes the computer programs / instructions stored in the memory; the computer programs / instructions are configured to implement the steps of the method described in the present invention.

[0050] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0051] Existing methods for detecting people based on a single 2D LiDAR can obtain accurate spatial positioning information by scanning the environment, but they can only capture the shape and behavior of people, failing to obtain accurate personalized features and accurately capture the person's identity information. Methods for identifying people based on a single wearable device carry accurate identity information, but lack global environmental scanning and accurate indoor positioning information, making it impossible to accurately locate and identify people in the environment. This invention fully utilizes the advantages of 2D LiDAR in target detection and local positioning, and wearable devices in identity perception and motion perception, achieving complementary advantages and overcoming the problems of low accuracy and poor robustness in person identification using a single sensor. Furthermore, this invention does not rely on visual image data, does not involve facial privacy, and the inertial sensing data used does not have the problem of person occlusion. Through highly fault-tolerant and stable and reliable association pairing, it maximizes the protection of person identification in various scenarios such as no light source, no image, and privacy protection. This invention overcomes the problems of low accuracy and poor robustness in person identification using a single sensor, maximizing the protection of accurate person identification in various scenarios such as no light source, no image, and privacy protection. Attached Figure Description

[0052] Figure 1 This is a flowchart of a method according to an embodiment of the present invention.

[0053] Figure 2 This is a schematic diagram of the movement and orientation features in an embodiment of the present invention. Detailed Implementation

[0054] like Figure 1 As shown, the specific implementation process of this embodiment of the invention includes:

[0055] Step 1: Use 2D LiDAR to acquire 2D point cloud data of the surrounding environment in real time;

[0056] Step 2: Preprocess the 2D point cloud data to obtain the character point matrix and locate the character's coordinates;

[0057] Step 3: Use the wearable device worn by the person to obtain the person's identity information and inertial sensing data;

[0058] Step 4: Extract human behavioral feature sequences from point cloud data and inertial sensing data respectively;

[0059] Step 5: Normalize the two sets of behavioral feature sequences and perform short-term pairing based on the similarity calculation of the two sets of behavioral feature sequences;

[0060] Step 6: Use historical information to correct short-term pairing biases and obtain stable long-term pairing and person identification results.

[0061] In this embodiment of the invention, step 1, acquiring 2D point cloud data of the surrounding environment using a 2D lidar, refers to scanning the surrounding environment with the lidar device location as the starting point of the coordinates to acquire 2D point cloud data of the overall environment, thus obtaining each scan point P. i Multi-parameter representation: P i ={x i ,y i ,d i}, where x i and y i d represents the coordinates of the scan point in a rectangular coordinate system. i This indicates the radial distance from the scan point to the radar.

[0062] In this embodiment of the invention, step 2 preprocesses the 2D point cloud data, including:

[0063] Step 2.1: Referencing the 2D point cloud image scanned when there is no human activity, delete the static background data in the real-time acquired 2D point cloud data.

[0064] Step 2.2: Cluster the 2D point cloud data to identify the person's point set. Select scan point P. i (x i ,y i Let ) be the cluster center, and set the cluster area radius T. c Iterate through each scan point, when scan point P j coordinates (x) j ,y j When within the cluster region, i.e. The scan point P j Join the dot set When scanning point P j coordinates (x) j ,y j When the scan point P is outside the cluster area, the scan point P will be... j Add candidate points to the candidate point set until all scan points have been traversed; select a scan point from the candidate point set as the cluster center of the next new point set, and then check whether the remaining scan points in the candidate point set are within the cluster area of ​​the new point set; continue until all scan points are assigned to the point set, thus obtaining a set C of all point sets at time t. t .

[0065] Step 2.3, for each point set Curve fitting is performed, and point sets that do not conform to the human body shape are deleted. The upper body of the human body is scanned using a 2D LiDAR scanner to obtain a set of human body point sets, and the range of the length of the human body point set [L] is statistically determined. min ,L max ], L min L is the minimum length of the human body point set. maxThe maximum length of the human body point set; for set C t The i-th person point set like The length of the point set is in [L] min ,L max Within the range, then Considered as a human body, select a set of points The center point is taken as the centroid of the human body, and the coordinates of this centroid represent the position of the human body, denoted as . Otherwise From C t Delete it.

[0066] Step 2.4: Connect the point sets belonging to the same person in consecutive frames. Given the point set C at frame t-1 and frame t. t-1 and C t For C t-1 point set in The trajectory of this point set within frame t-1 is known. Prediction using a Kalman filter In the next frame, denoted as For C t-1 and C t Pair up all points in the set, based on and The cost matrix is ​​constructed using the Euclidean distance; based on the cost matrix, the Hungarian algorithm is used to calculate C. t-1 and C t The correlation between midpoint sets. If and The one with the highest correlation will be Adding the trajectory yields the time point set C for frame t. t trajectory in,

[0067] In this embodiment of the invention, step 3 utilizes a wearable device worn by the person to obtain the person's identity information and inertial sensing data, including:

[0068] Data collection wearable devices The inertial sensing data from the three types of sensors—accelerometer, gyroscope, and magnetometer—are denoted as follows: All inertial sensing data is low-pass filtered to eliminate short-term fluctuations and maintain long-term trends. Wearable devices are also utilized. The account information stored in the database is used to obtain the person's identity information, which is denoted as Profile. j .

[0069] In this embodiment of the invention, step 4 extracts human behavioral feature sequences from point cloud data and inertial sensing data, respectively, including:

[0070] Step 4.1: Obtain the nearest trajectory of the person from the point cloud data of the 2D LiDAR. (It is known that...) For point set The set of trajectories of that point set at frame t, if If the number of midpoint sets is not less than m, then select the m point sets closest to the trajectory to form a sequence. Otherwise, form a sequence from all points in the trajectory. And fill in the empty point set with empty coordinates until... The number of midpoint sets is m.

[0071] Step 4.2: Extract human behavior feature sequences from the point cloud data of 2D LiDAR, where the behavior feature sequences include: movement features, orientation features, acceleration features, and turning features.

[0072] Step 4.3: Obtain the inertial sensing data of the person from the wearable device. (Acquiring data from the wearable device) The data from the accelerometer, gyroscope, and magnetometer are respectively denoted as and The number of data points in the sensing data is kept the same as the number of point sets in the point cloud data, both being m.

[0073] Step 4.4: Extract the human behavior feature sequence from the inertial sensing data, where the behavior feature sequence includes: movement features, orientation features, acceleration features, and turning features.

[0074] In this embodiment of the invention, step 4.2, extracting human behavioral feature sequences from the point cloud data of the 2D LiDAR, includes:

[0075] Step 4.2.1: Extract the movement and orientation features of the person from the point cloud data. Using the point set at time t-1... A square region is defined with the midpoint as the location. This square region consists of 9 nodes with sides of length T. s The data is composed of squares, with the center of the central square as the starting point, and the direction from that starting point toward the midpoints of the other 8 squares as the orientation. A value is assigned to each square; the point set in the preprocessed 2D point cloud data is then determined. For the square in which the point set is located, set the movement and orientation features of the point set to the corresponding values ​​of the square; such as the point set. Compared to The direction of movement is due east, i.e., the point set. If the movement vector is v8, then The movement and orientation features are assigned a value of 8.

[0076] Step 4.2.2: Extract the acceleration features of the person from the point cloud data. (Through...) The system records the positions of the character points, calculates the physical distance the character moves, and then calculates the acceleration. The acceleration feature is marked as The normalization is divided into 4 levels, represented by 0, 1, 2, and 3 respectively, where 0 represents the smallest eigenvalue.

[0077] Step 4.2.3: Extract the turning features of the person from the point cloud data. Turning features It is a sequence The cumulative value of the angle change. Let... For the time point set of frame t The movement vector and the time set of the (t-1)th frame The angle between the movement vectors, i.e. Then the time point set of frame t The turning characteristic value is the point set The cumulative change in the angle between all adjacent motion vectors up to frame t, i.e. in

[0078] In this embodiment of the invention, step 4.4, extracting the human behavioral feature sequence from the inertial sensing data, includes:

[0079] Step 4.4.1: Extract the movement features of the person from the inertial sensing data. First, determine the wearable device. Whether movement has occurred is determined using acceleration values ​​and the Z-score algorithm to detect peak values. Let SW represent the sliding window size. The moving average and standard deviation at time t are calculated as follows:

[0080]

[0081]

[0082] Let β represent the predefined weight, and the peak detection criteria are as follows:

[0083]

[0084] If it does not meet the above peak detection standards, then Setting it to 0 indicates the character is in a stopped state; otherwise, the gyroscope is used. and magnetometer To calculate quaternions. Through transformation, we obtain... The azimuth angle. We use the average azimuth angle of m azimuth angles as the azimuth angle value at time t, and according to... Figure 2 Direction of movement Set it to a value between 1 and 8.

[0085] Step 4.4.2: Extract the orientation features of the person from the inertial sensing data. The person's orientation is obtained by calculating quaternions using a gyroscope and magnetometer. Then, the quaternions are converted to Euler angles to obtain the azimuth angle. The orientation features are defined as the square region whose azimuth value at time t is closest to the target square.

[0086] Step 4.4.3: Extract the acceleration features of the person from the inertial sensing data. Based on the acceleration data, a low-pass filter is used to remove noise, thus extracting the acceleration features. Normalization is divided into 4 levels, represented by 0, 1, 2, and 3 respectively, where 0 represents the smallest eigenvalue.

[0087] Step 4.4.4: Extract the person's turning characteristics from the inertial sensing data. (In wearable devices) A movement vector is obtained between every two adjacent data points. Let... Wearable device at frame t movement vector With the (t-1)th frame movement vector The included angle, that is Then at frame t Steering characteristic value for The cumulative change in the angle between all adjacent motion vectors at the time of frame arrival, i.e. in

[0088] In this embodiment of the invention, step 5 normalizes the two sets of behavioral feature sequences and performs short-term pairing based on the similarity calculation of the two sets of behavioral feature sequences, including:

[0089] Step 5.1: Calculate the similarity between the two sets of behavioral feature sequences, i.e., calculate the similarity of each set of behavioral feature sequences extracted from the laser data and inertial data. and The similarity for each feature f k Calculate the similarity if k = 1, 2, 3, 4. Let X[1:M] and Y[1:N] represent the time interval T. s Inside and Features f kThe sequence values. We use the Dynamic Time Warping (DTW) algorithm to calculate the similarity. Let DTW(i,j) represent the distance between subsequences X[1:i] and Y[1:j], where i≤M,j≤N. The initial condition is DTW(1,1)=d(X[1],Y[1]), where d() represents the Euclidean distance between the two data points. The distance DTW(i,j) between subsequences X[1:i] and Y[1:j] is:

[0090] DTW(i,j)=d(X[i],Y[j])+min{DTW(i-1,j)},DTW(i-1,j-1),DTW(i,j-1)}. (4)

[0091] Step 5.2, similarity numerical normalization. Based on step 5.1, the distance DTW(M,N) of the subsequences and the maximum distance DTW can be obtained. max For each feature f k k = 1, 2, 3, 4, similarity It can be normalized to:

[0092]

[0093] Step 5.3, Calculate the overall similarity. Based on Step 5.2, obtain the normalized similarity values ​​for each feature, and then obtain the overall similarity by weighted summation. for:

[0094]

[0095] Where α k For weighted weights.

[0096] Step 5.4: Perform short-term pairing based on similarity. First, define a similarity matrix S[1:I,1:J], where I and J represent the similarity values ​​at time interval T. s The number of people detected from laser and inertial data is... The priority of pairing is determined by the degree of execution. For each wearable device... The confidence level of the device pairing is calculated using the following variance formula; a larger variance indicates a higher confidence level:

[0097]

[0098] Where S[1:I,J] represents the character dot matrix. and The similarity sequence is S[1:I,J]=[S[i,J]],i∈[1,I]; select the wearable device with the highest current confidence. In the similarity matrix, select the row with the largest value in the j-th column, and select the point set represented by that row to form a pair. For example, if the point set represented by that row is... Then they form pairs. Matching results Add to short-term paired results P short In (t), then remove row i and column j from matrix S, and select the next wearable device with the highest confidence for pairing sequentially until all pairings are completed, obtaining all short-term pairing results P. short (t).

[0099] In this embodiment of the invention, step 6 uses historical information to correct short-term matching biases to obtain the final matching and person identification results, including:

[0100] Step 6.1: Update the long-term pairing results based on short-term pairing results and historical pairing information. Let the long-term pairing results within the previous time interval t-1 be... Among them, group Represents a point set With wearable devices Pairing, and their long-term pairing similarity score is For the following three cases, calculate the long-term pairing result P within the current time interval t. long (t): Represents a point set With wearable devices Similarity;

[0101] When the results of short-term pairing and long-term pairing are consistent, i.e. In short-term pairing P short In (t), In long-term pairing P long In (t-1), and Then Add to P long In (t), where Calculated as follows, where α is the weighting parameter:

[0102]

[0103] When the results of short-term pairing and long-term pairing are inconsistent, i.e., if In short-term pairing P short In (t), In long-term pairing P long In (t-1), when hour, like This indicates that the previous long-term pairing results are more reliable, and therefore... Add to P long In (t), where Calculate using the following method:

[0104]

[0105] like This indicates that the new pairing result is more reliable. Add to P long In (t), where Calculate using the following method:

[0106]

[0107] When a new pairing occurs in a short-term pairing, i.e., if Appearing in short-term pairing P short (t) in, but It did not appear in long-term paired P long In (t-1), then Add to P long In (t), where Calculate using the following method:

[0108]

[0109] After calculating the short- and long-term pairing results for all wearable devices, the long-term pairing result P within the current time interval t can be obtained. long (t).

[0110] If the point cloud image contains a person's pixelation With wearable devices Once a stable, long-term pairing is achieved, it can automatically connect to wearable devices. The stored personal identity information profile j And mark it in the point cloud map simultaneously.

[0111] Experiments have shown that, even in indoor scenes without light sources, the method of this invention can still effectively identify each person wearing a wearable device in the environment in real time, and mark those without wearable devices as "unknown".

Claims

1. A method for person recognition based on 2D laser radar and inertial sensing data, characterized in that, The method comprises the following steps: S1, acquiring 2D point cloud data of the environment in real time by using a 2D laser radar; acquiring identity information and inertial sensing data of the person; S2, pre-processing the 2D point cloud data; S3, extracting a behavior feature sequence of the person from the pre-processed 2D point cloud data and the inertial sensing data respectively; The specific implementation process of step S3 comprises: Obtaining the nearest trajectory of the person from the pretreated 2D point cloud data, and extracting the first behavior feature sequence of the person, wherein the first behavior feature sequence comprises: movement features, orientation features, acceleration features, and turning features; wherein the nearest trajectory The obtaining process comprises: setting a frame time point set , and if the number of point sets in the frame time point set is not less than m, selecting m point sets closest to the trajectory to form a sequence ; otherwise, forming a sequence with all point sets in the trajectory, and filling empty point sets with empty coordinates until the number of point sets in the sequence is m. obtaining the inertial sensing data of the person from the wearable device, and extracting a second behavior feature sequence of the person from the inertial sensing data, wherein the second behavior feature sequence comprises: movement features, orientation features, acceleration features, and turning features; S4, normalizing the two groups of behavior feature sequences obtained in S3 respectively, and performing short-term pairing based on the similarity of the two groups of behavior feature sequences; The implementation process of step S4 comprises: The dynamic time warping method is used to calculate the similarity of the two groups of behavior feature sequences ; , respectively represent the two groups of extracted behavior feature sequences; a similarity matrix of all person point sets and wearable devices is obtained wherein and J represent the number of persons detected from laser data and inertial data in time interval , and let , the confidence of the pairing of each wearable device is calculated using the following variance formula ; wherein is the similarity sequence of person point set and , that is ; the wearable device with the highest current confidence is selected, the row with the largest value in the jth column of the similarity matrix is selected, the point set represented by the row is selected to form a pair, and the pairing result is added to the short-term pairing result; then the row and column are deleted from the similarity matrix , and the wearable device with the next highest confidence is selected to be paired in turn until all pairing is completed, obtaining all short-term pairing results ; S5, correcting the short-term pairing deviation by using historical information to obtain a stable long-term pairing result and a person recognition result. 2.The person recognition method based on 2D laser radar and inertial sensing data according to claim 1, characterized in that, The specific implementation process of step S2 comprises: 2.1, deleting static background data in the 2D point cloud data; 2.2, cluster the 2D point cloud data processed in 2.1 to identify all the person point sets at the frame moment; frame moment; 2.3, performing curve fitting on each person point set, and deleting point sets that do not conform to the shape of a human body; 2.4, connecting point sets belonging to the same person in consecutive frames together to obtain pre-processed 2D point cloud data for the point sets processed in 2.

3. 3.The method of recognizing a person based on 2D lidar and inertial sensing data according to claim 2, wherein, The specific implementation process of step 2.2 comprises: selected scan point set the cluster area radius as the cluster center traverse each scan point, when the coordinate of the scan point is within the cluster area, that is , add the scan point to the point set ; when the coordinate of the scan point is outside the cluster area, add the scan point to the candidate point set, until all scan points are traversed; set the boundary threshold value selecting a scanning point in the candidate point set as the cluster center of the next new point set, and sequentially judging whether the remaining scanning points in the candidate point set are within the cluster area of the new point set; until all the scanning points are divided into the point set, obtaining the set of all the character point sets at the t-th frame moment ; Wherein, the 2D laser radar is installed as a right-angle coordinate system with x axis eastward, y axis northward, and z axis upward, and respectively represent the distance of the scanning point in the x direction and the y direction translation. 4.The method of recognizing a person based on 2D lidar and inertial sensing data according to claim 2, wherein, The specific implementation process of step 2.3 includes: scanning the upper body of a human body by using a 2D laser radar to obtain a set of human body point sets, and counting to obtain the length range of the human body point set , is the minimum length of the human body point set, is the maximum length of the human body point set; for the i-th human body point set in the set , , if the point set length of is in the range of , then is considered to be a human body, and the center point of the point set is selected as the center of mass of the human body, and the coordinates of the center of mass represent the position of the human body, denoted as , otherwise is deleted from .

5. The method of claim 2, wherein the 2D lidar and inertial sensing data based person recognition method is characterized by, The specific implementation process of step 2.4 comprises: Given the set of points at frame t-1 and frame t. and ,for point set in The trajectory of this point set within frame t-1 is known. Predicting point sets using Kalman filters At the position of the next frame, i.e., the predicted point set The centroid at frame t is denoted as ;right and Pair up all points in the set, based on and The cost matrix is ​​constructed using the Euclidean distance; based on the cost matrix, the Hungarian algorithm is used to calculate... and Correlation between midpoint sets; if and The one with the highest correlation will be Add the trajectory to get the first Frame Time Point Set trajectory ;in, , the time point set of the t-th frame trajectory That is, the preprocessed 2D point cloud data.

6. The method of claim 1, wherein, In the first behavior feature sequence: The process of obtaining movement and orientation features includes: using the point set at time t-1 A square region is defined with the midpoint as the location. This square region consists of 9 squares with sides of length [missing information]. The data is composed of squares, with the center of the central square as the starting point, and the direction from that starting point toward the midpoints of the other 8 squares as the orientation. A value is assigned to each square; the point set in the preprocessed 2D point cloud data is then determined. For the square in which the point set is located, set the movement and orientation features of the point set to the corresponding values ​​of the square. The turning feature acquisition process comprises: For the first Frame time point set The moving vector The angle between the moving vector of the first t-1 frame time point set The angle between the moving vector of the first t-1 frame time point set , that is The turning feature value of the first t frame time point set The cumulative value of the angle change of the point set To all adjacent moving vectors of the first t frame, that is , wherein ; In the second behavior feature sequence: The mobile feature determination process comprises: acquiring data of an accelerometer, a gyroscope and a magnetometer of the wearable device , , , ; if the peak detection criterion is not met, setting 0, indicating that the character is in a stop state; otherwise, using the gyroscope and the magnetometer to calculate a quaternion, calculating an azimuth angle of the character, using an average azimuth angle of m azimuth angles as an azimuth angle value at the moment, and setting a moving direction to a value from 1 to 8 according to the square region; wherein the peak detection criterion is: ; , ; SW represents a set sliding window size . , ; The orientation feature determination process comprises: setting the direction of the square region close to the bearing value at time t as the orientation feature; The acceleration feature determination process includes: determining the acceleration feature by The normalization is divided into 4 levels, represented by 0, 1, 2, 3 respectively, wherein 0 represents the minimum feature value; The steering feature determination process includes: in wearable devices A movement vector is obtained between every two adjacent data points. Let... For the first Frame-time wearable devices movement vector With the (t-1)th frame movement vector The included angle, that is Then at frame t The steering characteristic value is The cumulative change in the angle between all adjacent motion vectors at the time of frame arrival, i.e. ,in .

7. The method of claim 1, wherein, The implementation process of step S5 comprises: Let the previous time interval The long-term pairing results within are Group Represents a point set With wearable devices Pairing, and their long-term pairing similarity score is Calculate the current time interval for each of the following three cases. Intra-long-term pairing results Represents a point set With wearable devices Similarity; When the short-term pairing is consistent with the long-term pairing result, i.e. In the short-term pairing , , In the long-term pairing , and , , the is added to the long-term pairing result , wherein The calculation formula is: ; is a weight parameter; represents the similarity of the two groups of behavior feature sequences ; When the short-term pairing is inconsistent with the long-term pairing result, i.e. if In the short-term pairing , In the long-term pairing , when , ; if , the is added to ; wherein ; if , the is added to ; wherein ; When a new pairing appears in the short-term pairing, i.e. if appears in the short-term pairing , but does not appear in the long-term pairing , the is added to , wherein .

8. A terminal device, comprising: comprising a processor and a memory; the memory stores computer programs / instructions; the processor executes the computer programs / instructions stored in the memory; the computer programs / instructions are configured to implement the steps of the method of any one of claims 1-6.

Images