Pedestrian identity information identification method based on multi-source information fusion

Abstract

The invention relates to a pedestrian identity information identification method based on multi-source information fusion, and belongs to the technical field of computer vision. The method comprises the following steps: firstly, wearing a UWB bracelet for each patient, binding the real identity information of the patient with the label of the worn UWB bracelet, and then simultaneously carrying out video shooting and UWB positioning coordinate acquisition; obtaining midpoint coordinates of a pedestrian detection frame of each patient through a video image by adopting a target detection algorithm Yolov5 model, and tracking the pedestrian detection frame through a target tracking algorithm DeepSort; thirdly, performing timestamp alignment and coordinate system alignment on the UWB / IMU positioning coordinates and the midpoint coordinates of the pedestrian detection frame; the method comprises the steps that firstly, coordinate points of the two modes are subjected to initial matching through a Hungary algorithm, mismatching is recognized, finally, real information of a patient on the UWB bracelet is accurately given to pedestrians detected in an image, and reliable pedestrian identity information is provided for downstream behavior analysis tasks.

Description

Technical Field

[0001] The invention belongs to the technical field of computer vision and relates to a method for identifying pedestrian identity information by fusing multi-source information. Background Art

[0002] With the development of the Internet and big data, people have higher and higher requirements for the safety of various places, and intelligent video surveillance technology is widely used in all walks of life. Video surveillance systems are widely used in the safety management of nursing in closed psychiatric wards. However, the task of relying entirely on manual observation to record patient behavior will consume a lot of human resources and is prone to errors. In this case, the application of artificial intelligence technology is particularly necessary. Through computer vision technology, such as target detection and behavior recognition, abnormal behaviors of patients can be automatically identified, prompting medical staff to take intervention measures, reduce the incidence of adverse events such as property destruction and escape, and effectively improve the quality of nursing safety management in psychiatric wards. In addition, convolutional neural networks and recurrent neural networks are used to extract features from surveillance videos and classify behaviors, automatically analyze patient behavior patterns and performance, including sleep, eating, and social interactions, which can help medical staff evaluate the patient's status and changes in condition. At the same time, under video surveillance, pattern recognition technology can also monitor whether patients take medication and receive treatment as prescribed by doctors to ensure the implementation of treatment plans and control of the disease. Finally, using the data recorded by video surveillance, medical staff can conduct quantitative and qualitative evaluations of patient behavior, thereby formulating personalized treatment plans and intervention measures to improve patient safety and treatment effects. The application of the above intelligent monitoring system not only needs to obtain the patient's behavioral information from the video images, but also needs to be able to identify the identity information of each patient in order to achieve the purpose of personalized treatment.

[0003] At present, the mainstream technical method for obtaining personal identity information from video images is face recognition. Face recognition technology identifies individual identities by analyzing facial features in video images, such as facial contours, eyes, mouth, etc. This technology has high accuracy when the face is not obstructed, but it is difficult to avoid face obstruction in hospital scenes. Another type of gesture recognition technology identifies individual identities by analyzing the posture and movement of the human body in the video. It has a certain degree of robustness, but it has high requirements on the camera viewing angle and shooting quality, and the recognition accuracy is low in complex scenes. It can be seen that neither face recognition technology nor gesture recognition technology is suitable for the hospital scene.

[0004] Therefore, there is an urgent need for a method that can identify the identity information of pedestrians in hospital scenarios. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a pedestrian identity information recognition method with multi-source information fusion. In view of the recognition failure problem of face recognition technology in hospital scenes caused by face occlusion, ultra-wideband / inertial (UWB / IMU) positioning technology is introduced. By combining UWB / IMU positioning data with the pixel coordinates of the pedestrian detection frame in the video image, the Hungarian algorithm is used for matching, which can efficiently and accurately match the patient's real identity information with the pedestrian in the video image, thereby providing reliable identity information support for subsequent behavior analysis. A review mechanism is designed to check and correct the mismatch problem caused by the inconsistency of the relative position of pedestrians, which significantly improves the accuracy of pedestrian identity information recognition in a multi-target crowded environment.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A method for identifying pedestrian identity information by fusion of multi-source information, specifically comprising the following steps:

[0008] S1: Data collection: First, the UWB system and camera need to be deployed indoors to ensure that the images captured by the camera are parallel to the spatial coordinate system of the UWB positioning system. Then, each patient needs to wear a UWB tag (bracelet), bind the patient's real identity information (ID) with the UWB bracelet number they wear, and simultaneously perform video capture and obtain UWB positioning coordinates.

[0009] S2: Data calculation and analysis: First, bind the patient's real identity information (ID) with the label of the UWB bracelet he wears, and then use the target detection algorithm Yolov5 model to obtain the midpoint coordinates of the pedestrian detection frame of each patient through the video image, and track the pedestrian detection frame through the target tracking algorithm DeepSort; then, align the UWB / IMU positioning coordinates with the midpoint coordinates of the pedestrian detection frame for timestamps and coordinate systems; then, use the Hungarian algorithm to initially match the coordinate points of the image and UWB data, and then identify the mismatch, and finally accurately give the real information of the patient on the UWB bracelet to the pedestrian detected in the image, providing reliable pedestrian identity information for downstream behavior analysis tasks. In addition, the introduced UWB positioning technology can also provide real-time pedestrian coordinates for downstream dangerous area intrusions.

[0010] Further, in step S2, the detection steps of the target detection algorithm Yolov5 model are as follows: first, the input image I is preprocessed to meet the input requirements of the model; the preprocessing includes adjusting the image size to 416×416 and normalizing it: I'=resize(I,416×416) / 255.0; then CSPDarknet53 is used as the feature extraction backbone network;

[0011] In the Focus module: the input image I′ is divided into blocks and convolved to reduce information loss: X 1 = Focus(I′);

[0012] In the CSP module: X is trained through a cross-stage partial network (CSP). 1 Perform deeper feature extraction X 2 =CSP(X 1 );

[0013] In the Neck part, FPN (Feature Pyramid Network) and PAN (Pixel Aggregation Network) structures are used for feature fusion. First, FPN is used to realize bottom-up feature extraction and top-down feature fusion: F = FPN (X 2 ); Then, a bottom-up path is added through PAN to achieve bidirectional feature fusion: P = PAN (F);

[0014] In the detection head, a convolution operation is used to predict the category and position of the target. Assuming P is the input feature map, the output of the target detection head is: O = σ(Conv(P)), where σ is the activation function and O contains the predicted bounding box position, size and category probability. Finally, the non-maximum suppression (NMS) method is used to filter out overlapping detection boxes and only retain boxes with high confidence.

[0015] The NMS formula is as follows:

[0016]

[0017] Among them, i represents the pedestrian detection frames selected and retained in the initial detection set O (those frames with the highest score or confidence and whose overlap (IOU) with other detection frames is less than the set threshold), IOU represents the intersection-over-union ratio of two bounding boxes, and threshold is the set threshold; the present invention records the midpoint coordinates of the retained pedestrian detection frame {(x 0 ,y 0 ),(x 1 ,y 1 ),…,(x n ,y n )}, n is the number of pedestrian boxes detected.

[0018] Furthermore, in step S2, the UWB / IMU positioning coordinates are timestamp aligned and the coordinate system aligned with the midpoint coordinates of the pedestrian detection frame, specifically including: selecting the UWB positioning data with the smallest time interval with each frame of the image to match it; using the same host to provide timestamps for the video image and the UWB system respectively, first calculating the absolute timestamps of the collected two modal data, and then taking the difference between the timestamps of the two modal data. If the timestamp difference (Δt) is less than the set threshold (δ=4.5ms), the two frames of data are used as the registration data at the same time.

[0019] Further, in step S2, the coordinate points of the image and the UWB data are initially matched by the Hungarian algorithm, specifically including: before formally collecting data, a series of edge UWB positioning data are collected along the edge of the image captured by the camera to form a closed UWB area corresponding to the image area; after this operation, the UWB coordinates outside the image captured by the camera do not participate in the image-UWB data matching, so that the number of collected UWB data is equal to the number of pedestrian detection frames in the image;

[0020] In order to accurately match the UWB positioning coordinates with the midpoint of the pedestrian detection frame of the camera image in the Hungarian algorithm, the present invention needs to unify the coordinates of the two different coordinate systems to the same scale. First, the conversion ratio is calculated: x =Δx pixel / Δx uwb ,λ y =Δy pixel / Δy uwb , where Δx pixel , Δy pixel is the pixel coordinate difference between the X and Y axes in the image, Δx uwb , Δy uwb is the real world coordinate difference between X and Y axes in the UWB system; uwb =x pixel / λ x ,y uwb =y pixel / λ y , so that the camera coordinates (x pixel ,y pixel )(pixel value) is converted to the UWB coordinate (x uwb ,y uwb ) consistent scale coordinates;

[0021] The Hungarian algorithm is used to match the UWB positioning coordinates obtained at the same time with the midpoint of the pedestrian detection frame coordinates obtained from the camera image. First, the pedestrian coordinate set U obtained by UWB is defined as U={u 1 ,u 2 ,…,u n}, where u i=(x i ,y i ), and the pedestrian midpoint coordinate set V = {v,v 2 ,…,v n}, where v i =(x′ j ,y′ j ); Next, the Euclidean distance between each UWB coordinate and each visual coordinate is calculated and used as the element c in the cost matrix C ij ; The specific formula is:

[0022]

[0023] The cost matrix C is in the form of:

[0024]

[0025] Then, the cost matrix C is input into the Hungarian algorithm to solve the minimum weight matching problem; the Hungarian algorithm will output a matching relationship M that minimizes the total cost; this matching relationship corresponds the UWB coordinates to the visual coordinates one by one, ensuring that the real information of the patient on the UWB bracelet is accurately mapped to the pedestrians detected in the image.

[0026] Further, in step S2, mismatch identification specifically includes: within a period of time, if the relative position relationship change of any two coordinate points in the image positioning data is different from the relative position relationship change of any two coordinate points in the UWB / IMU positioning data, it is considered that there is a drift error in the UWB / IMU system or the part of the UWB tag worn by the pedestrian is offside, and the pedestrian identity binding data of this frame is recorded as the pedestrian identity binding data of the previous frame; an algorithm based on the temporal characteristics of the two-modal positioning data is used to identify mismatches, and it is assumed that represents the coordinate vector of pedestrian A relative to pedestrian B in the previous frame image, It represents the coordinate vector of pedestrian A relative to pedestrian B in the current image frame. The tracking of pedestrian detection frames in the previous and next two frames is completed by DeepSort. It means that the relative positions of pedestrians A and B in the two frames of images have not changed, indicating that the ID change of the pedestrian detection frame is an abnormal match. Therefore, the current matching ID is discarded and the ID of the previous frame is true.

[0027] The beneficial effects of the present invention are as follows: In order to solve the recognition failure problem of face recognition technology in hospital scenarios due to face occlusion, the present invention introduces ultra-wideband / inertial (UWB / IMU) positioning technology, and by combining UWB / IMU positioning data with the pixel coordinates of the pedestrian detection frame in the video image and using the Hungarian algorithm for matching, the patient's real identity information can be efficiently and accurately matched with the pedestrian in the video image, thereby providing reliable identity information support for subsequent behavior analysis. In addition, the present invention designs a review mechanism to check and correct the mismatch problem caused by the inconsistency of the relative positions of pedestrians, which significantly improves the accuracy of pedestrian identity information recognition in a multi-target crowded environment. In summary, the present invention can provide stable and reliable identity information for patients in the video in a hospital environment, thereby helping doctors analyze patient behavior through intelligent monitoring and formulate personalized treatment plans.

[0028] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0030] Figure 1 It is the image-UWB data frame timestamp;

[0031] Figure 2 This is a schematic diagram of image-UWB coordinate system alignment;

[0032] Figure 3 Schematic diagram of the relative position of image and UWB in a crowded environment;

[0033] Figure 4 This is a schematic diagram of mismatch identification;

[0034] Figure 5 Flowchart of the pedestrian identity binding process based on the Hungarian algorithm. DETAILED DESCRIPTION

[0035] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0036] See also Figure 1 to Figure 5 In view of the problem that the current mainstream pedestrian identity information recognition method (face recognition) fails to recognize the identity information because the face is easily blocked in the hospital ward scene, the present invention provides a pedestrian identity information recognition method that integrates UWB / IMU / image. The high-precision UWB (ultra-wideband) / IMU (inertial measurement unit) positioning coordinates at the same time and the pixel coordinates of the pedestrian detection frame in the image are matched one by one through the Hungarian algorithm to form a one-to-one correspondence, so that the pedestrian identity information on the UWB bracelet is accurately and efficiently assigned to the pedestrian detected in the image.

[0037] The pedestrian identity information recognition method that integrates UWB / IMU / image proposed in the present invention may encounter the following situation in a multi-target crowded environment: on a two-dimensional plane, pedestrian A stands on the left side of pedestrian B, but pedestrian A puts his hand wearing a UWB tag on the right side of pedestrian B's hand wearing a UWB tag. In this case, the relative position of the pedestrian in the UWB coordinate system (AB) and the relative position of the pedestrian detection box in the image (BA) do not correspond one to one. In this case, the Hungarian algorithm based on the principle of the shortest total Euclidean distance cannot be used to match pedestrian identities. Therefore, the present invention uses the time series characteristics of the two-modal positioning data and designs a review mechanism to check whether there is a mismatch caused by the inconsistency of the relative position of the pedestrian in the two modes, thereby improving the accuracy of pedestrian identity information recognition in a multi-target crowded environment.

[0038] In summary, the present invention provides a method for identifying pedestrian identity information by fusion of multi-source information, which includes two key parts: data acquisition and data calculation and analysis. In the data acquisition part, it is first necessary to deploy the UWB system and the camera indoors to ensure that the picture taken by the camera is parallel to the spatial coordinate system of the UWB positioning system. Then a UWB tag (bracelet) is worn by each patient, and video shooting and UWB positioning coordinate acquisition are performed at the same time. In the data calculation and analysis part, the patient's real identity information (ID) is first bound to the label of the UWB bracelet worn by the patient, and then the pixel coordinates of the pedestrian detection frame of each patient are obtained through the video image, and the pixel coordinates of the pedestrian detection frame are matched with the UWB positioning coordinates obtained at the same time through the Hungarian algorithm, and then the mismatch is identified, and finally the real information of the patient on the UWB bracelet is accurately given to the pedestrian detected in the image, providing reliable pedestrian identity information for downstream behavior analysis tasks.

[0039] The data calculation and analysis part mainly consists of the following five modules:

[0040] 1) Pedestrian target detection module

[0041] The YOLOv5 model is used to detect pedestrians in video images and obtain pedestrian detection frames in each frame. First, a large amount of indoor pedestrian image data is collected and annotated, and then the annotated data is input into the YOLOv5 model for training. Then, the detection accuracy and recall rate of the model are evaluated through the validation set to ensure the performance of the model, so that it can identify pedestrians in indoor scenes.

[0042] The detection process of YOLOv5 can be divided into the following key steps. First, the input image I is preprocessed to meet the input requirements of the model. The preprocessing includes resizing the image to 416×416 and normalizing it: I'=resize(I,416×416) / 255.0. Then CSPDarknet53 is used as the feature extraction backbone network. In the Focus module: The input image I' is divided into blocks and convolved to reduce information loss: X 1 = Focus(I′); In the CSP module: X is trained through the cross-stage partial network (CSP). 1 Perform deeper feature extraction X 2 =CSP(X 1 In the neck part, FPN and PAN structures are used for feature fusion. First, FPN is used to realize bottom-up feature extraction and top-down feature fusion: F = FPN (X 2). Then, a bottom-up path is added through PAN to achieve bidirectional feature fusion: P = PAN (F). In the detection head, a convolution operation is used to predict the category and position of the target. Assuming P is the input feature map, the output of the target detection head is: O = σ (Conv (P)), where σ is the activation function and O contains the predicted bounding box position, size and category probability. Finally, the non-maximum suppression (NMS) method is used to filter out overlapping detection boxes and only retain boxes with high confidence. The NMS formula is as follows Where IOU represents the intersection-over-union ratio of two bounding boxes, and threshold is a set threshold. The present invention records the midpoint coordinates of the retained pedestrian detection frame {(x0, y0), (x1, y1), ..., (xn, yn)} (n is the number of detected pedestrian frames).

[0043] 2) Pedestrian tracking module

[0044] In order to continuously track pedestrians in multiple video frames, the present invention combines YOLOv5 with DeepSort to track detected pedestrians across frames. DeepSort is an advanced multi-target tracking algorithm that uses a convolutional neural network (CNN) to extract feature vectors from the detection frame to match the same pedestrian in different frames, and then uses Kalman filtering to predict the motion state of pedestrians to improve tracking stability and continuity. Finally, the Hungarian algorithm is used to solve the matching problem between the detection frame and the tracking trajectory to ensure efficient target association.

[0045] 3) Image-UWB data frame time alignment module

[0046] In order to ensure that the UWB positioning coordinates and the pedestrian detection frame coordinates in the image can be accurately matched one by one, the present invention needs to align the two modal data on the time axis as much as possible. Since the frequencies of the acquired video image and the UWB data frame are inconsistent, the image frame rate is about 40 frame / s, and the UWB frame rate is about 112 frame / s. Figure 1 As shown, the present invention needs to select the UWB positioning data with the smallest time interval with each frame of the picture to match it. The present invention uses the same host to provide timestamps for the video image and the UWB system respectively, first calculates the absolute timestamps of the collected two-modal data, and then takes the difference of the timestamps of the two-modal data. If the timestamp difference (Δt) is less than the set threshold (δ=4.5ms), the two frames of data are used as the registration data at the same time.

[0047] 4) Image-UWB data matching module

[0048] When matching image-UWB data, the number of data in the two modalities needs to be equal, otherwise the matching will not converge. Therefore, before formally collecting data, the present invention first collects a series of edge UWB positioning data along the edge of the picture taken by the camera to form a closed UWB area to correspond to the image area. After this operation, the UWB coordinates outside the picture taken by the camera do not participate in the image-UWB data matching, so that the number of collected UWB data is equal to the number of pedestrian detection frames in the image.

[0049] In order to accurately match the UWB positioning coordinates with the midpoint of the pedestrian detection frame of the camera image in the Hungarian algorithm, the present invention needs to unify the coordinates of the two different coordinate systems to the same scale, such as Figure 2 As shown. First calculate the conversion ratio: x =Δx pixel / Δx uwb ,λ y =Δy pixel / Δy uwb , where Δx pixel , Δy pixel is the pixel coordinate difference between the X and Y axes in the image, Δx uwb , Δy uwb is the real world coordinate difference between the X and Y axes in the UWB system. uwb =x pixel / λ x ,y uwb =y pixel / λ y , thereby converting the camera coordinates (pixel values) into scale coordinates consistent with UWB coordinates.

[0050] In order to match the UWB positioning coordinates obtained at the same time with the midpoint of the pedestrian detection frame coordinates obtained by the camera image, the Hungarian algorithm can be used. First, define the pedestrian coordinate set U obtained by UWB = {u 1 ,u 2 ,…,y n}, where u i =(x i ,y i ), and the pedestrian midpoint coordinate set V = {v,v 2 ,…,v n}where v i =(x′ j ,y′ j ). Next, the Euclidean distance between each UWB coordinate and each visual coordinate is calculated and used as the element c in the cost matrix C. ij The specific formula is:

[0051]

[0052] The cost matrix C is in the form of:

[0053]

[0054] Then, the cost matrix C is input into the Hungarian algorithm to solve the minimum weight matching problem. The Hungarian algorithm will output a matching relationship M that minimizes the total cost. This matching relationship corresponds the UWB coordinates to the visual coordinates one by one, ensuring that the real information of the patient on the UWB bracelet is accurately mapped to the pedestrians detected in the image.

[0055] 5) Image-UWB data mismatch identification module

[0056] In a crowded environment with multiple targets, UWB positioning has the problem of inaccurate relative position, and it is impossible to directly use the Hungarian algorithm based on the principle of the shortest total Euclidean distance to match pedestrian identities. The relative position of the pedestrian in the UWB / IMU coordinate system (BA) and the relative position of the pedestrian detection frame in the image (AB) do not correspond one to one. Figure 3 As shown in the figure, on the two-dimensional plane, pedestrian A stands on the left side of pedestrian B, but pedestrian A puts the hand wearing the UWB tag on the right side of pedestrian B's hand wearing the UWB tag. At this time, the Hungarian algorithm based on the principle of the shortest total Euclidean distance will mistakenly match pedestrian B's UWB tag information to pedestrian A in the image, and mistakenly match pedestrian A's UWB tag information to pedestrian B in the image. In addition, in addition to the above matching errors caused by the partial cross-over of the UWB tags worn by pedestrians, in complex multi-target crowded situations, the drift error of the UWB / IMU system will cause inaccurate UWB positioning and will also cause mismatching.

[0057] In response to the above problems, the present invention adds a review mechanism based on the initial matching of the Hungarian algorithm to check whether there is a false match. Over a period of time, if the change in the relative position relationship between any two coordinate points in the image positioning data is different from the change in the relative position relationship between any two coordinate points in the UWB / IMU positioning data, it is considered that there is a drift error in the UWB / IMU system or that part of the UWB tag worn by the pedestrian is offside, and this frame of pedestrian identity binding data is recorded as the previous frame of pedestrian identity binding data. The present invention designs an algorithm based on the temporal characteristics of two-modal positioning data to identify false matches, such as Figure 4 As shown. represents the coordinate vector of pedestrian A relative to pedestrian B in the previous frame image, It represents the coordinate vector of pedestrian A relative to pedestrian B in the current image frame. The tracking of pedestrian detection frames in the previous and next two frames is completed by DeepSort. It means that the relative positions of pedestrians A and B in the two frames of images have not changed, indicating that the ID change of the pedestrian detection frame is an abnormal match. Therefore, the current matching ID is discarded and the ID of the previous frame is true.

[0058] The present invention provides an optimal implementation scheme of a pedestrian identification method based on multi-source information fusion, such as Figure 5 As shown in the figure, firstly, a UWB bracelet is put on each patient, and the patient's real identity information (ID) is bound to the bracelet number he wears, and then video shooting and UWB positioning coordinates are obtained at the same time. The video image obtains the midpoint coordinates of the pedestrian detection frame of each patient through the target detection algorithm Yolov5, and the pedestrian detection frame is tracked by the target tracking algorithm DeepSort. Then, the UWB / IMU positioning coordinates are aligned with the midpoint coordinates of the pedestrian detection frame for timestamp alignment and coordinate system alignment. Then, the coordinate points of the two modes are initially matched through the Hungarian algorithm, and then the mismatch is identified. Finally, the real information of the patient on the UWB bracelet is accurately given to the pedestrian detected in the image, providing reliable pedestrian identity information for downstream behavior analysis tasks.

[0059] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A pedestrian identification method based on multi-source information fusion, characterized in that: The method specifically comprises the following steps: S1: Data collection: First, the UWB system and camera need to be deployed indoors to ensure that the images captured by the camera are parallel to the spatial coordinate system of the UWB positioning system. Then, each patient needs to wear a UWB bracelet, bind the patient's real identity information with the UWB bracelet number they wear, and simultaneously perform video capture and obtain UWB positioning coordinates. S2: Data calculation and analysis: First, bind the patient's real identity information to the label of the UWB bracelet he / she wears, and then use the target detection algorithm Yolov5 model through the video image to obtain the midpoint coordinates of the pedestrian detection frame of each patient, and track the pedestrian detection frame through the target tracking algorithm DeepSort; then, align the UWB / IMU positioning coordinates with the midpoint coordinates of the pedestrian detection frame for timestamps and coordinate systems; then, use the Hungarian algorithm to initially match the coordinate points of the image and UWB data, and then identify false matches, and finally accurately give the patient's real information on the UWB bracelet to the pedestrian detected in the image, providing reliable pedestrian identity information for downstream behavior analysis tasks.

2. The pedestrian identification information recognition method according to claim 1, characterized in that: In step S2, the detection steps of the target detection algorithm Yolov5 model are as follows: first, the input image I is preprocessed to meet the input requirements of the model; the preprocessing includes adjusting the image size to 416×416 and normalizing it: I'=resize(I,416×416) / 255.0; then CSPDarknet53 is used as the feature extraction backbone network; In the Focus module: the input image I′ is divided into blocks and convolved to reduce information loss: X1 = Focus(I′); In the CSP module: X1 is subjected to deeper feature extraction through the cross-stage partial network CSP X2 = CSP (X1); In the Neck part, FPN and PAN structures are used for feature fusion. First, FPN is used to realize bottom-up feature extraction and top-down feature fusion: F = FPN (X2). Then, PAN is used to add a bottom-up path to realize bidirectional feature fusion: P = PAN (F). In the detection head, convolution operation is used to predict the category and position of the target. Assuming P is the input feature map, the output of the target detection head is: O = σ(Conv(P)), where σ is the activation function and O contains the predicted bounding box position, size and category probability. Finally, the non-maximum suppression (NMS) method is used to filter out overlapping detection boxes and only retain boxes with high confidence. The NMS formula is as follows: Among them, i represents the pedestrian detection box selected and retained in the initial detection set O, IOU represents the intersection-over-union ratio of two bounding boxes, and threshold is the set threshold; record the coordinates of the midpoints of the retained pedestrian detection box {(x0,y0),(x1,y1),…,(x n ,y n )}, n is the number of pedestrian boxes detected.

3. The pedestrian identification information recognition method according to claim 1, characterized in that: In step S2, the UWB / IMU positioning coordinates are aligned with the midpoint coordinates of the pedestrian detection frame for timestamp alignment and coordinate system alignment, specifically including: selecting the UWB positioning data with the smallest time interval with each frame of the image to match it; using the same host to provide timestamps for the video image and the UWB system respectively, first calculating the absolute timestamps of the collected two-modal data, and then taking the difference between the timestamps of the two-modal data. If the timestamp difference is less than the set threshold, the two frames of data are used as the registration data at the same time.

4. The pedestrian identification information recognition method according to claim 1, characterized in that: In step S2, the coordinate points of the image and the UWB data are initially matched by the Hungarian algorithm, which specifically includes: before formally collecting data, a series of edge UWB positioning data are collected along the edge of the image captured by the camera to form a closed UWB area corresponding to the image area; after this operation, the UWB coordinates outside the image captured by the camera do not participate in the image-UWB data matching, so that the number of collected UWB data is equal to the number of pedestrian detection frames in the image; To unify the coordinates of the two different coordinate systems to the same scale, first calculate the conversion ratio: x =Δx pixel Δx uwb ,λ y =Δy pixel / Δy uwb , where Δx pixel , Δy pixel is the pixel coordinate difference between the X and Y axes in the image, Δx uwb , Δy uwb is the real world coordinate difference between X and Y axes in the UWB system; uwb = = x pixel / λ x ,y uwb = =y pixel / λ y , so that the camera coordinates (x pixel ,y pixel ) is converted to the UWB coordinate (x uwb ,y uwb ) consistent scale coordinates; The Hungarian algorithm is used to match the UWB positioning coordinates obtained at the same time with the midpoint of the pedestrian detection frame coordinates obtained from the camera image. First, the pedestrian coordinate set U obtained by UWB is defined as U = {u1, u2, …, u n }, where u i =(x i ,y i ), and the pedestrian midpoint coordinate set V = {v,v2,…,v n }, where v i =(x′ j ,y′ j ); Next, the Euclidean distance between each UWB coordinate and each visual coordinate is calculated and used as the element c in the cost matrix C ij ; The specific formula is: The cost matrix C is in the form of: Then, the cost matrix C is input into the Hungarian algorithm to solve the minimum weight matching problem; the Hungarian algorithm will output a matching relationship M that minimizes the total cost; this matching relationship corresponds the UWB coordinates to the visual coordinates one by one, ensuring that the real information of the patient on the UWB bracelet is accurately mapped to the pedestrians detected in the image.

5. The pedestrian identification information recognition method according to claim 1, characterized in that: In step S2, mismatch identification specifically includes: within a period of time, if the relative position relationship change of any two coordinate points in the image positioning data is different from the relative position relationship change of any two coordinate points in the UWB / IMU positioning data, it is considered that there is a drift error in the UWB / IMU system or the pedestrian is wearing a UWB tag that is partially offside, and the pedestrian identity binding data of this frame is recorded as the pedestrian identity binding data of the previous frame; an algorithm based on the temporal characteristics of the two-modal positioning data is used to identify mismatches, and it is assumed that represents the coordinate vector of pedestrian A relative to pedestrian B in the previous frame image, It represents the coordinate vector of pedestrian A relative to pedestrian B in the current image frame. The tracking of pedestrian detection frames in the previous and next two frames is completed by DeepSort. It means that the relative positions of pedestrians A and B in the two frames of images have not changed, indicating that the ID change of the pedestrian detection frame is an abnormal match. Therefore, the current matching ID is discarded and the ID of the previous frame is true.

Images