A method for pedestrian identity information recognition through multi-source information fusion

Abstract

This invention relates to a pedestrian identification method based on multi-source information fusion, belonging to the field of computer vision technology. The method first involves equipping each patient with a UWB wristband, binding the patient's real identity information to the UWB wristband's identifier, and then simultaneously capturing video and acquiring UWB positioning coordinates. Next, the method uses the YOLOv5 object detection algorithm to obtain the coordinates of the points in the pedestrian detection box for each patient from the video image, and tracks the pedestrian detection box using the DeepSort object tracking algorithm. Then, the UWB / IMU positioning coordinates are timestamped and aligned with the coordinates of the points in the pedestrian detection box. Finally, the Hungarian algorithm is used to initially match the coordinates of the two modalities, identifying mismatches, and ultimately accurately assigning the patient's real information from the UWB wristband to the pedestrians detected in the image, providing reliable pedestrian identification information for downstream behavior analysis tasks.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology and relates to a method for pedestrian identity information recognition through multi-source information fusion. Background Technology

[0002] With the development of the internet and big data, people have increasingly higher requirements for security in various places, and intelligent video surveillance technology is widely used in various industries. Video surveillance systems are widely used in the nursing safety management of closed psychiatric wards. However, relying entirely on manual observation and recording of patient behavior is labor-intensive and prone to errors. In this context, the application of artificial intelligence technology is particularly necessary. Through computer vision technologies, such as object detection and behavior recognition, abnormal patient behavior can be automatically identified, prompting medical staff to take intervention measures, reducing the incidence of adverse events such as property damage and escape, and effectively improving the quality of nursing safety management in psychiatric wards. In addition, by using convolutional neural networks and recurrent neural networks to extract features from surveillance videos and classify behaviors, the behavior patterns and performance of patients, including sleep, eating, and social interactions, can be automatically analyzed, helping medical staff to assess the patient's condition and changes in illness. At the same time, under video surveillance, pattern recognition technology can also monitor whether patients take medication and receive treatment as prescribed, ensuring the implementation of treatment plans and the control of the condition. Finally, using the data recorded by video surveillance, medical staff can conduct quantitative and qualitative assessments of patient behavior, thereby developing personalized treatment plans and intervention measures to improve patient safety and treatment outcomes. The application of the above-mentioned intelligent monitoring system not only needs to obtain patients' behavioral information from video images, but also needs to be able to identify each patient's identity information in order to achieve the goal of personalized treatment.

[0003] Currently, the mainstream technology for extracting personal identification information from video images is facial recognition. Facial recognition technology identifies individuals by analyzing facial features in video images, such as facial contours, eyes, and mouth. This technology is highly accurate when faces are unobstructed; however, in a hospital setting, facial occlusion is unavoidable. Another technology, pose recognition, identifies individuals by analyzing the posture and movements of people in the video. It has a certain degree of robustness, but it requires high-quality camera angles and image quality, and its accuracy is lower in complex scenes. Therefore, neither facial recognition nor pose recognition technologies are suitable for a hospital setting.

[0004] Therefore, there is an urgent need for a method that can identify pedestrians in hospital settings. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a pedestrian identity information recognition method based on multi-source information fusion. Addressing the recognition failure problem caused by facial occlusion in hospital scenarios, this invention introduces ultra-wideband / inertial (UWB / IMU) positioning technology. By combining UWB / IMU positioning data with the pixel coordinates of pedestrian detection boxes in video images, and using the Hungarian algorithm for matching, it can efficiently and accurately match the patient's real identity information with pedestrians in video images, thus providing reliable identity information support for subsequent behavior analysis. Furthermore, a review mechanism is designed to check and correct mismatches caused by inconsistent relative positions of pedestrians, significantly improving the accuracy of pedestrian identity information recognition in multi-target crowded environments.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A method for pedestrian identity information recognition through multi-source information fusion specifically includes the following steps:

[0008] S1: Data Acquisition: First, a UWB system and cameras need to be deployed indoors to ensure that the images captured by the cameras are parallel to the spatial coordinate system of the UWB positioning system; then, each patient wears a UWB tag (wristband) to bind the patient's real identity information (ID) with the UWB wristband number they wear, while simultaneously recording video and acquiring UWB positioning coordinates;

[0009] S2: Data Calculation and Analysis: First, the patient's real identity information (ID) is bound to the UWB wristband number they wear. Then, the coordinates of the points in the pedestrian detection box for each patient are obtained using the Yolov5 object detection algorithm from the video image. The pedestrian detection box is tracked using the DeepSort object tracking algorithm. Next, the UWB / IMU positioning coordinates are timestamped and aligned with the coordinates of the points in the pedestrian detection box. Then, the Hungarian algorithm is used to initially match the coordinate points of the image and UWB data, and mismatches are identified. Finally, the real information of the patient on the UWB wristband is accurately assigned to the pedestrians detected in the image, providing reliable pedestrian identity information for downstream behavior analysis tasks. In addition, the introduced UWB positioning technology can also provide pedestrian coordinates in real time for downstream intrusion into dangerous areas.

[0010] Further, in step S2, the detection steps of the target detection algorithm Yolov5 model are as follows: First, the input image I is preprocessed to meet the input requirements of the model; the preprocessing includes adjusting the image size to 416×416 and normalizing it: I'=resize(I,416×416) / 255.0; then CSPDarknet53 is used as the feature extraction backbone network;

[0011] In the Focus module: the input image I′ is divided into blocks and convolutional to reduce information loss: X1 = Focus(I′);

[0012] In the CSP module: X1 is further feature extracted through the cross-stage partial network (CSP) X2 = CSP(X1);

[0013] In the Neck section, feature fusion is performed using FPN (Feature Pyramid Network) and PAN (Pixel Aggregation Network) structures. First, bottom-up feature extraction and top-down feature fusion are achieved through FPN: F = FPN(X2). Then, bidirectional feature fusion is achieved by adding bottom-up paths through PAN: P = PAN(F).

[0014] In the detection head, convolution operations are used to predict the category and location of the target. Assuming P is the input feature map, the output of the target detection head is: O = σ(Conv(P)), where σ is the activation function and O contains the predicted bounding box location, size, and category probability. Finally, non-maximum suppression (NMS) is used to filter out overlapping detection boxes, keeping only the boxes with high confidence.

[0015] The NMS formula is as follows:

[0016]

[0017] Among them, o i This represents the pedestrian detection boxes selected and retained from the initial detection set O (those boxes with the highest score or confidence and an Intersection over Union (IOU) of less than a set threshold), where IOU represents the intersection-union ratio of two bounding boxes, and threshold is the set threshold value; this invention records the coordinates of the points in the retained pedestrian detection boxes {(x0,y0),(x1,y1),…,(x... n ,y n )}, where n is the number of detected pedestrian boxes.

[0018] Furthermore, in step S2, the UWB / IMU positioning coordinates are aligned with the coordinates of the points in the pedestrian detection box in terms of timestamps and coordinate systems. Specifically, this includes: selecting the UWB positioning data with the smallest time interval with each frame of the image for matching; using the same host to provide timestamps for the video image and the UWB system respectively; first calculating the absolute timestamps of the two modal data collected; then taking the difference between the timestamps of the two modal data; if the timestamp difference (Δt) is less than the set threshold (δ = 4.5ms), then the two frames of data are used as the registration data at the same time.

[0019] Furthermore, in step S2, the coordinates of the image and UWB data are initially matched using the Hungarian algorithm. Specifically, this includes: before formally collecting data, collecting a series of edge UWB positioning data along the edge of the image captured by the camera to form a closed UWB region corresponding to the image region; after this operation, the UWB coordinates outside the image captured by the camera do not participate in the image-UWB data matching, so that the number of collected UWB data is equal to the number of pedestrian detection boxes in the image;

[0020] To accurately match UWB positioning coordinates with pedestrian detection bounding boxes in camera-captured images using the Hungarian algorithm, this invention requires unifying the coordinates of the two different coordinate systems to the same scale. First, the transformation ratio λ is calculated. x =Δx pixel / Δx uwb , λ y =Δy pixel / Δy uwb , where Δx pixel Δy pixel Δx represents the pixel coordinate difference between the X and Y axes in the image. uwb Δy uwb The difference between the real-world coordinates of the X and Y axes in the UWB system; x uwb =x pixel / λ x y uwb =y pixel / λ y This is used to determine the camera coordinates (x) pixel ,y pixel (pixel value) converted to UWB coordinates (x uwb ,y uwb Consistent proportional coordinates;

[0021] The Hungarian algorithm is used to match the midpoints of the UWB positioning coordinates obtained at the same time with the pedestrian detection box coordinates obtained from the camera image. First, the set of pedestrian coordinates obtained by UWB is defined as U = {u1, u2, ..., u...}. n}, where u i =(x i ,y i ), and the set of pedestrian midpoint coordinates V = {v, v2, ..., v} obtained through camera image detection boxes. n}, where v i =(x′) j ,y′ j Next, the Euclidean distance between each UWB coordinate and each visual coordinate is calculated and used as an element c in the cost matrix C. ij The specific formula is:

[0022]

[0023] The cost matrix C is in the form of:

[0024]

[0025] Then, the cost matrix C is input into the Hungarian algorithm to solve the minimum weight matching problem. The Hungarian algorithm will output a matching relationship M that minimizes the total cost. This matching relationship maps the UWB coordinates to the visual coordinates one by one, ensuring that the patient's real information on the UWB wristband is accurately mapped to the pedestrians detected in the image.

[0026] Further, in step S2, mismatch identification specifically includes: if the relative positional relationship between any two coordinate points in the image positioning data differs from that in the UWB / IMU positioning data over a period of time, it is considered that there is a drift error in the UWB / IMU system or that the pedestrian wearing the UWB tag has partially overstepped its bounds; this frame of pedestrian identity binding data is recorded as the previous frame of pedestrian identity binding data; an algorithm based on the temporal features of the two-modal positioning data is used to identify mismatches, assuming... This represents the coordinate vector of pedestrian A relative to pedestrian B in the previous frame image. This represents the coordinate vector of pedestrian A relative to pedestrian B in the current image frame. The tracking of pedestrian detection boxes in the two frames is done by DeepSort. This indicates that the relative positions of pedestrians A and B in the two consecutive frames have not changed, which means that the ID change of the pedestrian detection box is an abnormal match. Therefore, the ID of the current match is discarded, and the ID of the previous frame is taken as true.

[0027] The beneficial effects of this invention are as follows: Addressing the problem of facial recognition failure due to face occlusion in hospital settings, this invention introduces ultra-wideband / inertial (UWB / IMU) positioning technology. By combining UWB / IMU positioning data with the pixel coordinates of pedestrian detection boxes in video images, and using the Hungarian algorithm for matching, it can efficiently and accurately match the patient's real identity information with pedestrians in video images, thus providing reliable identity information support for subsequent behavior analysis. Furthermore, this invention designs a review mechanism to check and correct mismatches caused by inconsistent relative positions of pedestrians, significantly improving the accuracy of pedestrian identity information recognition in multi-target crowded environments. In summary, this invention can provide stable and reliable identity information for patients in video in a hospital environment, thereby helping doctors to analyze patient behavior and develop personalized treatment plans through intelligent monitoring.

[0028] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0029] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0030] Figure 1 For image-UWB data frame timestamps;

[0031] Figure 2 This is a schematic diagram illustrating the alignment of the image with the UWB coordinate system.

[0032] Figure 3 This is a schematic diagram illustrating the relative position of the image and UWB in a congested environment.

[0033] Figure 4 This is a diagram illustrating the identification of mismatches.

[0034] Figure 5 This is a flowchart of the pedestrian identity binding process based on the Hungarian algorithm. Detailed Implementation

[0035] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0036] Please see Figures 1-5 To address the problem of recognition failure caused by occlusion of faces in hospital ward scenarios, which is a common issue in current mainstream pedestrian identification methods (face recognition), this invention provides a pedestrian identification method that integrates UWB / IMU / image. It utilizes high-precision UWB (Ultra-Wideband) / IMU (Inertial Measurement Unit) positioning coordinates at the same time and the pixel coordinates of the pedestrian detection box in the image to establish a one-to-one correspondence through a Hungarian algorithm. This allows for the accurate and efficient assignment of pedestrian identification information from the UWB wristband to the detected pedestrians in the image.

[0037] The pedestrian identification method integrating UWB / IMU / image proposed in this invention may encounter the following situation in multi-target crowded environments: In a two-dimensional plane, pedestrian A stands to the left of pedestrian B, but pedestrian A's hand wearing a UWB tag is placed to the right of pedestrian B's hand wearing a UWB tag. In this case, the relative positions of the pedestrians in the UWB coordinate system (AB) and the relative positions of the pedestrian detection boxes in the image (BA) do not correspond one-to-one. In this situation, the Hungarian algorithm based on the principle of shortest total Euclidean distance cannot be used to match pedestrian identities. Therefore, this invention utilizes the temporal characteristics of two-modal positioning data to design a review mechanism to check for mismatches caused by inconsistencies in the relative positions of pedestrians in the two modalities, thereby improving the accuracy of pedestrian identification in multi-target crowded environments.

[0038] In summary, this invention provides a pedestrian identification method based on multi-source information fusion, comprising two key parts: data acquisition and data computation and analysis. In the data acquisition part, a UWB system and camera are first deployed indoors, ensuring that the camera's captured image is parallel to the spatial coordinate system of the UWB positioning system. Then, each patient wears a UWB tag (wristband), and video recording and UWB positioning coordinate acquisition are performed simultaneously. In the data computation and analysis part, the patient's real identity information (ID) is first bound to the UWB wristband number. Then, the pixel coordinates of the pedestrian detection box for each patient are obtained from the video image. These coordinates are matched with the UWB positioning coordinates acquired at the same time using a Hungarian algorithm, and mismatches are identified. Finally, the patient's real information from the UWB wristband is accurately assigned to the pedestrian detected in the image, providing reliable pedestrian identification information for downstream behavior analysis tasks.

[0039] The data calculation and analysis section mainly consists of the following five modules:

[0040] 1) Pedestrian Target Detection Module

[0041] The YOLOv5 model is used to detect pedestrians in video images, obtaining pedestrian detection boxes in each frame. First, a large amount of indoor pedestrian image data is collected and labeled. Then, the labeled data is input into the YOLOv5 model for training. Finally, the detection accuracy and recall of the model are evaluated through a validation set to ensure the model performance and enable it to recognize pedestrians in indoor scenes.

[0042] The YOLOv5 detection process can be divided into the following key steps. First, the input image I is preprocessed to meet the model's input requirements. Preprocessing includes resizing the image to 416×416 and normalizing it: I' = resize(I, 416×416) / 255.0. Next, CSPDarknet53 is used as the feature extraction backbone network. In the Focus module: the input image I' is segmented and convolutional to reduce information loss: X1 = Focus(I'); in the CSP module: X1 undergoes deeper feature extraction through the Cross-Stage Part Network (CSP): X2 = CSP(X1). In the Neck part, feature fusion is performed using FPN and PAN structures. First, bottom-up feature extraction and top-down feature fusion are achieved through FPN: F = FPN(X2). Then, bidirectional feature fusion is achieved by adding bottom-up paths through PAN: P = PAN(F). In the detection head part, convolutional operations are used to predict the target's category and location. Assuming P is the input feature map, the output of the object detection head is: O = σ(Conv(P)), where σ is the activation function, and O contains the predicted bounding box position, size, and class probability. Finally, non-maximum suppression (NMS) is used to filter out overlapping detection boxes, retaining only those with high confidence. The NMS formula is as follows: Where IOU represents the intersection-union ratio of two bounding boxes, and threshold is the set threshold. This invention records the coordinates of the points in the pedestrian detection boxes {(x0,y0),(x1,y1),…,(xn,yn)} (n is the number of detected pedestrian boxes).

[0043] 2) Pedestrian tracking module

[0044] To enable continuous pedestrian tracking across multiple video frames, this invention combines YOLOv5 with DeepSort for cross-frame pedestrian tracking. DeepSort is an advanced multi-object tracking algorithm that uses a convolutional neural network (CNN) to extract feature vectors from detection boxes to match the same pedestrian in different frames. It then uses Kalman filtering to predict the pedestrian's motion state, improving tracking stability and continuity. Finally, it employs the Hungarian algorithm to solve the matching problem between the detection boxes and the tracking trajectory, ensuring efficient target association.

[0045] 3) Image-UWB data frame time alignment module

[0046] To ensure accurate matching between UWB positioning coordinates and pedestrian detection box coordinates in the image, this invention requires aligning the two modal data as much as possible on the time axis. Since the frequencies of the acquired video images and UWB data frames are different (the image frame rate is approximately 40 frames / s, and the UWB frame rate is approximately 112 frames / s), ... Figure 1As shown, this invention requires selecting the UWB positioning data with the smallest time interval between each frame of the image for matching. This invention uses the same host to provide timestamps for both the video image and the UWB system. First, the absolute timestamps of the two acquired modal data are calculated. Then, the difference between the timestamps of the two modal data is taken. If the timestamp difference (Δt) is less than a set threshold (δ = 4.5ms), then these two frames of data are used as registration data at the same time.

[0047] 4) Image-UWB data matching module

[0048] Image-UWB data matching requires an equal number of data points for both modalities; otherwise, the matching may fail to converge. Therefore, this invention first collects a series of edge UWB positioning data along the edges of the image captured by the camera before formally acquiring data, forming a closed UWB region corresponding to the image region. Through this operation, UWB coordinates outside the camera's captured image do not participate in image-UWB data matching, thus ensuring that the number of acquired UWB data points is equal to the number of pedestrian detection boxes in the image.

[0049] To accurately match UWB positioning coordinates with pedestrian detection bounding boxes in camera-captured images within the Hungarian algorithm, this invention requires unifying the coordinates of the two different coordinate systems to the same scale, such as... Figure 2 As shown. First, calculate the conversion ratio: λ x =Δx pixel / Δx uwb , λ y =Δy pixel / Δy uwb , where Δx pixel Δy pixel Δx represents the pixel coordinate difference between the X and Y axes in the image. uwb Δy uwb This represents the difference between the real-world coordinates of the X and Y axes in the UWB system. uwb =x pixel / λ x y uwb =y pixel / λ y This converts the camera coordinates (pixel values) into proportional coordinates that are consistent with UWB coordinates.

[0050] To match the midpoint of the pedestrian detection bounding box obtained from the camera image at the same time point using the UWB positioning coordinates acquired simultaneously, the Hungarian algorithm can be used. First, define the set of pedestrian coordinates obtained via UWB as U = {u1, u2, ..., y}. n}, where u i =(x i ,y i), and the set of pedestrian midpoint coordinates V = {v, v2, ..., v} obtained through camera image detection boxes. n}where v i =(x′) j ,y′ j Next, the Euclidean distance between each UWB coordinate and each visual coordinate is calculated and used as an element c in the cost matrix C. ij The specific formula is:

[0051]

[0052] The cost matrix C is in the form of:

[0053]

[0054] Then, the cost matrix C is input into the Hungarian algorithm to solve the minimum weight matching problem. The Hungarian algorithm outputs a matching relation M that minimizes the total cost. This matching relation maps UWB coordinates to visual coordinates one-to-one, ensuring that the patient's real information on the UWB wristband is accurately mapped to the pedestrians detected in the image.

[0055] 5) Image-UWB data mismatch identification module

[0056] In crowded, multi-target environments, UWB localization suffers from relative position inaccuracies, making it impossible to directly use the Hungarian algorithm, based on the principle of minimizing total Euclidean distance, to match pedestrian identities. There is a lack of one-to-one correspondence between the relative position of a pedestrian (BA) in the UWB / IMU coordinate system and the relative position (AB) of the pedestrian detection bounding box in the image. For example... Figure 3 As shown, in a two-dimensional plane, pedestrian A stands to the left of pedestrian B, but pedestrian A places their hand (wearing a UWB tag) to the right of pedestrian B's hand (wearing a UWB tag). In this case, the Hungarian algorithm, based on the principle of minimizing total Euclidean distance, will incorrectly match pedestrian B's UWB tag information to pedestrian A in the image, and vice versa. Furthermore, besides the partial overlap of pedestrians wearing UWB tags causing these matching errors, in complex multi-target congestion situations, drift errors in the UWB / IMU system leading to inaccurate UWB positioning can also cause mismatches.

[0057] To address the above issues, this invention incorporates a review mechanism into the initial matching of the Hungarian algorithm to check for mismatches. If, over a period of time, the relative positional relationship between any two coordinate points in the image positioning data differs from that in the UWB / IMU positioning data, it is considered that there is a drift error in the UWB / IMU system or that the pedestrian wearing the UWB tag has partially moved out of position. The pedestrian identity binding data for this frame is recorded as the previous frame's pedestrian identity binding data. This invention designs an algorithm based on the temporal characteristics of two-modal positioning data to identify mismatches, such as... Figure 4 As shown. Among them. This represents the coordinate vector of pedestrian A relative to pedestrian B in the previous frame image. This represents the coordinate vector of pedestrian A relative to pedestrian B in the current image frame. The tracking of pedestrian detection boxes in the two frames is done by DeepSort. This indicates that the relative positions of pedestrians A and B in the two consecutive frames have not changed, which means that the ID change of the pedestrian detection box is an abnormal match. Therefore, the ID of the current match is discarded, and the ID of the previous frame is taken as true.

[0058] This invention provides an optimal implementation scheme for a pedestrian identity information recognition method based on multi-source information fusion, such as... Figure 5 As shown, each patient is first fitted with a UWB wristband, and their real identity information (ID) is bound to the wristband's identifier. Then, video recording and UWB positioning coordinates are acquired simultaneously. The video images are processed using the YOLOv5 object detection algorithm to obtain the coordinates of the points in the pedestrian detection bounding boxes for each patient, and the DeepSort object tracking algorithm is used to track the pedestrian detection boxes. Next, the UWB / IMU positioning coordinates are timestamped and aligned with the coordinates of the points in the pedestrian detection boxes. Then, the Hungarian algorithm is used to initially match the coordinates of the two modalities, identify mismatches, and finally accurately assign the patient's real information from the UWB wristband to the pedestrians detected in the image, providing reliable pedestrian identity information for downstream behavior analysis tasks.

[0059] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for pedestrian identity information recognition through multi-source information fusion, characterized in that, The method specifically includes the following steps: S1: Data Acquisition: First, a UWB system and cameras need to be deployed indoors to ensure that the images captured by the cameras are parallel to the spatial coordinate system of the UWB positioning system; then, each patient wears a UWB wristband, and the patient's real identity information is bound to the UWB wristband number they wear, while video recording and UWB positioning coordinate acquisition are performed simultaneously. S2: Data Calculation and Analysis: First, the coordinates of the points in the pedestrian detection box for each patient are obtained using the YOLOv5 object detection algorithm based on the acquired video images. The pedestrian detection box is then tracked using the DeepSort object tracking algorithm. Next, the UWB / IMU positioning coordinates are aligned with the coordinates of the points in the pedestrian detection box using timestamps and coordinate systems. Then, the coordinate points of the image and UWB data are initially matched using the Hungarian algorithm, and mismatches are identified. Finally, the real information of the patient on the UWB wristband is accurately assigned to the pedestrians detected in the image, providing reliable pedestrian identity information for downstream behavior analysis tasks. Mismatch identification specifically includes: if the relative positional relationship between any two coordinate points in the image positioning data differs from that in the UWB / IMU positioning data over a period of time, it is considered that there is a drift error in the UWB / IMU system or that the pedestrian wearing the UWB tag has partially out of position. This frame of pedestrian identity binding data is recorded as the previous frame of pedestrian identity binding data. An algorithm based on the temporal features of two-modal positioning data is used to identify mismatches. This represents the coordinate vector of pedestrian A relative to pedestrian B in the previous frame image. This represents the coordinate vector of pedestrian A relative to pedestrian B in the current image frame. The tracking of pedestrian detection boxes in the two frames is done by DeepSort. This indicates that the relative positions of pedestrians A and B in the two consecutive frames have not changed, which means that the ID change of the pedestrian detection box is an abnormal match. Therefore, the ID of the current match is discarded, and the ID of the previous frame is taken as true.

2. The pedestrian identity information recognition method according to claim 1, characterized in that, In step S2, the detection steps of the target detection algorithm Yolov5 model are as follows: First, the input image I is preprocessed to meet the input requirements of the model; the preprocessing includes adjusting the image size to 416×416 and normalizing it. Next, CSPDarknet53 was used as the feature extraction backbone network. In the Focus module: input image Perform block segmentation and convolution operations to reduce information loss: ; In the CSP module: CSP is used across phased partial networks. Perform deeper feature extraction ; In the Neck section, feature fusion is performed using FPN and PAN structures. First, bottom-up feature extraction and top-down feature fusion are achieved through FPN: Then, a bottom-up path is added through PAN to achieve bidirectional feature fusion: ; In the detection head, convolution operations are used to predict the category and location of the target; assuming P is the input feature map, the output of the target detection head is: ,in For activation function, It includes the predicted bounding box location, size, and class probability; finally, it filters out overlapping detection boxes using the non-maximum suppression (NMS) method, retaining only boxes with high confidence. The NMS formula is as follows: in, This represents the selected and retained pedestrian detection boxes in the initial detection set O, where IOU represents the intersection-union ratio of the two bounding boxes, and threshold is the set threshold value; the coordinates of the points within the retained pedestrian detection boxes are recorded. , n This represents the number of pedestrian frames detected.

3. The pedestrian identity information recognition method according to claim 1, characterized in that, In step S2, the UWB / IMU positioning coordinates are aligned with the coordinates of the points in the pedestrian detection box in terms of timestamps and coordinate systems. Specifically, this includes: selecting the UWB positioning data with the smallest time interval with each frame of the image for matching; using the same host to provide timestamps for the video image and the UWB system respectively; first calculating the absolute timestamps of the two modal data; then taking the difference between the timestamps of the two modal data; if the timestamp difference is less than the set threshold, then the two frames of data are used as the registration data at the same time.

4. The pedestrian identity information recognition method according to claim 1, characterized in that, In step S2, the coordinates of the image and UWB data are initially matched using the Hungarian algorithm. Specifically, this includes: before formally collecting data, collecting a series of edge UWB positioning data along the edge of the image captured by the camera to form a closed UWB region corresponding to the image region; after this operation, the UWB coordinates outside the image captured by the camera do not participate in the image-UWB data matching, so that the number of collected UWB data is equal to the number of pedestrian detection boxes in the image. To unify coordinates from two different coordinate systems to the same scale, the transformation ratio must first be calculated: , ,in , The difference between the pixel coordinates on the X and Y axes of the image. , The difference between the real-world coordinates of the X and Y axes in the UWB system; = , = This is used to determine the camera coordinates ( Convert to UWB coordinates ( Consistent proportional coordinates; The Hungarian algorithm is used to match the midpoint of the UWB positioning coordinates obtained at the same time with the midpoint of the pedestrian detection box coordinates obtained from the camera image. First, a set of pedestrian coordinates obtained through UWB is defined. ,in and the set of pedestrian midpoint coordinates obtained through camera image detection boxes. ,in Next, the Euclidean distance between each UWB coordinate and each visual coordinate is calculated and used as an element in the cost matrix C. The specific formula is: The cost matrix 𝐶 is in the form of: Then, the cost matrix The input is fed into the Hungarian algorithm to solve the minimum weight matching problem; the Hungarian algorithm will output a matching relation. This minimizes the total cost; this matching relationship maps UWB coordinates to visual coordinates one-to-one, ensuring that the patient's real information on the UWB wristband is accurately mapped to the pedestrians detected in the image.

Images