Behavior identification method and system based on UWB positioning

Abstract

The invention relates to the technical field of motion behavior recognition, and discloses a behavior recognition method and system based on UWB positioning, and the method comprises the steps: confirming a behavior recognition environment based on a behavior recognition instruction, carrying out the data collection of a plurality of initial positioning targets based on a data collection unit, and obtaining a behavior recognition result; performing time synchronization alignment on the original video stream and the UWB positioning data based on a multi-modal fusion unit, acquiring an image positioning coordinate set and a UWB positioning coordinate set based on the synchronous video stream and the synchronous UWB positioning data, calculating an Euclidean distance based on the image positioning coordinate set and the UWB positioning coordinate set, and performing identity matching based on the Euclidean distance, and feature extraction is performed on the UWB positioning data based on a feature extraction unit and the positioning target ID, and a final behavior score is calculated based on a visual spatial-temporal feature vector and a position feature vector. According to the invention, the accuracy, efficiency and multi-target applicability of behavior recognition can be improved.

Description

Technical Field

[0001] This invention relates to the field of motion behavior recognition technology, and in particular to a behavior recognition method and system based on UWB positioning. Background Technology

[0002] As sports training and competition analysis moves towards digitalization and intelligence, the automated behavior recognition and quantitative evaluation of athletes' training videos are becoming increasingly important.

[0003] Existing technical solutions have significant limitations: purely visual analysis methods struggle to maintain stable identity tracking when athletes are uniformly dressed, occlude each other, or move rapidly, leading to incorrect behavior attribution; while simple UWB localization provides continuous location information, it lacks visual semantics describing action details. Although some multimodal fusion attempts exist, they generally suffer from limitations in data spatiotemporal synchronization accuracy and cross-modal identity consistency. Therefore, a behavior recognition method capable of achieving accurate spatiotemporal alignment, stable identity association, and intelligent weighted fusion is urgently needed. Summary of the Invention

[0004] This invention provides a behavior recognition method and system based on UWB positioning, the main purpose of which is to improve the accuracy, efficiency and multi-target applicability of behavior recognition.

[0005] To achieve the above objectives, the present invention provides a behavior recognition method based on UWB positioning, comprising: The system confirms receipt of a behavior recognition instruction and confirms the behavior recognition environment based on the instruction. The behavior recognition environment includes a behavior recognition system and multiple initial positioning targets. The behavior recognition system includes a data acquisition unit, a multimodal fusion unit, a feature extraction unit, and a recognition and decision-making unit. The data acquisition unit includes a video data acquisition unit and a UWB positioning data acquisition unit. Based on the data acquisition unit, data is acquired from the multiple initial positioning targets to obtain the original video stream and UWB positioning data; Based on the multimodal fusion unit, the original video stream and UWB positioning data are time-synchronized and aligned to obtain synchronized video stream and synchronized UWB positioning data; Based on the synchronous video stream and synchronous UWB positioning data, obtain image positioning coordinate set and UWB positioning coordinate set, calculate Euclidean distance based on the image positioning coordinate set and UWB positioning coordinate set, and perform identity matching based on the Euclidean distance to obtain multiple positioning target IDs; Perform the following operations on all of the aforementioned location target IDs: Based on the feature extraction unit and the positioning target ID, the original video stream is subjected to position feature extraction to obtain a visual spatiotemporal feature vector. Based on the feature extraction unit and the positioning target ID, the UWB positioning data is subjected to feature extraction to obtain a position feature vector. The final behavior score is calculated based on the visual spatiotemporal feature vector and the location feature vector. The behavior is classified based on the final behavior score and the recognition and decision unit to obtain the recognition result. Based on the recognition result, the behavior recognition of multiple initial positioning targets is realized.

[0006] Optionally, the step of acquiring data from the plurality of initial positioning targets based on the data acquisition unit to obtain the original video stream and UWB positioning data includes: A detection area for data acquisition is obtained. Based on a preset frame rate, the video data acquisition unit acquires video images of multiple initial positioning targets in the detection area to obtain multiple video image frames. Timestamps are appended to the multiple video image frames to obtain multiple timestamped image frames, and the original video stream is obtained based on the multiple timestamped image frames. Based on multiple pre-built UWB positioning tags and pre-built UWB base stations, wireless communication connections are established with the multiple initial positioning targets, and the distance between the UWB base station and the multiple initial positioning targets is measured based on the UWB positioning data acquisition unit to obtain multiple measured distances; Add timestamps to the multiple measured distances to obtain multiple timestamp measured distances; By integrating the distance measurements from the multiple timestamps, UWB positioning data is obtained.

[0007] Optionally, the step of performing time synchronization alignment on the original video stream and UWB positioning data based on the multimodal fusion unit to obtain synchronized video stream and synchronized UWB positioning data includes: The original video stream is timestamped based on the multimodal fusion unit to obtain a video stream timestamp sequence, wherein the video stream timestamp sequence includes multiple video stream timestamps arranged in chronological order. A reference timeline is established using the timestamp sequence of the video stream; Multiple UWB timestamps are obtained using the multimodal fusion unit and UWB positioning data, and the multiple UWB timestamps are mapped to the reference time axis to obtain a time sequence correspondence. Based on the aforementioned time-series correspondence, the original video stream and UWB positioning data are synchronized and aligned to obtain synchronized video stream and synchronized UWB positioning data.

[0008] Optionally, the step of obtaining the image positioning coordinate set and the UWB positioning coordinate set based on the synchronized video stream and synchronized UWB positioning data includes: For each of the multiple initial positioning targets, perform the following operation: The position of the initially located target in the image is detected by using a pre-trained target detection model and multiple timestamped image frames in a synchronous video stream, resulting in multiple target detection bounding boxes. An image pixel coordinate system is established based on the synchronous video stream. The pixel coordinates of the initial positioning target in multiple timestamp image frames are determined based on the multiple target detection bounding boxes and the image pixel coordinate system. Multiple positioning target pixel coordinates are obtained. The multiple positioning target pixel coordinates are sorted in order of timetamp from first to last to obtain a pixel position sequence. The initial position movement curve is obtained based on the pixel position sequence and a preset curve fitting algorithm; The overlapping region is detected frame by frame based on multiple target detection bounding boxes and multiple timestamp image frames to obtain multiple bounding box overlap degrees. The multiple bounding box overlap degrees are compared with a preset overlap threshold. If the bounding box overlap degree is greater than the overlap threshold, the pixel coordinates of the corresponding target detection bounding box are determined based on the initial position movement curve to obtain one or more determined pixel coordinates. The position movement curve is obtained based on the one or more determined pixel coordinates and the initial position movement curve. A set of reference point coordinates is obtained based on the position movement curve; a Cartesian coordinate system is established based on the detection area and the UWB base station; the set of reference point coordinates is mapped from the image pixel coordinate system to the Cartesian coordinate system based on the pre-constructed homography matrix to obtain the image positioning coordinate set; Based on the UWB positioning data and the Cartesian coordinate system, the coordinates of the multiple initial positioning targets are determined to obtain a UWB positioning coordinate set.

[0009] Optionally, the step of calculating Euclidean distance based on the image positioning coordinate set and the UWB positioning coordinate set, and performing identity matching based on the Euclidean distance to obtain multiple positioning target IDs, including: The image positioning coordinate set and the UWB positioning coordinate set are purified to obtain an effective image positioning coordinate set and an effective UWB positioning coordinate set, wherein the effective image positioning coordinate set and the effective UWB positioning coordinate set each contain multiple effective image positioning coordinates and effective UWB positioning coordinates. Calculate multiple Euclidean distances based on each valid image positioning coordinate in the set of valid image positioning coordinates and each valid UWB positioning coordinate in the set of valid UWB positioning coordinates, and obtain a two-dimensional distance matrix based on the multiple Euclidean distances. Based on the two-dimensional distance matrix and the pre-constructed matching algorithm, the matching relationship is solved, and multiple successful matching pairs are obtained; Each of the multiple successful matching pairs is assigned a unique identifier to obtain multiple location target IDs.

[0010] Optionally, the step of extracting location features from the original video stream based on the feature extraction unit and the positioning target ID to obtain a visual spatiotemporal feature vector includes: Based on the location target ID, 64 video frames are extracted from the original video stream. The feature extraction unit then performs the following operations on these 64 video frames: The 64 frames of video images are sampled based on a preset first sampling interval to obtain a first sampling sequence; The 64 frames of video images are sampled based on a preset second sampling interval to obtain a second sampling sequence; The first sampling sequence is input into a pre-constructed first three-dimensional convolutional neural network branch for spatiotemporal feature extraction to obtain the first feature map; The second sampling sequence is input into a pre-constructed second three-dimensional convolutional neural network branch for spatiotemporal feature extraction to obtain the second feature map; The second feature map is transformed in the time dimension, and the transformed second feature map is concatenated with the first feature map to obtain a fused visual feature map; A local feature map is obtained by extracting the feature map corresponding to the localized target ID from the fused visual feature map using a pre-constructed alignment algorithm. The local feature map is subjected to global average pooling to obtain the visual spatiotemporal feature vector.

[0011] Optionally, the step of extracting features from the UWB positioning data based on the feature extraction unit and the positioning target ID to obtain a location feature vector includes: Based on the UWB positioning data, the UWB coordinates associated with the positioning target ID are extracted sequentially according to the timestamp from first to last to obtain a UWB coordinate sequence, wherein the UWB coordinate sequence contains multiple UWB coordinates corresponding to different timestamps; Based on the feature extraction unit, the following operations are performed on each UWB coordinate point in the UWB coordinate sequence: Obtain multiple reference positioning target IDs, obtain multiple reference coordinate points based on the multiple reference positioning target IDs and the timestamps corresponding to the UWB coordinate points, and calculate the first relative distance vector based on the UWB coordinate points and the multiple reference points; The second relative distance vector is calculated based on UWB coordinate points and the coordinates of multiple preset fixed key points; The first relative distance vector and the second relative distance vector are merged and normalized to obtain the merged relative position vector. Arrange the merged relative position vectors corresponding to all timestamps in the UWB coordinate sequence in chronological order to obtain a relative position vector sequence. The relative position vector sequence is subjected to dimensionality reduction processing to obtain position feature vectors.

[0012] Optionally, the step of calculating a final behavior score based on the visual spatiotemporal feature vector and the location feature vector, and performing behavior classification based on the final behavior score and the recognition and decision-making unit to obtain a recognition result includes: The visual spatiotemporal feature vector is input into a pre-constructed fully connected layer to obtain a visual branch behavior score set, wherein the visual branch behavior score set includes i visual branch behavior scores. The location feature vector is input into a pre-constructed memory network to obtain a UWB branch behavior score set, wherein the UWB branch behavior score set includes i UWB branch behavior scores; Obtain the visual branch weight coefficients and UWB branch weight coefficients. Calculate the final score for the behavior based on the visual branch weight coefficients, UWB branch weight coefficients, visual branch behavior score set, and UWB branch behavior score set. The calculation formula is as follows:

[0013] in, Indicates the index value. The line with index i is the final score. This represents the visual branch behavior score for index i. This indicates the score of the UWB branch with index value i. Indicates the visual branch weight coefficient. Indicates the UWB branch weight coefficient; The highest final score is obtained based on the final score of the aforementioned behavior; The highest final score is mapped using a preset mapping relationship and an identification and decision-making unit to obtain the identification result.

[0014] Optionally, obtaining the visual branch weight coefficients and UWB branch weight coefficients includes: Obtain the current positioning target ID, and obtain the UWB positioning coordinate sequence based on the current positioning target ID, wherein the UWB positioning coordinate sequence includes multiple UWB positioning coordinates arranged in order of timestamp from first to last; The first positioning coordinate is obtained by arbitrarily extracting UWB positioning coordinates from the UWB positioning coordinate sequence; Based on the first positioning coordinate, the adjacent positioning coordinates are identified in the UWB positioning coordinates. The absolute difference between the timestamp corresponding to the first positioning coordinate and the timestamp corresponding to the adjacent positioning coordinate is calculated to obtain the time interval. The instantaneous velocity and instantaneous acceleration are calculated based on adjacent UWB positioning coordinates and time intervals in the UWB positioning coordinate sequence. Based on the UWB positioning coordinate sequence, obtain reference UWB positioning coordinates, calculate multiple Euclidean distances using the reference UWB positioning coordinates and multiple fixed key point coordinates, and take the minimum value among the multiple Euclidean distances as the minimum landmark distance; The visual branch weight coefficient and the UWB branch weight coefficient are calculated based on the instantaneous motion velocity, instantaneous acceleration, and minimum landmark distance. The calculation formula is as follows: in, , These represent the visual branch weight coefficient and the UWB branch weight coefficient, respectively. and These represent the initial weight coefficients, , , , and Indicates an adjustable parameter. For the Sigmoid function, Represents the hyperbolic tangent function. Represents an exponential function. Indicates instantaneous velocity. Indicates instantaneous acceleration. Indicates the minimum distance to a landmark.

[0015] To achieve the above objectives, the present invention also provides a behavior recognition system based on UWB positioning, comprising: An environment confirmation module is used to confirm the received behavior recognition command and confirm the behavior recognition environment based on the behavior recognition command. The behavior recognition environment includes a behavior recognition system and multiple initial positioning targets. The behavior recognition system includes a data acquisition unit, a multimodal fusion unit, a feature extraction unit, and a recognition and decision-making unit. The data acquisition unit includes a video data acquisition unit and a UWB positioning data acquisition unit. The data acquisition module is used to acquire data from the multiple initial positioning targets based on the data acquisition unit to obtain the original video stream and UWB positioning data; Based on the multimodal fusion unit, the original video stream and UWB positioning data are time-synchronized and aligned to obtain synchronized video stream and synchronized UWB positioning data; The feature extraction module is used to obtain image positioning coordinate sets and UWB positioning coordinate sets based on the synchronous video stream and synchronous UWB positioning data, calculate Euclidean distance based on the image positioning coordinate sets and UWB positioning coordinate sets, and perform identity matching based on the Euclidean distance to obtain multiple positioning target IDs. Perform the following operations on all of the aforementioned location target IDs: Based on the feature extraction unit and the positioning target ID, the original video stream is subjected to position feature extraction to obtain a visual spatiotemporal feature vector. Based on the feature extraction unit and the positioning target ID, the UWB positioning data is subjected to feature extraction to obtain a position feature vector. The identification and decision module is used to calculate the final behavior score based on the visual spatiotemporal feature vector and the location feature vector, perform behavior classification based on the final behavior score and the identification and decision unit, obtain the identification result, and realize the behavior identification of multiple initial positioning targets based on the identification result.

[0016] To address the above problems, the present invention also provides an electronic device, the electronic device comprising: Memory, storing at least one instruction; The processor executes the instructions stored in the memory to implement the behavior recognition method based on UWB positioning described above.

[0017] To address the aforementioned problems, the present invention also provides a computer-readable storage medium storing at least one instruction, which is executed by a processor in an electronic device to implement the behavior recognition method based on UWB positioning described above.

[0018] To address the problems described in the background art, this invention confirms the receipt of behavior recognition instructions and, based on these instructions, confirms the behavior recognition environment. The behavior recognition environment includes a behavior recognition system and multiple initial positioning targets. The behavior recognition system includes a data acquisition unit, a multimodal fusion unit, a feature extraction unit, and a recognition and decision-making unit. The data acquisition unit includes a video data acquisition unit and a UWB positioning data acquisition unit. Therefore, this invention considers the real-time requirements of complex multi-target scenarios during behavior recognition. Thus, by confirming the behavior recognition environment, it ensures modular system integration, providing a reliable foundation for subsequent multimodal data processing. This improves the overall adaptability and robustness of behavior recognition. Based on the data acquisition unit, data is acquired from the multiple initial positioning targets to obtain the original video stream and UWB positioning data. It is evident that this invention employs a dual-source data acquisition mechanism, combining video and UWB positioning to capture dynamic information, avoiding the limitations of a single modality, and thus laying a comprehensive data foundation for fusion analysis. Based on the multimodal fusion unit, the original video stream and UWB positioning data are time-synchronized and aligned to obtain synchronized video stream and synchronized UWB positioning data. It is evident that this embodiment of the invention introduces a time synchronization step to eliminate data timing deviations, thereby improving the synergy and accuracy of multimodal data. The invention obtains image positioning coordinate sets and UWB positioning coordinate sets from synchronized video streams and synchronized UWB positioning data. Based on these coordinate sets, Euclidean distance is calculated, and identity matching is performed to obtain multiple positioning target IDs. This invention achieves multi-target identity association through coordinate matching, avoiding identity confusion and thus improving tracking accuracy in multi-target scenarios. For each of the multiple positioning target IDs, the following operations are performed: positional feature extraction is performed on the original video stream based on the feature extraction unit and the positioning target IDs to obtain a visual spatiotemporal feature vector; and the UWB positioning data is then analyzed based on the feature extraction unit and the positioning target IDs. The data undergoes feature extraction to obtain a location feature vector. This invention performs bimodal feature extraction for each ID, quantifying visual and location dynamics to ensure the richness and specificity of the feature representation. Based on the visual spatiotemporal feature vector and the location feature vector, a final behavior score is calculated. Based on the final behavior score and the recognition and decision-making unit, behavior classification is performed to obtain the recognition result. Based on the recognition result, behavior recognition for multiple initial positioning targets is achieved. This invention integrates feature calculation, score setting, and classification to form a closed-loop decision-making mechanism. Furthermore, multimodal fusion enhances classification reliability. Therefore, this invention can improve the accuracy, efficiency, and multi-target applicability of behavior recognition. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating a behavior recognition method based on UWB positioning provided in an embodiment of the present invention. Figure 2 This is a functional block diagram of a behavior recognition system based on UWB positioning provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an electronic device that implements the behavior recognition method based on UWB positioning according to an embodiment of the present invention.

[0020] Explanation of reference numerals in the attached figures: 10. Electronic device; 11. Processor; 12. Memory; 13. Bus.

[0021] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0022] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0023] This application provides a behavior recognition method based on UWB positioning. The execution entity of the UWB-based behavior recognition method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the UWB-based behavior recognition method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster.

[0024] Reference Figure 1 The diagram shown is a flowchart illustrating a behavior recognition method based on UWB positioning according to an embodiment of the present invention. In this embodiment, the behavior recognition method based on UWB positioning includes: S1. Confirm receipt of behavior recognition instruction, and confirm behavior recognition environment based on behavior recognition instruction. The behavior recognition environment includes behavior recognition system and multiple initial positioning targets. The behavior recognition system includes data acquisition unit, multimodal fusion unit, feature extraction unit and recognition and decision unit. The data acquisition unit includes video data acquisition unit and UWB positioning data acquisition unit.

[0025] It should be explained that the behavior recognition instruction refers to the instruction issued by the person who wants to achieve behavior recognition; the behavior recognition environment refers to the necessary environment for achieving behavior recognition; the behavior system refers to a system capable of achieving behavior recognition; and the behavior recognition system includes a data acquisition unit, a multimodal fusion unit, a feature extraction unit, and a recognition and decision-making unit. The data acquisition unit includes a video data acquisition unit and a UWB positioning data acquisition unit. For specific applications, please refer to the following embodiments. The initial positioning target refers to the target object for behavior recognition by the behavior recognition system, such as an athlete on a basketball court. The purpose of this invention is to improve the accuracy of behavior recognition in different scenarios. In this invention, UWB is an abbreviation for Ultra Wide Band, which is a short-range wireless communication technology applied to Wireless Personal Area Networks (WPANs).

[0026] For example, Zhang is a referee at a basketball game. In order to improve the accuracy of behavior recognition in this scenario, Zhang issued a behavior recognition instruction and confirmed the behavior recognition environment.

[0027] S2. Based on the data acquisition unit, data is acquired from the multiple initial positioning targets to obtain the original video stream and UWB positioning data.

[0028] Furthermore, the step of acquiring data from the plurality of initial positioning targets based on the data acquisition unit to obtain the original video stream and UWB positioning data includes: A detection area for data acquisition is obtained. Based on a preset frame rate, the video data acquisition unit acquires video images of multiple initial positioning targets in the detection area to obtain multiple video image frames. Timestamps are appended to the multiple video image frames to obtain multiple timestamped image frames, and the original video stream is obtained based on the multiple timestamped image frames. Based on multiple pre-built UWB positioning tags and pre-built UWB base stations, wireless communication connections are established with the multiple initial positioning targets, and the distance between the UWB base station and the multiple initial positioning targets is measured based on the UWB positioning data acquisition unit to obtain multiple measured distances; Add timestamps to the multiple measured distances to obtain multiple timestamp measured distances; By integrating the distance measurements from the multiple timestamps, UWB positioning data is obtained.

[0029] It should be understood that the data acquisition unit refers to a functional module for data acquisition, including a video data acquisition unit and a UWB positioning data acquisition unit. The detection area refers to the area where the initial positioning target is active and where data acquisition is performed, such as a basketball court. The method of acquiring video images of multiple initial positioning targets in the detection area refers to using the video data acquisition unit to continuously capture images of the detection area at a preset fixed frame rate (e.g., 25 frames per second). The video image acquisition system refers to the functional sub-module in the data acquisition unit used for image acquisition. Optionally, a high-resolution camera can be used as the image acquisition unit. The multiple video image frames refer to multiple static images acquired by the video data acquisition unit. The method of adding timestamps to the multiple video image frames refers to obtaining a corresponding timestamp representing the completion time of the frame image acquisition from a clock source synchronized with the video data acquisition unit, and binding the timestamp to the frame image data. The multiple timestamped image frames refer to multiple video image frames that have been appended with corresponding timestamps. The method for obtaining the original video stream based on the multiple timestamped image frames refers to combining the multiple timestamped image frames in order of time indicated by their respective timestamps to obtain the original video stream. The original video stream refers to the video stream composed of the above-mentioned multiple timestamped image frames. The method of establishing wireless communication connections with the multiple initial positioning targets and measuring the distance between the UWB base station and the multiple initial positioning targets based on the UWB positioning data acquisition unit refers to the UWB positioning tag continuously emitting wireless pulse signals. After receiving the signals, the UWB base station deployed around the detection area calculates the distance between the UWB base station and the initial positioning targets by multiplying the known propagation speed of the signal by the propagation time. The UWB positioning tag and the UWB base station refer to a movable beacon worn by the athlete, having a unique identification and actively emitting ultra-wideband wireless signals, and a signal transceiver deployed in the detection area, used to receive tag signals and accurately measure the straight-line distance between the beacon and each tag. Both the UWB positioning tag and the UWB base station are based on short-range wireless communication technology of Wireless Personal Area Network (WPAN). The multiple measured distances refer to the distances measured between different UWB positioning tags and the UWB base station at different times, which can be represented by data tuples (UWB positioning tag, UWB base station, distance value). The method of adding timestamps to the multiple measured distances refers to adding a timestamp representing the time when the measurement was completed to each of the multiple measured distances. The UWB positioning data refers to a sequence set obtained by arranging the multiple timestamped measured distances in chronological order of the timestamps.

[0030] S3. Based on the multimodal fusion unit, the original video stream and UWB positioning data are time-synchronized and aligned to obtain synchronized video stream and synchronized UWB positioning data.

[0031] It should be explained that the step of performing time synchronization alignment on the original video stream and UWB positioning data based on the multimodal fusion unit to obtain synchronized video stream and synchronized UWB positioning data includes: The original video stream is timestamped based on the multimodal fusion unit to obtain a video stream timestamp sequence, wherein the video stream timestamp sequence includes multiple video stream timestamps arranged in chronological order. A reference timeline is established using the timestamp sequence of the video stream; Multiple UWB timestamps are obtained using the multimodal fusion unit and UWB positioning data, and the multiple UWB timestamps are mapped to the reference time axis to obtain a time sequence correspondence. Based on the aforementioned time-series correspondence, the original video stream and UWB positioning data are synchronized and aligned to obtain synchronized video stream and synchronized UWB positioning data.

[0032] Furthermore, the time synchronization alignment method refers to using a multimodal fusion unit to unify the timing information of the original video stream and UWB positioning data onto the same time reference, thereby eliminating deviations caused by acquisition delays or clock drift and ensuring the accuracy of subsequent analysis. The multimodal fusion unit refers to a functional module in the behavior recognition system used to integrate different modal data and ensure data consistency through operations such as time synchronization. The method for extracting timestamps from the original video stream involves reading additional timestamps from each timestamped image frame in the original video stream and arranging them in chronological order to form a sequence. The video stream timestamp sequence refers to an ordered set of time points representing the acquisition time of the video frames, used as a synchronization reference. The method for establishing a reference timeline using the video stream timestamp sequence involves constructing a continuous timeline as a unified reference framework for the multimodal data, starting from the earliest timestamp in the video stream timestamp sequence. Optionally, linear interpolation methods can be used to fill blank points on the timeline. The method for obtaining multiple UWB timestamps using the multimodal fusion unit and UWB positioning data refers to extracting the timestamp from the distance measured by each timestamp in the UWB positioning data. The multiple UWB timestamps refer to multiple UWB data acquisition times. The method for mapping the multiple UWB timestamps to the reference time axis refers to calculating the corresponding position for each UWB timestamp through interpolation to obtain a temporal correspondence. This temporal correspondence refers to the temporal matching and mapping between UWB data and video data. The method for synchronizing and aligning the original video stream and UWB positioning data based on the temporal correspondence refers to adjusting the timing of the UWB data (e.g., resampling) according to the correspondence to align the UWB data with the video stream in time, resulting in synchronized video stream and synchronized UWB positioning data. The synchronized video stream and synchronized UWB positioning data refer to the aligned video stream and UWB data sequence, respectively, where each data point corresponds to the same time point.

[0033] S4. Based on the synchronized video stream and synchronized UWB positioning data, obtain the image positioning coordinate set and UWB positioning coordinate set, calculate the Euclidean distance based on the image positioning coordinate set and UWB positioning coordinate set, and perform identity matching based on the Euclidean distance to obtain multiple positioning target IDs.

[0034] It should be understood that the acquisition of the image positioning coordinate set and the UWB positioning coordinate set based on the synchronized video stream and synchronized UWB positioning data includes: For each of the multiple initial positioning targets, perform the following operation: The position of the initially located target in the image is detected by using a pre-trained target detection model and multiple timestamped image frames in a synchronous video stream, resulting in multiple target detection bounding boxes. An image pixel coordinate system is established based on the synchronous video stream. The pixel coordinates of the initial positioning target in multiple timestamp image frames are determined based on the multiple target detection bounding boxes and the image pixel coordinate system. Multiple positioning target pixel coordinates are obtained. The multiple positioning target pixel coordinates are sorted in order of timetamp from first to last to obtain a pixel position sequence. The initial position movement curve is obtained based on the pixel position sequence and a preset curve fitting algorithm; The overlapping region is detected frame by frame based on multiple target detection bounding boxes and multiple timestamp image frames to obtain multiple bounding box overlap degrees. The multiple bounding box overlap degrees are compared with a preset overlap threshold. If the bounding box overlap degree is greater than the overlap threshold, the pixel coordinates of the corresponding target detection bounding box are determined based on the initial position movement curve to obtain one or more determined pixel coordinates. The position movement curve is obtained based on the one or more determined pixel coordinates and the initial position movement curve. A set of reference point coordinates is obtained based on the position movement curve; a Cartesian coordinate system is established based on the detection area and the UWB base station; the set of reference point coordinates is mapped from the image pixel coordinate system to the Cartesian coordinate system based on the pre-constructed homography matrix to obtain the image positioning coordinate set; Based on the UWB positioning data and the Cartesian coordinate system, the coordinates of the multiple initial positioning targets are determined to obtain a UWB positioning coordinate set.

[0035] It should be explained that the method of detecting the positions of multiple initially located targets in the image frame by frame refers to using a target detection model to perform target detection on each frame of the synchronized video stream, identifying the bounding boxes of multiple initially located targets. Optionally, YOLO or Faster R-CNN models can be used as the target detection model. The multiple target detection bounding boxes refer to the rectangular boxes surrounding the targets, used to represent the position and size of the targets in the image. The method of establishing an image pixel coordinate system based on the synchronized video stream refers to establishing a two-dimensional coordinate system with the upper left corner of the video frame as the origin. The image pixel coordinate system refers to the coordinate frame used to describe the position of pixels in the image.The method for determining the pixel coordinates of the initial positioning target in multiple timestamp image frames based on the multiple target detection bounding boxes and the image pixel coordinate system refers to extracting the coordinates of a representative point (such as the center point of the bounding box) from each target detection bounding box and using it as the pixel coordinates of the positioning target. The method for sorting the multiple positioning target pixel coordinates in chronological order of timestamps refers to sorting the positioning target pixel coordinates in chronological order of timestamps of the synchronized video stream. The pixel position sequence refers to the sequence of multiple positioning target pixel coordinates obtained in chronological order of timestamps. The method for obtaining the initial position movement curve based on the pixel position sequence and a preset curve fitting algorithm refers to performing continuous curve fitting on each pixel coordinate point based on the curve fitting algorithm to generate the initial position movement curve. Optionally, a Bezier curve fitting algorithm can be used as the curve fitting algorithm. The initial position movement curve refers to a curve that can characterize the position change trend of the initial positioning target in the image in the image pixel coordinate system. The method for detecting overlapping regions frame by frame based on multiple target detection bounding boxes and multiple timestamp image frames refers to checking whether there are pixel overlap regions between the target detection bounding box of the current initial positioning target and the bounding boxes of other initial positioning targets, and calculating the overlapping regions. The ratio of the pixel area to the area of ​​the union of the two bounding boxes is used as the bounding box overlap. The bounding box overlap is an indicator that quantifies the degree of overlap between bounding boxes. The overlap threshold is a preset threshold for overlap, such as 0.5. If the bounding box overlap is greater than the overlap threshold, it indicates that the target in that frame is occluded, and its initial detected bounding box coordinates are unreliable. They need to be recalculated based on the initial position movement curve. The method for recalculating the pixel coordinates of the corresponding target detection bounding box based on the initial position movement curve means that if the bounding box overlap is greater than the overlap threshold, it indicates that the initially located target being detected is obscured by other bounding boxes. Initial target occlusion is addressed by performing feature recognition on the initially detected target based on the target detection model, such as clothing color, head outline, jersey number, and skin color. Based on these features, an updated target detection bounding box is re-detected for the initially detected target, and the updated target pixel coordinates are reconfirmed based on the updated target detection bounding box. Finally, a curve fitting algorithm and the updated target pixel coordinates are used for refitting to obtain the proposed pixel coordinates. The methods for obtaining the updated target detection bounding box and the updated target pixel coordinates are similar to those for obtaining the target detection bounding box and the target pixel coordinates, and will not be elaborated here.The proposed pixel coordinates refer to the positioning target pixel coordinates represented by the target detection bounding box, which are re-proposed by the curve fitting algorithm. The method of obtaining the position movement curve based on the one or more proposed pixel coordinates and the initial position movement curve refers to removing the original positioning target pixel coordinates of the target detection bounding box and replacing them with the corresponding proposed pixel coordinates. The position movement curve refers to the initial position movement curve after eliminating the position deviation caused by occlusion. The method of obtaining the reference point coordinate set based on the position movement curve refers to extracting the pixel coordinates corresponding to each timestamp in the synchronous video stream from the position movement curve of each initial positioning target among multiple initial positioning targets, and integrating these coordinates in timestamp order to form a reference point coordinate set. The reference point coordinate set refers to the set of position points of multiple initial positioning targets in the image coordinate system. The method of establishing a Cartesian coordinate system based on the detection area and the UWB base station refers to establishing a two-dimensional Cartesian coordinate system with a fixed point in the detection area (such as the location of the UWB base station) as the origin, where the x-axis and y-axis correspond to the actual physical direction, and the unit is meters. The Cartesian coordinate system refers to the coordinate frame used to describe the actual position within the detection area. The method of mapping the reference point coordinate set from the image pixel coordinate system to the Cartesian coordinate system based on the pre-constructed homography matrix refers to using a homography matrix (a 3x3 transformation matrix) pre-calculated through camera calibration to perform perspective transformation on the reference point coordinates, realizing the conversion from image coordinates to the actual coordinates within the detection area, and obtaining an image positioning coordinate set. The image positioning coordinate set refers to the set of target positions in the Cartesian coordinate system. The pre-constructed homography matrix is ​​a transformation matrix calculated using at least four known corresponding point pairs, used to correct camera distortion and viewing angle tilt. The method of performing coordinate positioning of the multiple initial positioning targets based on the UWB positioning data and the Cartesian coordinate system refers to using the measured distance in the UWB positioning data, and calculating the coordinates of each initial positioning target in the Cartesian coordinate system through polygonal measurement or TDOA (Time Difference of Arrival) algorithm, to obtain a UWB positioning coordinate set. The UWB positioning coordinate set refers to the set of actual target position coordinates calculated based on UWB data.

[0036] Furthermore, the step of calculating Euclidean distance based on the image positioning coordinate set and the UWB positioning coordinate set, and performing identity matching based on the Euclidean distance, yields multiple positioning target IDs, including: The image positioning coordinate set and the UWB positioning coordinate set are purified to obtain an effective image positioning coordinate set and an effective UWB positioning coordinate set, wherein the effective image positioning coordinate set and the effective UWB positioning coordinate set each contain multiple effective image positioning coordinates and effective UWB positioning coordinates. Calculate multiple Euclidean distances based on each valid image positioning coordinate in the set of valid image positioning coordinates and each valid UWB positioning coordinate in the set of valid UWB positioning coordinates, and obtain a two-dimensional distance matrix based on the multiple Euclidean distances. Based on the two-dimensional distance matrix and the pre-constructed matching algorithm, the matching relationship is solved, and multiple successful matching pairs are obtained; Each of the multiple successful matching pairs is assigned a unique identifier to obtain multiple location target IDs.

[0037] It should be understood that the method of purifying the image positioning coordinate set and the UWB positioning coordinate set refers to removing outliers. For example, a threshold is set based on the range of the basketball court plane rectangular coordinate system (e.g., X-axis [0,28] meters, Y-axis [0,15] meters) to filter out all outlier coordinates (e.g., coordinates (30,32)) that exceed the physical area, ultimately retaining the effective image positioning coordinate set and the effective UWB positioning coordinate set. The effective image positioning coordinate set and the effective UWB positioning coordinate set refer to the sets of effective image positioning coordinates and effective UWB positioning coordinates obtained after the filtering operation, respectively, to improve matching accuracy. The method of calculating multiple Euclidean distances based on each effective image positioning coordinate in the effective image positioning coordinate set and each effective UWB positioning coordinate in the effective UWB positioning coordinate set refers to calculating the Euclidean distance between each effective image positioning coordinate in the effective image positioning coordinate set and all effective UWB positioning coordinates in the effective UWB positioning coordinate set, obtaining multiple Euclidean distances. The multiple Euclidean distances refer to all possible pairing distance values ​​between the two coordinate sets. The method for obtaining a two-dimensional distance matrix based on the multiple Euclidean distances refers to organizing these distances into an M (number of coordinates in the effective image positioning coordinate set) × N (number of coordinates in the UWB coordinate set) matrix, where rows correspond to effective image positioning coordinates in the effective image positioning coordinate set, columns correspond to effective UWB positioning coordinates in the UWB coordinate set, and matrix elements are the calculated Euclidean distances. The two-dimensional distance matrix refers to structured data used to represent all paired distances, i.e., the M × N matrix. The method for solving the matching relationship based on the two-dimensional distance matrix and a pre-built matching algorithm refers to using a matching algorithm to find the unique matching pair with the minimum distance in the matrix, ensuring that each image coordinate matches a UWB coordinate, resulting in multiple successful matching pairs. These multiple successful matching pairs refer to successfully associated image-UWB coordinate pairs. Optionally, the Hungarian algorithm can be used as the matching algorithm. The method for assigning unique identifiers to each of the multiple successful matching pairs refers to generating a unique ID (such as an incrementing integer) for each matching pair. The multiple positioning target IDs are identifiers used to uniquely identify each positioning target for subsequent feature extraction and behavior recognition. For example, ID1 corresponds to athlete A, and ID2 corresponds to athlete B.

[0038] S5. Based on the feature extraction unit and the positioning target ID, perform position feature extraction on the original video stream to obtain a visual spatiotemporal feature vector. Based on the feature extraction unit and the positioning target ID, perform feature extraction on the UWB positioning data to obtain a position feature vector.

[0039] It should be explained that the step of extracting positional features from the original video stream based on the feature extraction unit and the positioning target ID to obtain a visual spatiotemporal feature vector includes: Based on the location target ID, 64 video frames are extracted from the original video stream. The feature extraction unit then performs the following operations on these 64 video frames: The 64 frames of video images are sampled based on a preset first sampling interval to obtain a first sampling sequence; The 64 frames of video images are sampled based on a preset second sampling interval to obtain a second sampling sequence; The first sampling sequence is input into a pre-constructed first three-dimensional convolutional neural network branch for spatiotemporal feature extraction to obtain the first feature map; The second sampling sequence is input into a pre-constructed second three-dimensional convolutional neural network branch for spatiotemporal feature extraction to obtain the second feature map; The second feature map is transformed in the time dimension, and the transformed second feature map is concatenated with the first feature map to obtain a fused visual feature map; A local feature map is obtained by extracting the feature map corresponding to the localized target ID from the fused visual feature map using a pre-constructed alignment algorithm. The local feature map is subjected to global average pooling to obtain the visual spatiotemporal feature vector.

[0040] Furthermore, the method of extracting 64 video frames from the original video stream based on the positioning target ID refers to selecting 64 consecutive frames from the original video stream according to the positioning target ID. These frames contain the target corresponding to the positioning target ID and are used for subsequent feature extraction. The 64 video frames refer to fixed-length video segments to capture the dynamic behavior of the target. The feature extraction unit refers to a functional module that can receive the positioning target ID from the multimodal fusion unit and extract feature vectors representing the visual dynamics and spatial location patterns of the target individual from the original video stream and UWB positioning data in parallel. The method of sampling the 64 video frames based on a preset first sampling interval to obtain a first sampling sequence refers to using a larger sampling interval (e.g., sampling once every 4 frames) to extract fewer frames from the 64 frames to form a first sampling sequence. This first sampling sequence refers to a low frame rate sequence used to capture slow-moving motion. The method of sampling the 64 frames of video images based on a preset second sampling interval to obtain a second sampling sequence refers to extracting a large number of frames using a smaller sampling interval (e.g., sampling once every 2 frames) to form a second sampling sequence. The second sampling sequence refers to a high frame rate sequence used to capture fast-moving motion. The method of inputting the first sampling sequence into a pre-constructed first 3D convolutional neural network branch for spatiotemporal feature extraction refers to processing the first sampling sequence using a 3D CNN (e.g., the Slow branch of a SlowFast network) to extract spatiotemporal features. The first feature map refers to the feature tensor representing slow spatiotemporal semantic information output after processing the first sampling sequence using the first 3D convolutional neural network branch. The first 3D convolutional neural network branch refers to a pre-trained 3D CNN model used to process low frame rate inputs. The method of inputting the second sampling sequence into a pre-constructed second 3D convolutional neural network branch for spatiotemporal feature extraction to obtain a second feature map refers to processing the second sampling sequence using a 3D CNN (e.g., the Fast branch of a SlowFast network) to extract spatiotemporal features. The second feature map refers to a multidimensional tensor representing fast spatiotemporal semantic information. The pre-constructed second 3D convolutional neural network branch refers to a pre-trained 3D CNN model used to process high frame rate input. The method of transforming the second feature map in terms of time dimension involves compressing the time dimension of the second feature map to match that of the first feature map using a 3D convolution with a stride greater than 1, thus making them compatible. The method of concatenating the time-transformed second feature map with the first feature map involves using a tensor concatenation function to concatenate the second feature map with the first feature map along the channel dimension, obtaining a fused visual feature map. Optionally, the torch.cat function under the PyTorch framework can be used as the tensor concatenation function. The fused visual feature map refers to a comprehensive feature map obtained by integrating feature maps corresponding to slow and fast spatiotemporal semantic information.The method of extracting the feature map corresponding to the localized target ID from the fused visual feature map using a pre-constructed alignment algorithm refers to using the alignment algorithm to focus on the region where the target corresponding to the localized target ID is located. Optionally, the ROI Align algorithm can be used as the alignment algorithm. The local feature map refers to the target region features corresponding to the localized target ID obtained by cropping and resampling from the fused visual feature map using the alignment algorithm. The method of performing global average pooling on the local feature map to obtain a visual spatiotemporal feature vector refers to calculating the average value in the spatial and temporal dimensions and compressing it into a one-dimensional vector. The visual spatiotemporal feature vector is a one-dimensional mathematical vector obtained after the above pooling operation, which serves as the input to the subsequent fully connected layer, providing data support for subsequent recognition of specific behaviors.

[0041] It should be understood that the step of extracting features from the UWB positioning data based on the feature extraction unit and the positioning target ID to obtain a location feature vector includes: Based on the UWB positioning data, the UWB coordinates associated with the positioning target ID are extracted sequentially according to the timestamp from first to last to obtain a UWB coordinate sequence, wherein the UWB coordinate sequence contains multiple UWB coordinates corresponding to different timestamps; Based on the feature extraction unit, the following operations are performed on each UWB coordinate point in the UWB coordinate sequence: Obtain multiple reference positioning target IDs, obtain multiple reference coordinate points based on the multiple reference positioning target IDs and the timestamps corresponding to the UWB coordinate points, and calculate the first relative distance vector based on the UWB coordinate points and the multiple reference points; The second relative distance vector is calculated based on UWB coordinate points and the coordinates of multiple preset fixed key points; The first relative distance vector and the second relative distance vector are merged and normalized to obtain the merged relative position vector. Arrange the merged relative position vectors corresponding to all timestamps in the UWB coordinate sequence in chronological order to obtain a relative position vector sequence. The relative position vector sequence is subjected to dimensionality reduction processing to obtain position feature vectors.

[0042] It should be explained that the method of extracting UWB coordinates associated with the positioning target ID sequentially according to the timestamp of the UWB positioning data refers to filtering UWB positioning data according to the positioning target ID, extracting the coordinates of the corresponding timestamp, and sorting them in chronological order to form a sequence. The UWB coordinate sequence refers to the position trajectory sequence of the target (such as a basketball player) corresponding to the positioning target ID. The multiple reference positioning target IDs refer to multiple other positioning target IDs besides the positioning target ID. The method of obtaining multiple reference coordinate points based on multiple reference positioning target IDs and the timestamps corresponding to the UWB coordinate points refers to using the timestamp carried by the currently processed UWB coordinate point as the retrieval method to find the UWB coordinate points recorded at the same timestamp from the UWB positioning data corresponding to the multiple reference positioning target IDs. The method of calculating the first relative distance vector based on the UWB coordinate points of the UWB coordinate points and the UWB coordinate points corresponding to other positioning target IDs refers to calculating the Euclidean distance between the coordinate points of the current timestamp and the coordinate points of other target points to form a vector. The first relative distance vector is a vector representing the relative position between targets. The method for calculating the second relative distance vector based on UWB coordinates and multiple preset fixed key point coordinates refers to calculating the distance between the current coordinate point and a fixed point (such as a site marker or UWB base station) to form a vector. The second relative distance vector represents the relative position of the target and environmental key points. The multiple fixed key point coordinates refer to the coordinates of preset static reference points within the detection area, such as the position of the basketball hoop or the position of the UWB base station. The method for merging the first and second relative distance vectors and performing normalization processing refers to connecting the two vectors in a dimension and applying a min-max or z-score normalization algorithm for normalization. The merged relative position vector refers to the comprehensive feature vector representing the spatial positional relationship of the positioning target ID obtained after normalization. The method for arranging the merged relative position vectors corresponding to all timestamps in the UWB coordinate sequence in chronological order refers to sorting and combining all vectors in chronological order. The relative position vector sequence refers to a temporally sequenced position feature sequence. The method for dimensionality reduction of the relative position vector sequence refers to using PCA or Autoencoder to reduce the relative position vector sequence into a one-dimensional vector. The position feature vector refers to the one-dimensional vector obtained after the above dimensionality reduction process, which is used as the input of the subsequent memory network for subsequent visual feature fusion.

[0043] S6. Calculate the final behavior score based on the visual spatiotemporal feature vector and the position feature vector, classify the behavior based on the final behavior score and the recognition and decision unit, and obtain the recognition result. Based on the recognition result, the behavior recognition of multiple initial positioning targets is realized.

[0044] Furthermore, the step of calculating a final behavior score based on the visual spatiotemporal feature vector and the location feature vector, and then performing behavior classification based on the final behavior score and the recognition and decision-making unit to obtain a recognition result includes: The visual spatiotemporal feature vector is input into a pre-constructed fully connected layer to obtain a visual branch behavior score set, wherein the visual branch behavior score set includes i visual branch behavior scores. The location feature vector is input into a pre-constructed memory network to obtain a UWB branch behavior score set, wherein the UWB branch behavior score set includes i UWB branch behavior scores; Obtain the visual branch weight coefficients and UWB branch weight coefficients. Calculate the final score for the behavior based on the visual branch weight coefficients, UWB branch weight coefficients, visual branch behavior score set, and UWB branch behavior score set. The calculation formula is as follows:

[0045] in, Indicates the index value. The line with index i is the final score. This represents the visual branch behavior score for index i. This indicates the score of the UWB branch with index value i. Indicates the visual branch weight coefficient. Indicates the UWB branch weight coefficient; The highest final score is obtained based on the final score of the aforementioned behavior; The highest final score is mapped using a preset mapping relationship and an identification and decision-making unit to obtain the identification result.

[0046] It should be understood that the method of inputting the visual spatiotemporal feature vector into a pre-constructed fully connected layer to obtain a visual branch behavior score set refers to inputting the visual spatiotemporal feature vector into one or more fully connected layers and calculating the visual branch behavior score set through linear transformation. The visual branch behavior score set refers to a set containing i scores, where i corresponds to the number of categories of predefined behavior categories (such as shooting, dribbling, breakthrough, etc.), and each score represents the confidence of the visual data for that behavior. The fully connected layer refers to a pre-trained neural network layer used for classification projection. The method of inputting the position feature vector into a pre-constructed memory network to obtain a UWB branch behavior score set refers to using a memory network (such as LSTM) to process the position feature vector, capture temporal dependencies, and output the score for each behavior category. The UWB branch behavior score set refers to a set containing i scores, where each score represents the confidence of the UWB position data for that behavior. Optionally, LSTM (Long Short-Term Memory) can be used as the memory network. The method for calculating the final behavior score based on the visual branch weight coefficient, UWB branch weight coefficient, visual branch behavior score set, and UWB branch behavior score set refers to using a weighted fusion formula to linearly combine the scores of the two branches to obtain the final behavior score. The final behavior score is a score that integrates visual and location information and is used to represent confidence, for behavior classification. The method for obtaining the highest final score refers to selecting the maximum value from the final behavior scores. The final behavior score represents the confidence of the most likely behavior. The method of mapping the final score using a preset mapping relationship and an identification and decision-making unit refers to converting the highest score into a specific behavior label (e.g., dribbling, three-point shot) according to the mapping relationship. The identification result refers to the final behavior classification, such as "shooting" or "passing". The identification and decision-making unit refers to the functional module used to identify behaviors and make classification decisions. The mapping relationship refers to the correspondence between the final score and the actual behavior category. For example, assuming the system presets 7 behavior categories, they are: {1: "Dribbling", 2: "Breakthrough", 3: "Passing", 4: "Three-point shot", 5: "Two-point shot", 6: "Rebound", 7: "Screen"}. Calculation shows that for the target ID: A, its final behavior score vector is S = [0.1, 0.3, 0.05, 0.85, 0.15, 0.02, 0.08], where the highest score S... max =0.85, its corresponding index i max = 4. The identification and decision-making unit queries according to the mapping relationship, maps index 4 to the behavior category "three-point shot", and outputs this result as the current behavior identification result for target A.

[0047] It should be explained that obtaining the visual branch weight coefficients and UWB branch weight coefficients includes: Obtain the current positioning target ID, and obtain the UWB positioning coordinate sequence based on the current positioning target ID, wherein the UWB positioning coordinate sequence includes multiple UWB positioning coordinates arranged in order of timestamp from first to last; The first positioning coordinate is obtained by arbitrarily extracting UWB positioning coordinates from the UWB positioning coordinate sequence; Based on the first positioning coordinate, the adjacent positioning coordinates are identified in the UWB positioning coordinates. The absolute difference between the timestamp corresponding to the first positioning coordinate and the timestamp corresponding to the adjacent positioning coordinate is calculated to obtain the time interval. The instantaneous velocity and instantaneous acceleration are calculated based on adjacent UWB positioning coordinates and time intervals in the UWB positioning coordinate sequence. Based on the UWB positioning coordinate sequence, obtain reference UWB positioning coordinates, calculate multiple Euclidean distances using the reference UWB positioning coordinates and multiple fixed key point coordinates, and take the minimum value among the multiple Euclidean distances as the minimum landmark distance; The visual branch weight coefficient and the UWB branch weight coefficient are calculated based on the instantaneous motion velocity, instantaneous acceleration, and minimum landmark distance. The calculation formula is as follows: in, , These represent the visual branch weight coefficient and the UWB branch weight coefficient, respectively. and These represent the initial weight coefficients, , , , and Indicates an adjustable parameter. For the Sigmoid function, Represents the hyperbolic tangent function. Represents an exponential function. Indicates instantaneous velocity. Indicates instantaneous acceleration. Indicates the minimum distance to a landmark.

[0048] Furthermore, the method for obtaining the UWB positioning coordinate sequence based on the current positioning target ID refers to filtering out the coordinates associated with the current positioning target ID from the UWB positioning data and sorting them in chronological order by timestamp. The current positioning target ID refers to the positioning target ID corresponding to the target currently undergoing behavior recognition, and the UWB positioning coordinate sequence refers to the temporal location trajectory of the target, used for dynamic weight calculation. The method for arbitrarily extracting UWB positioning coordinates from the UWB positioning coordinate sequence to obtain the first positioning coordinate refers to randomly selecting a coordinate or selecting it according to a specific strategy (such as latest) as the first positioning coordinate. This first positioning coordinate is a reference point used to calculate motion parameters. The method for identifying adjacent positioning coordinates in the UWB positioning coordinate sequence based on the first positioning coordinate refers to finding coordinates that are temporally adjacent in the sequence. These adjacent positioning coordinates are coordinates of consecutive timestamps. The method of calculating the absolute difference between the timestamp corresponding to the first positioning coordinate and the timestamps corresponding to adjacent positioning coordinates to obtain the time interval refers to calculating the timestamp difference. The time interval is the absolute value of the time difference between the timestamp corresponding to the first positioning coordinate and the timestamps corresponding to adjacent positioning coordinates, which is used for subsequent calculation of instantaneous velocity and instantaneous acceleration. The method of calculating instantaneous velocity and instantaneous acceleration based on adjacent UWB positioning coordinates and time intervals in the UWB positioning coordinate sequence refers to first calculating the displacement distance using adjacent UWB positioning coordinates in the UWB positioning coordinate sequence, and then using the velocity calculation formula and acceleration calculation formula to calculate the instantaneous velocity and instantaneous acceleration. The velocity calculation formula and acceleration calculation formula are public knowledge and will not be elaborated here. The instantaneous velocity and instantaneous acceleration refer to the quantitative indicators of the target's current motion state. The method of obtaining reference UWB positioning coordinates based on the UWB positioning coordinate sequence refers to selecting a representative coordinate (such as average or latest) in the sequence. The reference UWB positioning coordinate is an arbitrarily selected point in the UWB positioning coordinate sequence used for distance calculation. The method of calculating multiple Euclidean distances using reference UWB positioning coordinates and multiple fixed key point coordinates, and taking the minimum value among the multiple Euclidean distances as the minimum landmark distance, refers to calculating the Euclidean distance between the reference point and each fixed key point, and taking the minimum value. The minimum landmark distance refers to the distance from the target to the nearest environmental key point, which is used to evaluate the location reliability. The multiple fixed key point coordinates refer to preset static points, such as UWB base stations.The method for calculating the visual branch weight coefficient and UWB branch weight coefficient based on the instantaneous motion speed, instantaneous acceleration, and minimum landmark distance refers to dynamically adjusting the weights by substituting them into the formula. When the motion is intense, the visual branch weight affected by the motion is reduced (due to blurring), while the UWB branch weight is increased. When approaching key points (such as the basket or three-point line), the decision weight with high UWB position information is adjusted to enhance the recognition ability of behaviors that rely on precise position determination, such as shooting and driving. The visual branch weight coefficient and UWB branch weight coefficient are adaptive parameters used to balance the contributions of the two branches. The initial weight coefficient is a preset initial weight with a default value of 1. The adjustable parameters are fixed parameters calculated in advance through grid search, Bayesian optimization algorithms, etc., for example. This refers to the speed threshold, a critical value used to determine whether the speed has entered a "high-speed state". This coefficient represents the magnitude of the influence of instantaneous motion speed and instantaneous acceleration on the visual weight branch coefficient. The smaller the value, the more sensitive the visual weight branch coefficient is to instantaneous motion speed and instantaneous acceleration. This is a speed compensation coefficient, used when the instantaneous speed exceeds a threshold. Then, additional compensation is applied to the visual weight branch coefficients. This represents the location sensitivity coefficient, used to control the degree of influence of minimum landmark distance on the UWB branch weight coefficient. This represents the location normalization coefficient, used to scale and normalize the minimum landmark distance to fit the input range of the tanh function. The sigmoid function, hyperbolic tangent function, and exponential function are all well-known and will not be elaborated upon here.

[0049] To address the problems described in the background art, this invention confirms the receipt of behavior recognition instructions and, based on these instructions, confirms the behavior recognition environment. The behavior recognition environment includes a behavior recognition system and multiple initial positioning targets. The behavior recognition system includes a data acquisition unit, a multimodal fusion unit, a feature extraction unit, and a recognition and decision-making unit. The data acquisition unit includes a video data acquisition unit and a UWB positioning data acquisition unit. Therefore, this invention considers the real-time requirements of complex multi-target scenarios during behavior recognition. Thus, by confirming the behavior recognition environment, it ensures modular system integration, providing a reliable foundation for subsequent multimodal data processing. This improves the overall adaptability and robustness of behavior recognition. Based on the data acquisition unit, data is acquired from the multiple initial positioning targets to obtain the original video stream and UWB positioning data. It is evident that this invention employs a dual-source data acquisition mechanism, combining video and UWB positioning to capture dynamic information, avoiding the limitations of a single modality, and thus laying a comprehensive data foundation for fusion analysis. Based on the multimodal fusion unit, the original video stream and UWB positioning data are time-synchronized and aligned to obtain synchronized video stream and synchronized UWB positioning data. It is evident that this embodiment of the invention introduces a time synchronization step to eliminate data timing deviations, thereby improving the synergy and accuracy of multimodal data. The invention obtains image positioning coordinate sets and UWB positioning coordinate sets from synchronized video streams and synchronized UWB positioning data. Based on these coordinate sets, Euclidean distance is calculated, and identity matching is performed to obtain multiple positioning target IDs. This invention achieves multi-target identity association through coordinate matching, avoiding identity confusion and thus improving tracking accuracy in multi-target scenarios. For each of the multiple positioning target IDs, the following operations are performed: positional feature extraction is performed on the original video stream based on the feature extraction unit and the positioning target IDs to obtain a visual spatiotemporal feature vector; and the UWB positioning data is then analyzed based on the feature extraction unit and the positioning target IDs. The data undergoes feature extraction to obtain a location feature vector. This invention performs bimodal feature extraction for each ID, quantifying visual and location dynamics to ensure the richness and specificity of the feature representation. Based on the visual spatiotemporal feature vector and the location feature vector, a final behavior score is calculated. Based on the final behavior score and the recognition and decision-making unit, behavior classification is performed to obtain the recognition result. Based on the recognition result, behavior recognition for multiple initial positioning targets is achieved. This invention integrates feature calculation, score setting, and classification to form a closed-loop decision-making mechanism. Furthermore, multimodal fusion enhances classification reliability. Therefore, this invention can improve the accuracy, efficiency, and multi-target applicability of behavior recognition.

[0050] like Figure 2 The diagram shown is a functional block diagram of a behavior recognition system based on UWB positioning provided in an embodiment of the present invention.

[0051] The behavior recognition system 100 based on UWB positioning described in this invention can be installed in an electronic device. Depending on the functions implemented, the behavior recognition system 100 based on UWB positioning may include an environment verification module 101, a data acquisition module 102, a feature extraction module 103, and a recognition decision module 104. The module described in this invention can also be called a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and are stored in the memory of the electronic device.

[0052] The environment confirmation module 101 is used to confirm the received behavior recognition instruction and confirm the behavior recognition environment based on the behavior recognition instruction. The behavior recognition environment includes a behavior recognition system and multiple initial positioning targets. The behavior recognition system includes a data acquisition unit, a multimodal fusion unit, a feature extraction unit, and a recognition and decision-making unit. The data acquisition unit includes a video data acquisition unit and a UWB positioning data acquisition unit. The data acquisition module 102 is used to acquire data from the plurality of initial positioning targets based on the data acquisition unit to obtain the original video stream and UWB positioning data. Based on the multimodal fusion unit, the original video stream and UWB positioning data are time-synchronized and aligned to obtain synchronized video stream and synchronized UWB positioning data; The feature extraction module 103 is used to obtain an image positioning coordinate set and a UWB positioning coordinate set based on the synchronous video stream and synchronous UWB positioning data, calculate the Euclidean distance based on the image positioning coordinate set and the UWB positioning coordinate set, and perform identity matching based on the Euclidean distance to obtain multiple positioning target IDs. Perform the following operations on all of the aforementioned location target IDs: Based on the feature extraction unit and the positioning target ID, the original video stream is subjected to position feature extraction to obtain a visual spatiotemporal feature vector. Based on the feature extraction unit and the positioning target ID, the UWB positioning data is subjected to feature extraction to obtain a position feature vector. The recognition decision module 104 is used to calculate the final behavior score based on the visual spatiotemporal feature vector and the position feature vector, perform behavior classification based on the final behavior score and the recognition and decision unit, obtain the recognition result, and realize the behavior recognition of multiple initial positioning targets based on the recognition result.

[0053] In detail, the modules in the UWB-based behavior recognition system 100 described in this embodiment of the invention employ the same methods as described above. Figure 1 The method uses the same technical means as the behavior recognition method based on UWB positioning described above and can produce the same technical effect, so it will not be repeated here.

[0054] like Figure 3 The diagram shown is a structural schematic of an electronic device that implements a behavior recognition method based on UWB positioning according to an embodiment of the present invention.

[0055] The electronic device 1 may include a processor 10, a memory 11 and a bus 12, and may also include a computer program stored in the memory 11 and capable of running on the processor 10, such as a behavior recognition method program based on UWB positioning.

[0056] The memory 11 includes at least one type of readable storage medium, such as flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as a portable hard drive. In other embodiments, the memory 11 can be an external storage device of the electronic device 1, such as a plug-in portable hard drive, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device 1. Furthermore, the memory 11 includes both internal storage units and external storage devices of the electronic device 1. The memory 11 can be used not only to store application software and various types of data installed on the electronic device 1, such as the code of a behavior recognition method program based on UWB positioning, but also to temporarily store data that has been output or will be output.

[0057] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the memory 11 (e.g., behavior recognition methods based on UWB positioning) and calls data stored in the memory 11 to perform various functions of the electronic device 1 and process data.

[0058] The bus 12 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus 12 can be divided into an address bus, a data bus, a control bus, etc. The bus 12 is configured to realize the connection and communication between the memory 11 and at least one processor 10, etc.

[0059] Figure 3 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 3 The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0060] For example, although not shown, the electronic device 1 may also include a power supply (such as a battery) to power the various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device 1 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0061] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, a Bluetooth interface, etc.), which is typically used to establish communication connections between the electronic device 1 and other electronic devices.

[0062] Optionally, the electronic device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), and optionally, a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device 1 and to display a visual user interface.

[0063] The behavior recognition method program based on UWB positioning stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When run in the processor 10, it can achieve the following: The system confirms receipt of a behavior recognition instruction and confirms the behavior recognition environment based on the instruction. The behavior recognition environment includes a behavior recognition system and multiple initial positioning targets. The behavior recognition system includes a data acquisition unit, a multimodal fusion unit, a feature extraction unit, and a recognition and decision-making unit. The data acquisition unit includes a video data acquisition unit and a UWB positioning data acquisition unit. Based on the data acquisition unit, data is acquired from the multiple initial positioning targets to obtain the original video stream and UWB positioning data; Based on the multimodal fusion unit, the original video stream and UWB positioning data are time-synchronized and aligned to obtain synchronized video stream and synchronized UWB positioning data; Based on the synchronous video stream and synchronous UWB positioning data, obtain image positioning coordinate set and UWB positioning coordinate set, calculate Euclidean distance based on the image positioning coordinate set and UWB positioning coordinate set, and perform identity matching based on the Euclidean distance to obtain multiple positioning target IDs; Perform the following operations on all of the aforementioned location target IDs: Based on the feature extraction unit and the positioning target ID, the original video stream is subjected to position feature extraction to obtain a visual spatiotemporal feature vector. Based on the feature extraction unit and the positioning target ID, the UWB positioning data is subjected to feature extraction to obtain a position feature vector. The final behavior score is calculated based on the visual spatiotemporal feature vector and the location feature vector. The behavior is classified based on the final behavior score and the recognition and decision unit to obtain the recognition result. Based on the recognition result, the behavior recognition of multiple initial positioning targets is realized.

[0064] Specifically, the processor 10's implementation method for the above instructions can be found in [reference needed]. Figures 1 to 3 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0065] Furthermore, if the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0066] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor of an electronic device, can perform the following: The system confirms receipt of a behavior recognition instruction and confirms the behavior recognition environment based on the instruction. The behavior recognition environment includes a behavior recognition system and multiple initial positioning targets. The behavior recognition system includes a data acquisition unit, a multimodal fusion unit, a feature extraction unit, and a recognition and decision-making unit. The data acquisition unit includes a video data acquisition unit and a UWB positioning data acquisition unit. Based on the data acquisition unit, data is acquired from the multiple initial positioning targets to obtain the original video stream and UWB positioning data; Based on the multimodal fusion unit, the original video stream and UWB positioning data are time-synchronized and aligned to obtain synchronized video stream and synchronized UWB positioning data; Based on the synchronous video stream and synchronous UWB positioning data, obtain image positioning coordinate set and UWB positioning coordinate set, calculate Euclidean distance based on the image positioning coordinate set and UWB positioning coordinate set, and perform identity matching based on the Euclidean distance to obtain multiple positioning target IDs; Perform the following operations on all of the aforementioned location target IDs: Based on the feature extraction unit and the positioning target ID, the original video stream is subjected to position feature extraction to obtain a visual spatiotemporal feature vector. Based on the feature extraction unit and the positioning target ID, the UWB positioning data is subjected to feature extraction to obtain a position feature vector. The final behavior score is calculated based on the visual spatiotemporal feature vector and the location feature vector. The behavior is classified based on the final behavior score and the recognition and decision unit to obtain the recognition result. Based on the recognition result, the behavior recognition of multiple initial positioning targets is realized.

[0067] In the embodiments provided by this invention, it should be understood that the disclosed devices, systems, and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative, and actual implementations may have other classification methods.

[0068] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0069] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0070] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0071] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A behavior recognition method based on UWB positioning, characterized in that, The method includes: The system confirms receipt of a behavior recognition instruction and confirms the behavior recognition environment based on the instruction. The behavior recognition environment includes a behavior recognition system and multiple initial positioning targets. The behavior recognition system includes a data acquisition unit, a multimodal fusion unit, a feature extraction unit, and a recognition and decision-making unit. The data acquisition unit includes a video data acquisition unit and a UWB positioning data acquisition unit. Based on the data acquisition unit, data is acquired from the multiple initial positioning targets to obtain the original video stream and UWB positioning data; Based on the multimodal fusion unit, the original video stream and UWB positioning data are time-synchronized and aligned to obtain synchronized video stream and synchronized UWB positioning data; Based on the synchronous video stream and synchronous UWB positioning data, obtain image positioning coordinate set and UWB positioning coordinate set, calculate Euclidean distance based on the image positioning coordinate set and UWB positioning coordinate set, and perform identity matching based on the Euclidean distance to obtain multiple positioning target IDs; Perform the following operations on all of the aforementioned location target IDs: Based on the feature extraction unit and the positioning target ID, the original video stream is subjected to position feature extraction to obtain a visual spatiotemporal feature vector. Based on the feature extraction unit and the positioning target ID, the UWB positioning data is subjected to feature extraction to obtain a position feature vector. The final behavior score is calculated based on the visual spatiotemporal feature vector and the location feature vector. The behavior is classified based on the final behavior score and the recognition and decision unit to obtain the recognition result. Based on the recognition result, the behavior recognition of multiple initial positioning targets is realized.

2. The behavior recognition method based on UWB positioning as described in claim 1, characterized in that, The step of acquiring data from the multiple initial positioning targets based on the data acquisition unit to obtain the original video stream and UWB positioning data includes: A detection area for data acquisition is obtained, and based on a preset frame rate, the video data acquisition unit acquires video images of multiple initial positioning targets in the detection area to obtain multiple video image frames. Timestamps are appended to the multiple video image frames to obtain multiple timestamped image frames, and the original video stream is obtained based on the multiple timestamped image frames. Based on multiple pre-built UWB positioning tags and pre-built UWB base stations, wireless communication connections are established with the multiple initial positioning targets, and the distance between the UWB base station and the multiple initial positioning targets is measured based on the UWB positioning data acquisition unit to obtain multiple measured distances; Add timestamps to the multiple measured distances to obtain multiple timestamp measured distances; By integrating the distance measurements from the multiple timestamps, UWB positioning data is obtained.

3. The behavior recognition method based on UWB positioning as described in claim 2, characterized in that, The step of performing time synchronization alignment on the original video stream and UWB positioning data based on the multimodal fusion unit to obtain synchronized video stream and synchronized UWB positioning data includes: The original video stream is timestamped based on the multimodal fusion unit to obtain a video stream timestamp sequence, wherein the video stream timestamp sequence includes multiple video stream timestamps arranged in chronological order. A reference timeline is established using the timestamp sequence of the video stream; Multiple UWB timestamps are obtained using the multimodal fusion unit and UWB positioning data, and the multiple UWB timestamps are mapped to the reference time axis to obtain a time sequence correspondence. Based on the aforementioned time-series correspondence, the original video stream and UWB positioning data are synchronized and aligned to obtain synchronized video stream and synchronized UWB positioning data.

4. The behavior recognition method based on UWB positioning as described in claim 3, characterized in that, The process of obtaining the image positioning coordinate set and the UWB positioning coordinate set based on the synchronized video stream and synchronized UWB positioning data includes: For each of the multiple initial positioning targets, perform the following operation: The position of the initially located target in the image is detected by using a pre-trained target detection model and multiple timestamped image frames in a synchronous video stream, resulting in multiple target detection bounding boxes. An image pixel coordinate system is established based on the synchronous video stream. The pixel coordinates of the initial positioning target in multiple timestamp image frames are determined based on the multiple target detection bounding boxes and the image pixel coordinate system. Multiple positioning target pixel coordinates are obtained. The multiple positioning target pixel coordinates are sorted in order of timetamp from first to last to obtain a pixel position sequence. The initial position movement curve is obtained based on the pixel position sequence and a preset curve fitting algorithm; The overlapping region is detected frame by frame based on multiple target detection bounding boxes and multiple timestamp image frames to obtain multiple bounding box overlap degrees. The multiple bounding box overlap degrees are compared with a preset overlap threshold. If the bounding box overlap degree is greater than the overlap threshold, the pixel coordinates of the corresponding target detection bounding box are determined based on the initial position movement curve to obtain one or more determined pixel coordinates. The position movement curve is obtained based on the one or more determined pixel coordinates and the initial position movement curve. A set of reference point coordinates is obtained based on the position movement curve; a Cartesian coordinate system is established based on the detection area and the UWB base station; the set of reference point coordinates is mapped from the image pixel coordinate system to the Cartesian coordinate system based on the pre-constructed homography matrix to obtain the image positioning coordinate set; Based on the UWB positioning data and the Cartesian coordinate system, the coordinates of the multiple initial positioning targets are determined to obtain a UWB positioning coordinate set.

5. The behavior recognition method based on UWB positioning as described in claim 4, characterized in that, The process involves calculating Euclidean distance based on the image positioning coordinate set and the UWB positioning coordinate set, and then performing identity matching based on the Euclidean distance to obtain multiple positioning target IDs, including: The image positioning coordinate set and the UWB positioning coordinate set are purified to obtain an effective image positioning coordinate set and an effective UWB positioning coordinate set, wherein the effective image positioning coordinate set and the effective UWB positioning coordinate set each contain multiple effective image positioning coordinates and effective UWB positioning coordinates. Calculate multiple Euclidean distances based on each valid image positioning coordinate in the set of valid image positioning coordinates and each valid UWB positioning coordinate in the set of valid UWB positioning coordinates, and obtain a two-dimensional distance matrix based on the multiple Euclidean distances. Based on the two-dimensional distance matrix and the pre-constructed matching algorithm, the matching relationship is solved, and multiple successful matching pairs are obtained; Each of the multiple successful matching pairs is assigned a unique identifier to obtain multiple location target IDs.

6. The behavior recognition method based on UWB positioning as described in claim 5, characterized in that, The step of extracting location features from the original video stream based on the feature extraction unit and the positioning target ID to obtain a visual spatiotemporal feature vector includes: Based on the location target ID, 64 video frames are extracted from the original video stream. The feature extraction unit then performs the following operations on these 64 video frames: The 64 frames of video images are sampled based on a preset first sampling interval to obtain a first sampling sequence; The 64 frames of video images are sampled based on a preset second sampling interval to obtain a second sampling sequence; The first sampling sequence is input into a pre-constructed first three-dimensional convolutional neural network branch for spatiotemporal feature extraction to obtain the first feature map; The second sampling sequence is input into a pre-constructed second three-dimensional convolutional neural network branch for spatiotemporal feature extraction to obtain the second feature map; The second feature map is transformed in the time dimension, and the transformed second feature map is concatenated with the first feature map to obtain a fused visual feature map; A local feature map is obtained by extracting the feature map corresponding to the localized target ID from the fused visual feature map using a pre-constructed alignment algorithm. The local feature map is subjected to global average pooling to obtain the visual spatiotemporal feature vector.

7. The behavior recognition method based on UWB positioning as described in claim 6, characterized in that, The step of extracting features from the UWB positioning data based on the feature extraction unit and the positioning target ID to obtain a location feature vector includes: Based on the UWB positioning data, the UWB coordinates associated with the positioning target ID are extracted sequentially according to the timestamp from first to last to obtain a UWB coordinate sequence, wherein the UWB coordinate sequence contains multiple UWB coordinates corresponding to different timestamps; Based on the feature extraction unit, the following operations are performed on each UWB coordinate point in the UWB coordinate sequence: Obtain multiple reference positioning target IDs, obtain multiple reference coordinate points based on the multiple reference positioning target IDs and the timestamps corresponding to the UWB coordinate points, and calculate the first relative distance vector based on the UWB coordinate points and the multiple reference points; The second relative distance vector is calculated based on UWB coordinate points and the coordinates of multiple preset fixed key points; The first relative distance vector and the second relative distance vector are merged and normalized to obtain the merged relative position vector. Arrange the merged relative position vectors corresponding to all timestamps in the UWB coordinate sequence in chronological order to obtain a relative position vector sequence. The relative position vector sequence is subjected to dimensionality reduction processing to obtain position feature vectors.

8. The behavior recognition method based on UWB positioning as described in claim 7, characterized in that, The process involves calculating a final behavior score based on the visual spatiotemporal feature vector and the location feature vector, and then classifying the behavior based on the final behavior score and the recognition and decision-making unit to obtain a recognition result, including: The visual spatiotemporal feature vector is input into a pre-constructed fully connected layer to obtain a visual branch behavior score set, wherein the visual branch behavior score set includes i visual branch behavior scores. The location feature vector is input into a pre-constructed memory network to obtain a UWB branch behavior score set, wherein the UWB branch behavior score set includes i UWB branch behavior scores; Obtain the visual branch weight coefficient and the UWB branch weight coefficient. Calculate the final score for the behavior based on the visual branch weight coefficient, the UWB branch weight coefficient, the visual branch behavior score set, and the UWB branch behavior score set. The calculation formula is as follows: Among them, Indicates the index value. The line with index i is the final score. This represents the visual branch behavior score for index i. This indicates the score of the UWB branch with index value i. Indicates the visual branch weight coefficient. Indicates the UWB branch weight coefficient; The highest final score is obtained based on the final score of the aforementioned behavior; The highest final score is mapped using a preset mapping relationship and an identification and decision-making unit to obtain the identification result.

9. The behavior recognition method based on UWB positioning as described in claim 8, characterized in that, The acquisition of the visual branch weight coefficients and UWB branch weight coefficients includes: Obtain the current positioning target ID, and obtain the UWB positioning coordinate sequence based on the current positioning target ID, wherein the UWB positioning coordinate sequence includes multiple UWB positioning coordinates arranged in order of timestamp from first to last; The first positioning coordinate is obtained by arbitrarily extracting UWB positioning coordinates from the UWB positioning coordinate sequence; Based on the first positioning coordinate, the adjacent positioning coordinates are identified in the UWB positioning coordinates. The absolute difference between the timestamp corresponding to the first positioning coordinate and the timestamp corresponding to the adjacent positioning coordinate is calculated to obtain the time interval. The instantaneous velocity and instantaneous acceleration are calculated based on adjacent UWB positioning coordinates and time intervals in the UWB positioning coordinate sequence. Based on the UWB positioning coordinate sequence, obtain reference UWB positioning coordinates, calculate multiple Euclidean distances using the reference UWB positioning coordinates and multiple fixed key point coordinates, and take the minimum value among the multiple Euclidean distances as the minimum landmark distance; The visual branch weight coefficient and the UWB branch weight coefficient are calculated based on the instantaneous motion velocity, instantaneous acceleration, and minimum landmark distance. The calculation formula is as follows: in, , These represent the visual branch weight coefficient and the UWB branch weight coefficient, respectively. and These represent the initial weight coefficients, , , , and Indicates an adjustable parameter. For the Sigmoid function, Represents the hyperbolic tangent function. Represents an exponential function. Indicates instantaneous velocity. Indicates instantaneous acceleration. Indicates the minimum distance to a landmark.

10. A behavior recognition system based on UWB positioning, characterized in that, The device includes: An environment confirmation module is used to confirm the received behavior recognition command and confirm the behavior recognition environment based on the behavior recognition command. The behavior recognition environment includes a behavior recognition system and multiple initial positioning targets. The behavior recognition system includes a data acquisition unit, a multimodal fusion unit, a feature extraction unit, and a recognition and decision-making unit. The data acquisition unit includes a video data acquisition unit and a UWB positioning data acquisition unit. The data acquisition module is used to acquire data from the multiple initial positioning targets based on the data acquisition unit to obtain the original video stream and UWB positioning data; Based on the multimodal fusion unit, the original video stream and UWB positioning data are time-synchronized and aligned to obtain synchronized video stream and synchronized UWB positioning data; The feature extraction module is used to obtain image positioning coordinate sets and UWB positioning coordinate sets based on the synchronous video stream and synchronous UWB positioning data, calculate Euclidean distance based on the image positioning coordinate sets and UWB positioning coordinate sets, and perform identity matching based on the Euclidean distance to obtain multiple positioning target IDs. Perform the following operations on all of the aforementioned location target IDs: Based on the feature extraction unit and the positioning target ID, the original video stream is subjected to position feature extraction to obtain a visual spatiotemporal feature vector. Based on the feature extraction unit and the positioning target ID, the UWB positioning data is subjected to feature extraction to obtain a position feature vector. The identification and decision module is used to calculate the final behavior score based on the visual spatiotemporal feature vector and the location feature vector, perform behavior classification based on the final behavior score and the identification and decision unit, obtain the identification result, and realize the behavior identification of multiple initial positioning targets based on the identification result.

Images