Indoor target positioning method and system based on machine vision and deep learning

Abstract

The invention relates to an indoor target positioning method and system based on machine vision and deep learning, and belongs to the technical field of target positioning. The method comprises the following steps: acquiring positioning data transmitted by a plurality of long-distance radio beacons deployed indoors in real time, and anchoring the positioning data to an indoor three-dimensional coordinate grid model node to obtain a global space reference framework; a real-time video image is obtained and preprocessed, and an LW-DETR algorithm is adopted to detect initial position information and size information of a target on an image plane; calculating a distance and an azimuth angle between a target and each beacon by combining beacon signal characteristics to obtain spatial position associated data; performing spatial registration and fusion on the multi-source data in combination with a global spatial reference frame to obtain target position information; and tracking and predicting the motion trail and speed information of the target, and integrating to obtain the final predicted position of the target. According to the method, the hardware collaborative dependence can be reduced, the anti-interference capability in an extreme environment is enhanced, and the indoor target positioning stability and precision are remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of target localization technology, and in particular relates to an indoor target localization method and system based on machine vision and deep learning. Background Technology

[0002] With the development of indoor positioning technology, indoor positioning technologies that fuse multi-source data have emerged. These technologies use machine vision to obtain preliminary location information, combine it with wireless sensor networks or Bluetooth beacons to supplement positioning data, and use complex algorithms to fuse multi-source data to improve accuracy, thus forming the current indoor positioning methods based on machine vision and wireless sensor networks, or machine vision and deep learning.

[0003] Traditional technologies rely on machine vision to identify target locations through image preprocessing and feature extraction, or on wireless sensor networks and Bluetooth beacons to calculate distances based on signal strength for positioning. In some scenarios, only two data results are simply combined, lacking a systematic hardware and software co-design and precise fusion mechanism.

[0004] However, these positioning methods currently have many shortcomings. First, they are highly dependent on hardware, requiring the collaboration of multiple devices such as cameras, sensor nodes, and Bluetooth beacons. Failure of any device, such as a power outage of a sensor or a damaged camera, will lead to missing or inaccurate positioning data, affecting the normal operation of the system. Second, they have poor adaptability to extreme environments. Strong electromagnetic interference can cause the wireless signal model to become inaccurate, and sudden changes in lighting or high-density occlusion can cause machine vision recognition to fail, making it difficult to guarantee positioning accuracy. Summary of the Invention

[0005] Therefore, it is necessary to provide an indoor target localization method and system based on machine vision and deep learning that can reduce hardware collaboration dependence, enhance anti-interference capability in extreme environments, and improve positioning accuracy in response to the above-mentioned technical problems.

[0006] Firstly, this application provides an indoor target localization method based on machine vision and deep learning, including:

[0007] The positioning data transmitted in real time from multiple long-range radio beacons deployed indoors is acquired, and the positioning data is anchored to the nodes of the indoor three-dimensional coordinate grid model to obtain a global spatial reference frame.

[0008] Acquire real-time video images and preprocess them to obtain preprocessed video images;

[0009] The LW-DETR algorithm is used to detect targets in the preprocessed video image to obtain the initial position and size information of the targets in the image plane;

[0010] Based on the initial position and size information, the distance and azimuth between the target and each long-range radio beacon are calculated using the signal characteristics of the long-range radio beacon, thus obtaining the spatial position association data of the long-range radio beacon;

[0011] By combining initial location information, size information, and spatial location association data of long-range radio beacons with a global spatial reference frame for spatial registration and multi-source data fusion, the location information of the target is obtained.

[0012] Based on location coordinates and long-range radio positioning data, the motion state of the target is tracked and predicted to obtain the target's trajectory and speed information within a preset time period;

[0013] By integrating position coordinates, motion trajectory, and velocity information within a global spatial reference frame, the final predicted position of the target is obtained.

[0014] In one embodiment, positioning data transmitted in real time from multiple long-range radio beacons deployed indoors is acquired, and the positioning data is anchored to nodes of an indoor three-dimensional coordinate grid model to obtain a global spatial reference frame, including:

[0015] The positioning data transmitted in real time by multiple long-range radio beacons deployed indoors is acquired, and the positioning data is denoised using a filtering algorithm to obtain the denoised positioning data.

[0016] Abnormal data in the noise-reduced positioning data is removed by using data validity verification rules to obtain preprocessed long-range radio beacon positioning data;

[0017] Based on the preprocessed long-range radio beacon positioning data, image acquisition and feature extraction instructions are generated using SLAM automatic calibration technology;

[0018] The image acquisition and feature extraction command is sent to the camera. The image acquisition and feature extraction command is used to instruct the camera to capture indoor environment images at a preset frequency and extract environmental feature points to obtain an environmental feature point set.

[0019] The spatial relationship between the set of environmental feature points and each beacon is established by using a feature point matching algorithm, thus obtaining spatial correlation data between feature points and beacons.

[0020] Based on the spatial correlation data of feature points and beacons, a coordinate transformation algorithm is used to calculate the mapping relationship between the camera and the beacon coordinate system, and the mapping parameters are obtained.

[0021] Based on the mapping parameters, the local coordinate systems corresponding to multiple beacons are fused and uniformly calibrated to obtain the unified spatial coordinate system parameters.

[0022] Obtain the actual physical dimensions of the indoor environment, and construct a uniform three-dimensional coordinate grid of the indoor space using a three-dimensional mesh modeling method based on the unified spatial coordinate system parameters to obtain the three-dimensional coordinate grid model of the indoor space.

[0023] The unified spatial coordinate system parameters are integrated into the three-dimensional coordinate grid model, and the beacon position is bound to the grid node through the coordinate anchoring algorithm to establish a global spatial reference frame.

[0024] In one embodiment, acquiring a real-time video image and preprocessing the real-time video image to obtain a preprocessed video image includes:

[0025] Acquire real-time video stream, extract continuous video frames at a preset frame rate, and timestamp each video frame to obtain a timestamped video frame.

[0026] Median filtering is applied to the timestamped video frames to denoise them, resulting in denoised video frames.

[0027] The denoised video frames are subjected to global histogram equalization to obtain the denoised and enhanced video frames.

[0028] Correcting radial distortion in the denoised and enhanced video frame yields a distortion-corrected image.

[0029] The original geometric shape of the image after distortion correction is restored by bilinear interpolation algorithm to obtain the preprocessed video image.

[0030] In one embodiment, the LW-DETR algorithm is used to detect targets in the preprocessed video image to obtain the initial position and size information of the targets in the image plane, including:

[0031] The LW-DETR algorithm is used to detect targets in the preprocessed video image to obtain target candidate region images;

[0032] A lightweight convolutional backbone network is used to extract hierarchical features from the target candidate region image to obtain multi-scale feature maps.

[0033] Multi-scale feature maps are input into the Transformer encoder for context modeling, and long-distance dependencies are captured through a multi-head self-attention mechanism to obtain context-enhanced feature representations.

[0034] The context-enhanced feature representation is input into the Transformer decoder for target query interaction, and a dynamic anchor point allocation strategy is used to determine the target candidate box.

[0035] Multi-scale prediction processing is performed on the target candidate boxes to obtain the preliminary localization of the target candidate boxes;

[0036] The initial positioning is processed by bounding box regression to obtain initial position and size information.

[0037] In one embodiment, based on initial position and size information, the distance and azimuth between the target and each long-range radio beacon are calculated using the signal characteristics of the long-range radio beacon, resulting in spatial position association data of the long-range radio beacons, including:

[0038] Based on the initial position and size information, the image coordinates of the target center point and the size parameters of the target's bounding rectangle are extracted to obtain the target's basic feature data;

[0039] Based on the center point image coordinates of the target's basic feature data and the camera's intrinsic parameter matrix, the initial estimate of the target's three-dimensional position in the camera coordinate system is calculated through inverse perspective projection transformation, thus obtaining the target's position in the camera coordinate system.

[0040] Obtain the three-dimensional position coordinate data of each long-range radio beacon deployed indoors within a global spatial reference frame to obtain the beacon spatial coordinate set;

[0041] Based on the beacon spatial coordinate set and the target camera coordinate system position, the distance data from the target to each beacon is calculated using the long-distance radio signal time difference model, thus obtaining the target-to-beacon distance set.

[0042] By using a spatial geometric orientation calculation algorithm, and combining the position coordinates of each beacon with the distance data from the target to the beacon, the azimuth angle data of the target relative to each beacon is calculated, and the set of target relative azimuth angles is obtained.

[0043] The set of distances from the target to the beacon and the corresponding set of azimuth angles are spatiotemporally aligned and integrated into spatial location association data of long-range radio beacons, thus obtaining spatial location association data of long-range radio beacons.

[0044] In one embodiment, the initial location information, size information, and spatial location association data of long-range radio beacons are combined with a global spatial reference frame for spatial registration and multi-source data fusion to obtain the target's location information, including:

[0045] Based on the spatial coordinate mapping rules of the global spatial reference frame, the initial position information, size information and spatial position association data of long-distance radio beacons are unified in dimension through coordinate transformation algorithm to obtain multi-source data registration results;

[0046] For the registration results of multi-source data, a preset multi-source information fusion algorithm is used to perform data association and weight allocation, and the fusion accuracy is optimized by combining a spatiotemporal consistency constraint mechanism to obtain the fused target spatial coordinates;

[0047] The confidence level of the fused target spatial coordinates is evaluated and error compensation is performed to obtain the corrected spatial coordinates.

[0048] The corrected spatial coordinates are then used to calibrate the position deviation using a coordinate correction algorithm to obtain the target's position information.

[0049] In one embodiment, the motion state of the target is tracked and predicted based on location coordinates and long-range radio positioning data to obtain the target's trajectory and speed information, including:

[0050] The Kalman filter algorithm is used to fuse position coordinates and long-range radio positioning data to obtain fused position data. The state vector of the Kalman filter algorithm contains three-dimensional position and velocity, and the observation vector contains position coordinates and long-range radio positioning results.

[0051] Based on the fused position data, the motion trajectory is fitted using the least squares method to obtain the fitted trajectory curve;

[0052] A linear regression model is constructed by fitting the trajectory curve;

[0053] The future trajectory of the target is predicted by a linear regression model and a long short-term memory network model. The future trajectory is then optimized using a particle filter algorithm to obtain the target's motion trajectory and velocity information.

[0054] Secondly, this application also provides an indoor target localization system based on machine vision and deep learning, comprising:

[0055] A framework module is constructed to acquire real-time positioning data transmitted by multiple long-range radio beacons deployed indoors, and to anchor the positioning data to nodes of the indoor three-dimensional coordinate grid model to obtain a global spatial reference frame.

[0056] The preprocessing module is used to acquire real-time video images and preprocess them to obtain preprocessed video images.

[0057] The target detection module is used to detect targets in the preprocessed video image using the LW-DETR algorithm to obtain the initial position and size information of the target in the image plane.

[0058] The spatial correlation calculation module is used to calculate the distance and azimuth between the target and each long-range radio beacon based on the initial position and size information and the signal characteristics of the long-range radio beacon, so as to obtain the spatial position correlation data of the long-range radio beacon;

[0059] The location calculation module is used to combine the initial location information, size information and spatial location association data of long-distance radio beacons with the global spatial reference frame to perform spatial registration and multi-source data fusion to obtain the target's location information;

[0060] The motion state detection module is used to track and predict the motion state of the target based on the location coordinates and long-range radio positioning data, and obtain the target's motion trajectory and speed information within a preset time period;

[0061] The location integration module is used to track and predict the motion state of the target based on the location coordinates and long-range radio positioning data, and obtain the target's motion trajectory and speed information within a preset time period.

[0062] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the methods in the first aspect of this application.

[0063] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any of the methods in the first aspect of this application.

[0064] This application provides an indoor target localization method based on machine vision and deep learning. The method first acquires real-time positioning data from multiple long-range radio beacons deployed indoors, anchoring this data to nodes of an indoor 3D coordinate grid model to obtain a global spatial reference frame. Multi-beacon collaboration reduces dependence on single hardware components and avoids positioning interruptions due to single-device failures. Next, real-time video images are acquired and pre-processed to optimize quality and resist the effects of sudden changes in illumination. The LW-DETR algorithm is used to detect the target and obtain initial position and size information, reducing errors in the face of high-density occlusion. The distance and azimuth angle between the target and each beacon are calculated by combining beacon signal characteristics. Multi-source data is then registered and fused with the global frame to counteract electromagnetic interference and improve accuracy. Finally, the target's motion state is tracked and predicted, and the information is integrated to obtain the final predicted position, ensuring continuous and accurate positioning. This significantly reduces hardware collaboration dependence, enhances anti-interference capabilities in extreme environments, and significantly improves the stability and accuracy of indoor target localization. Attached Figure Description

[0065] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0066] Figure 1 A schematic diagram of an implementation environment provided for one embodiment of the present invention;

[0067] Figure 2 This is a flowchart of an indoor target localization method based on machine vision and deep learning in one embodiment of the present invention;

[0068] Figure 3 This is a schematic diagram of an indoor target localization system based on machine vision and deep learning, according to one embodiment of the present invention. Detailed Implementation

[0069] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0070] First, a brief introduction to the terms used in the embodiments of this application will be given.

[0071] Long-range radio beacon: A radio transmitter deployed in indoor spaces to transmit positioning signals. It continuously transmits positioning data containing its own identification and location information to the surrounding area via radio waves, providing a spatial reference for indoor positioning systems. It features long-distance transmission, low power consumption, and resistance to electromagnetic interference, and supports the fusion of multi-source data such as machine vision to achieve high-precision target positioning.

[0072] LW-DETR algorithm (Lightweight Deformable Encoder-decoder Transformer): A lightweight end-to-end object detection algorithm based on an improved Transformer architecture. It replaces part of the traditional Transformer structure with a lightweight convolutional backbone network to reduce computational complexity, while retaining the multi-head self-attention mechanism to achieve long-distance dependency modeling. It supports directly outputting the image planar position and size information of the target from preprocessed video images, and has the characteristics of high detection accuracy and real-time compatibility.

[0073] SLAM automatic calibration technology: a spatial perception technology based on the principle of simultaneous localization and mapping. It constructs a preliminary outline of indoor space by analyzing pre-processed long-distance radio beacon positioning data, and automatically generates instruction parameters such as the key area range for image acquisition, preset shooting frequency, and environmental feature point extraction type and accuracy requirements adapted to the scene requirements by combining indoor building structure features and beacon distribution density. This provides a precise spatial perception foundation and technical parameter support for subsequent camera image acquisition, feature point extraction and multi-coordinate system fusion.

[0074] Kalman filter algorithm: a multi-source data fusion technique based on state estimation theory. It constructs a state vector model containing the target's three-dimensional position and velocity, and dynamically fuses the position coordinates acquired by machine vision with long-range radio positioning data. Considering process noise and observation noise, it gradually converges to the optimal estimate through state prediction and update iteration, thereby accurately outputting the fused target position data, providing high-confidence spatiotemporal reference support for subsequent motion trajectory fitting, velocity calculation and target tracking prediction.

[0075] The indoor target localization method based on machine vision and deep learning provided in this application can be applied to, for example... Figure 1 In the application environment shown, sensor 101 and camera 102 are connected to terminal 100 via a network. Terminal 100 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. Sensors can be, but are not limited to, inertial sensors, depth sensors, ambient light sensors, acoustic sensors, and long-range radio beacons.

[0076] In one exemplary embodiment, such as Figure 2 As shown, an indoor target localization method based on machine vision and deep learning is provided, which is then applied to... Figure 1 Taking terminal 101 as an example, the method includes:

[0077] S101: Acquire real-time positioning data transmitted by multiple long-range radio beacons deployed indoors, anchor the positioning data to nodes of the indoor three-dimensional coordinate grid model, and obtain a global spatial reference frame.

[0078] Terminal 100 acquires real-time positioning data transmitted by multiple long-range radio beacons deployed indoors. Long-range radio beacons are deployed in different areas of the room, and each beacon transmits positioning data containing its own identifier at a preset frequency. A three-dimensional coordinate mesh model of the room is constructed, generated based on the building's structural parameters and containing multiple discrete nodes. The positioning data transmitted by each beacon is associated with the corresponding nodes in the three-dimensional coordinate mesh model, mapping the positioning data to three-dimensional spatial positions, thereby establishing a global spatial reference frame.

[0079] S102: Acquire real-time video images and preprocess them to obtain preprocessed video images.

[0080] For example, real-time video images are acquired by deploying an indoor camera 102, with the image acquisition device covering key indoor areas to ensure comprehensive target capture. The terminal 100 performs preprocessing operations on the acquired real-time video images, including noise suppression, contrast adjustment, and size normalization, to improve image quality. Specifically, blurred, distorted, and noisy features in the original video image are transformed into a high-definition, geometrically accurate, noise-suppressed, and consistent resolution preprocessed video image.

[0081] S103: The LW-DETR algorithm is used to detect targets in the preprocessed video image to obtain the initial position and size information of the targets in the image plane.

[0082] The targets in the preprocessed video images can be indoor entities whose spatial location needs to be obtained in real time, such as pedestrians, indoor mobile machinery and facilities, cargo boxes in warehouse scenes, or office equipment in office areas. For example, the terminal can use the LW-DETR algorithm to perform target detection on the preprocessed video images, identify target objects in the images, and output the coordinate information of the target in the image plane as initial position information and parameters that can characterize the range occupied by the target in the image as size information.

[0083] S104: Based on the initial position and size information, calculate the distance and azimuth between the target and each long-range radio beacon using the signal characteristics of the long-range radio beacon to obtain the spatial position association data of the long-range radio beacon.

[0084] For example, based on the initial position and size information of the target, combined with the signal propagation characteristics of long-range radio beacons, including signal attenuation patterns and angular measurement references, the straight-line distance between the target and each long-range radio beacon, as well as the azimuth angle of the target relative to each beacon, are calculated by analyzing the reflection or obstruction relationship of the beacon signal by the target. These straight-line distances and azimuth angles are then integrated into spatial position association data of the long-range radio beacons.

[0085] S105: Combine the initial position information, size information, and spatial location association data of long-distance radio beacons with the global spatial reference frame to perform spatial registration and multi-source data fusion to obtain the target's position information.

[0086] For example, terminal 100 integrates the target's initial position information, size information, and spatial location association data of long-range radio beacons, and performs spatial registration based on a global spatial reference frame to align data from different sources in the same three-dimensional spatial coordinate system. The above data is then synthesized using a multi-source data fusion algorithm, which weighs the reliability weights of each data point during the fusion process, and outputs the target's position coordinates in three-dimensional space.

[0087] S106: Based on the location coordinates and long-range radio positioning data, track and predict the target's motion state to obtain the target's trajectory and speed information within a preset time period.

[0088] The target's motion state can include real-time position change trends, instantaneous speed, direction of motion, and continuity between positions at different times. For example, terminal 100 uses the target's position coordinates and long-range radio positioning data, employing a motion state tracking algorithm to continuously monitor the target's motion. Combining the target's historical motion data with real-time position information, it predicts the target's motion path within a preset time period as its trajectory, and calculates the target's displacement rate within that time period as its velocity information.

[0089] S107: Integrate the position coordinates, motion trajectory, and velocity information within a global spatial reference frame to obtain the final predicted position of the target.

[0090] For example, the target's position coordinates, trajectory, and velocity information are uniformly integrated within a global spatial reference framework, ensuring consistency across spatial and temporal dimensions during the integration process. By comprehensively analyzing this information, the target's position at the end of a preset time period is determined as its final predicted position.

[0091] The technical solution provided in this application includes the following technical effects: First, real-time positioning data of multiple long-range radio beacons deployed indoors is acquired through an indoor target positioning method. These data are then anchored to nodes of an indoor three-dimensional coordinate grid model to obtain a global spatial reference frame. Multi-beacon collaboration reduces dependence on single hardware and avoids positioning interruptions due to single-device failures. Next, real-time video images are acquired and pre-processed to optimize quality and resist the effects of sudden changes in illumination. The LW-DETR algorithm is used to detect targets and obtain initial position and size information to reduce errors in the face of high-density occlusion. The distance and azimuth angle between the target and each beacon are calculated by combining beacon signal characteristics. Multi-source data is registered and fused with the global frame to counteract electromagnetic interference and improve accuracy. The target's motion state is tracked and predicted, and the information is integrated to obtain the final predicted position, ensuring continuous and accurate positioning. This significantly reduces hardware collaboration dependence, enhances anti-interference capabilities in extreme environments, and significantly improves the stability and accuracy of indoor target positioning.

[0092] Based on the above embodiments, positioning data transmitted in real time from multiple long-range radio beacons deployed indoors is acquired, and the positioning data is anchored to nodes of an indoor three-dimensional coordinate grid model to obtain a global spatial reference frame, including:

[0093] Step 201: Obtain the real-time positioning data transmitted by multiple long-range radio beacons deployed indoors, and use a filtering algorithm to denoise the positioning data to obtain the denoised positioning data.

[0094] Optionally, multiple long-range radio beacons are deployed in a pre-defined indoor area. Each beacon continuously transmits positioning data containing its unique identifier and initial location information at a set period. The terminal device collects the positioning data transmitted by each beacon in real time through a wireless receiving module. For random noise present in the collected positioning data, a Kalman filter algorithm is used for processing. The calculation formula is as follows:

[0095] k = ko +K k (z k -H ko )

[0096] in, k The location data at time k is after denoising. ko Let K be the prior state estimate at time k. k For Kalman gain, z k Let H be the original positioning observation data at time k, and H be the observation matrix. The real-time acquired positioning data is dynamically corrected to effectively reduce the impact of random noise on the data accuracy, resulting in denoised positioning data. The denoised positioning data reduces the random noise in the original positioning data compared to the original data, resulting in higher data accuracy and more accurately reflecting the actual location information of the long-distance radio beacon deployed indoors.

[0097] Step 202: Remove abnormal data from the noise-reduced positioning data using data validity verification rules to obtain preprocessed long-range radio beacon positioning data.

[0098] For example, the data validity verification rules may include data threshold judgment, temporal consistency check, and data fluctuation range limitation. The data threshold judgment is set based on the signal strength and location data range when the beacon is operating normally. The temporal consistency check excludes abrupt data changes by comparing the trends of data at adjacent times. The data fluctuation range limitation filters data that exceeds a reasonable fluctuation range. Through these verification rules, outliers and invalid data in the positioning data are removed, resulting in preprocessed long-range radio beacon positioning data.

[0099] Step 203: Based on the preprocessed long-range radio beacon positioning data, generate image acquisition and feature extraction instructions using SLAM automatic calibration technology.

[0100] For example, terminal 100 initiates SLAM automatic calibration technology based on preprocessed long-range radio beacon positioning data. Terminal 100 constructs a preliminary indoor spatial outline by analyzing the beacon positioning data, and determines key areas and frequency parameters for image acquisition by combining beacon distribution density and indoor structural features. Based on the key areas and frequency parameters for image acquisition, it generates image acquisition and feature extraction instructions, which include the camera's shooting area range, preset shooting frequency, and the type and accuracy requirements for extracting environmental feature points.

[0101] Step 204: Send the image acquisition and feature extraction command to the camera. The image acquisition and feature extraction command is used to instruct the camera to capture indoor environmental images at a preset frequency and extract environmental feature points to obtain an environmental feature point set.

[0102] For example, terminal 100 sends the generated image acquisition and feature extraction instructions to camera 102 deployed indoors. Upon receiving the instructions, camera 102 continuously acquires images of the indoor environment according to the shooting frequency set in the instructions, covering the key areas specified in the instructions. Based on the feature extraction requirements in the instructions, camera 102 extracts feature points from each frame of the image. The extracted environmental feature points include, but are not limited to, corner points, edge points, and texture feature points in the image. Terminal 100 organizes all extracted environmental feature points according to the acquisition time sequence and spatial location to obtain an environmental feature point set.

[0103] Step 205: Establish the spatial relationship between the environmental feature point set and each beacon using a feature point matching algorithm to obtain feature point beacon spatial association data.

[0104] For example, terminal 100 uses the ORB algorithm to process the set of environmental feature points and beacon location information, calculates the environmental feature descriptor for each environmental feature point and the location feature descriptor corresponding to the beacon location information, compares the similarity between the environmental feature descriptors and the location feature descriptors, and determines the matching relationship. The successfully matched environmental feature points are then data-bound with the corresponding beacon location information to form feature point-beacon spatial association data, establishing a mapping between environmental features and beacon spatial locations.

[0105] Step 206: Based on the feature point beacon spatial association data, use a coordinate transformation algorithm to calculate the mapping relationship between the camera and the beacon coordinate system, and obtain the mapping parameters.

[0106] For example, based on the feature point beacon spatial correlation data, the terminal 100 uses a coordinate transformation algorithm to calculate the mapping relationship between the camera coordinate system and the beacon coordinate system, the expression of which is:

[0107] =R +t

[0108] in, Let R be the coordinates of the same feature point in the beacon coordinate system, R be the rotation matrix, and t be the translation vector. Let R be the coordinates of the feature point in the camera coordinate system, R be the rotation matrix, and t be the translation vector. The mapping parameters between R and t are obtained by solving for R and t using the least squares method.

[0109] Step 207: Based on the mapping parameters, fuse and unify the local coordinate systems corresponding to multiple beacons to obtain the unified spatial coordinate system parameters.

[0110] For example, terminal 100 performs fusion calibration on the local coordinate systems of multiple beacons based on mapping parameters, selects one beacon coordinate system as the reference, and transforms the position data in the local coordinate systems of other beacons to the reference coordinate system through coordinate transformation. During the transformation process, the mapping parameters are used to correct deviations. An error optimization algorithm is used to eliminate the coordinate errors after transformation, resulting in unified spatial coordinate system parameters.

[0111] Step 208: Obtain the actual physical dimensions of the indoor environment. Based on the unified spatial coordinate system parameters, construct a uniform three-dimensional coordinate grid for the indoor space using a three-dimensional mesh modeling method to obtain a three-dimensional coordinate grid model of the indoor space.

[0112] For example, the terminal 100 needs to use building information modeling data parsing technology or on-site 3D laser scanning equipment to acquire the actual physical size parameters of the indoor environment. This involves collecting comprehensive spatial information on the length, width, height of the indoor space, as well as the building structure such as walls, columns, doors, and windows, to ensure that the physical size data and 3D coordinate range are obtained with millimeter-level accuracy. Based on the unified spatial coordinate system parameters and combined with the positioning scene accuracy requirements, the granularity and spatial boundary range of the 3D grid are dynamically determined. Through voxelization modeling algorithm, the continuous physical space is discretized into uniformly distributed 3D grid nodes, with each grid node corresponding to a unique 3D spatial coordinate, thus constructing a 3D coordinate grid model of the indoor space.

[0113] Step 209: Integrate the unified spatial coordinate system parameters into the three-dimensional coordinate grid model, bind the beacon position to the grid node through the coordinate anchoring algorithm, and establish a global spatial reference frame.

[0114] Among them, the global spatial reference frame is an indoor spatial reference system with a unique spatiotemporal reference, which is constructed by anchoring long-distance radio beacon positioning data to grid nodes and integrating unified spatial coordinate system parameters based on an indoor three-dimensional coordinate grid model. It provides a unified spatial reference for the spatial registration and fusion of multi-source information such as machine vision image data and radio positioning data, and supports the accurate realization of positioning functions such as target position calculation and motion trajectory prediction.

[0115] For example, terminal 100 integrates the unified spatial coordinate system parameters into a three-dimensional coordinate grid model to establish a global coordinate reference for the grid model. It processes each long-distance radio beacon using a coordinate anchoring algorithm, calculates the Euclidean distance between the beacon's three-dimensional coordinates and all grid nodes, and uses the algorithm to quickly locate the grid node with the smallest distance from the beacon. It then establishes and stores the binding relationship between the beacon identifier and the grid node. A reprojection error verification mechanism is used to reverse-map the beacon positioning data to image feature points and calculate the position deviation after coordinate transformation. If the deviation exceeds a preset threshold, a dynamic adjustment strategy for grid granularity is triggered. Using the bound grid nodes as spatial anchors, a global spatial reference frame with a unique spatiotemporal reference is constructed.

[0116] In one embodiment of the present invention, acquiring a real-time video image and preprocessing the real-time video image to obtain a preprocessed video image includes:

[0117] Step 301: Acquire the real-time video stream, extract continuous video frames at a preset frame rate, and timestamp each video frame to obtain a timestamped video frame.

[0118] For example, when the terminal 100 acquires a real-time video stream, it constructs a distributed edge computing cluster and deploys it at key indoor nodes. Each node is equipped with a lightweight video acquisition module and a local preprocessing unit. The video stream is split into frames in parallel at a preset frame rate. A multi-threaded pipeline architecture is used to process multiple video streams simultaneously. A millisecond-level timestamp based on network time protocol is added to each frame. The timestamp generation module is hard-connected to the time synchronization system of the indoor long-distance radio beacon to ensure that the time deviation between the image frame timestamp and the beacon positioning data is less than 1ms. Each extracted video image is given a unique time sequence identifier, resulting in a video frame with a timestamp.

[0119] Step 302: Perform median filtering on the timestamped video frames to denoise them, and obtain the denoised video frames.

[0120] For example, the terminal 100 uses a median filtering algorithm to denoise the interference such as salt and pepper noise and impulse noise that may exist in the video frame with timestamp. It selects a 5×5 square pixel neighborhood window, traverses each pixel in the video frame, sorts the gray values ​​of all pixels within the window coverage area in ascending order, takes the median value after sorting as the new gray value of the center pixel of the window, and replaces the original gray value of the pixel to obtain the video frame with timestamp.

[0121] Step 303: Perform global histogram equalization on the denoised video frames to obtain denoised and enhanced video frames.

[0122] For example, terminal 100 performs global histogram equalization on the denoised video frames. By adjusting the discrete distribution of image grayscale values, the grayscale levels are uniformly distributed throughout the entire dynamic range. The grayscale transformation formula is: S k =

[0123] Among them, S k Let L be the target gray value corresponding to the transformed gray level k, L be the total number of gray levels in the image (here, we take 256), and M×N be the total number of pixels in the video frame. This represents the number of pixels at gray level i in the original video frame. The cumulative number of pixels from gray level 0 to k is used to calculate the target gray value corresponding to each original gray level. The gray level information of the video frame is updated pixel by pixel, which significantly improves the overall contrast and detail clarity of the image, resulting in a denoised and enhanced video frame.

[0124] Step 304: Correct the radial distortion of the video frame after denoising and enhancement to obtain the image after distortion correction.

[0125] For example, due to manufacturing errors, assembly deviations, and optical characteristics of the lens in the image acquisition device, the denoised and enhanced video frames are prone to radial distortion, which needs to be corrected based on a high-precision distortion model. A second-order radial distortion correction model is adopted, and the formula is:

[0126] r corr =r dist (1+k1r) 2 dist +k2r 4 dist )

[0127] Where, r dist r is the distance from a pixel in a distorted video frame to the principal point of the image. corr To correct the distance from the pixel to the principal point, k1 and k2 are the radial distortion coefficients of the camera obtained in advance through Zhang's calibration method. The corrected coordinates of each distorted pixel are calculated. The original gray value of the distorted pixel is replaced with the gray value of the pixel corresponding to the corrected coordinates through the coordinate mapping relationship. The radial distortion correction of the video frame is completed pixel by pixel to obtain the image after distortion correction.

[0128] Step 305: Restore the original geometric shape of the distortion-corrected image using a bilinear interpolation algorithm to obtain the preprocessed video image.

[0129] For example, during distortion correction, the adjustment of pixel coordinates may cause geometric shape shifts and pixel loss in some areas of the image. It is necessary to restore the original geometric shape of the image by using a bilinear interpolation algorithm. For the target pixel that needs to be supplemented with gray values ​​after distortion correction, the four adjacent effective pixels in the image are determined. The weight coefficient of each adjacent pixel is calculated based on the coordinate distance between the target pixel and the four adjacent pixels. The gray values ​​of the four adjacent pixels are multiplied by the corresponding weight coefficients and summed to obtain the gray value of the target pixel. This gray value is then filled into the target pixel position. The restored image geometry is consistent with the actual indoor scene, resulting in the preprocessed video image.

[0130] In one embodiment of the present invention, the LW-DETR algorithm is used to detect targets in a preprocessed video image to obtain the initial position and size information of the targets in the image plane, including:

[0131] Step 401: Use the LW-DETR algorithm to detect targets in the preprocessed video image to obtain target candidate region images.

[0132] For example, terminal 100 inputs the preprocessed video image into the LW-DETR algorithm model. By lightweighting and optimizing the Transformer structure of the traditional DETR model, the feature processing dimension is reduced and the attention calculation process is simplified to adapt to the real-time detection requirements of indoor scenes. Global features are initially extracted from the input video image. A preset target proposal generation mechanism filters regions in the image whose pixel grayscale and texture features match the target feature distribution, excluding background regions that clearly do not contain the target. A series of rectangular regions that may contain the target are identified, and these regions are used as target candidate region images.

[0133] Step 402: Extract hierarchical features from the target candidate region image using a lightweight convolutional backbone network to obtain multi-scale feature maps.

[0134] For example, terminal 100 uses a lightweight convolutional backbone network to perform hierarchical feature extraction on the target candidate region image. The lightweight convolutional backbone network is built based on lightweight convolutional modules, which retains key feature information while reducing the amount of computation. The network extracts features from the shallow layer of the candidate region image. The shallow convolutional layers focus on capturing the edge, texture and other detailed features of the target to obtain shallow feature maps. The deep convolutional layers aggregate feature information through multiple sets of convolution and pooling operations to generate high-level features containing target semantic information to obtain deep feature maps. By acquiring feature images with different resolutions output from different convolutional layers of the backbone network, they are integrated to form multi-scale feature maps.

[0135] Step 403: Input the multi-scale feature maps into the Transformer encoder for context modeling, capture long-distance dependencies through a multi-head self-attention mechanism, and obtain context-enhanced feature representations.

[0136] For example, multi-scale feature maps are input into a Transformer encoder. The encoder first performs dimensionality unification on the feature maps at each scale to ensure that the feature data format is compatible with the attention computation requirements. Internally, the encoder processes the features in parallel through a multi-head self-attention mechanism, dividing the feature vector into multiple sub-vectors to construct attention weight matrices. Each attention head focuses on the correlation between different positions in the feature map, breaking through the limitations of traditional local feature extraction and capturing long-distance dependencies between the target and the background, as well as between different parts of the target. Through iterative processing of the multi-layer encoder, effective feature signals are gradually strengthened, redundant information is suppressed, and a context-enhanced feature representation is output.

[0137] Step 404: Input the context-enhanced feature representation into the Transformer decoder for target query interaction, and use a dynamic anchor point allocation strategy to determine the target candidate box.

[0138] For example, terminal 100 inputs the context-enhanced feature representation into the Transformer decoder, along with a preset number of target query vectors. The decoder uses a cross-attention mechanism to interactively update the target query vectors and the encoder's output features, gradually aligning the query vectors with the feature distribution of the target in the image. A dynamic anchor point allocation strategy is employed, dynamically generating anchor points adapted to different target shapes based on the feature similarity of the updated target query vectors and the possible size distribution of the target, rather than using fixed-size anchor points. By calculating the matching degree between the anchor points and the target features, anchor points with matching scores higher than a preset threshold are selected and identified as target candidate boxes.

[0139] Step 405: Perform multi-scale prediction processing on the target candidate boxes to obtain the preliminary localization of the target candidate boxes.

[0140] For example, multi-scale feature maps are associated with target candidate boxes. Each scale feature map outputs a corresponding predicted location value and confidence score for the multi-scale candidate box. The prediction results are output by weighted fusion of feature maps of different scales. The prediction results of high-resolution feature maps are given higher weights to improve location accuracy, while the prediction results of low-resolution feature maps help optimize the overall target localization. The terminal 100 filters the candidate boxes based on the fused confidence scores, eliminating candidate boxes with confidence scores below a set threshold, retaining candidate boxes with higher confidence scores, and determining their approximate coordinate range in the image plane to obtain the preliminary localization of the target candidate boxes.

[0141] Step 406: Process the initial positioning through bounding box regression to obtain initial position and size information.

[0142] For example, the initial localization of the target candidate box is optimized based on the bounding box regression algorithm. Using the coordinates of the initially located candidate box as initial values, a regression loss function is constructed to measure the deviation between the candidate box and the actual position of the target. The regression parameters are iteratively updated through backpropagation, adjusting the center coordinates, width, and height of the candidate box to gradually conform to the actual contour of the target in the image. When the loss function value converges to a preset range or the number of iterations reaches a set threshold, parameter updates are stopped. At this point, the coordinates and size of the candidate box accurately reflect the position and size of the target in the image plane, deriving the position and size information of the candidate box, thus obtaining the initial position and size information of the target.

[0143] In one embodiment of the present invention, based on initial position information and size information, the distance and azimuth between the target and each long-range radio beacon are calculated using the signal characteristics of the long-range radio beacon, to obtain spatial position association data of the long-range radio beacons, including:

[0144] Step 501: Based on the initial position and size information, extract the image coordinates of the target center point and the size parameters of the target's bounding rectangle to obtain the target's basic feature data.

[0145] For example, based on the initial position and size information of the target, the terminal 100 determines the bounding box coordinate range of the target in the image plane, obtains the x-coordinate of the center point by calculating the average of the horizontal left and right coordinates of the bounding box, and obtains the y-coordinate of the center point by calculating the average of the vertical up and down coordinates, thus obtaining the image coordinates of the target center point. Based on the size information, the width and height parameters of the target's circumscribed rectangle are determined, where the width is the difference in the horizontal coordinates of the bounding box and the height is the difference in the vertical coordinates. The image coordinates of the target center point are integrated with the width and height parameters of the circumscribed rectangle to obtain the target's basic feature data.

[0146] Step 502: Based on the center point image coordinates of the target's basic feature data and the camera intrinsic parameter matrix, calculate the initial estimate of the target's three-dimensional position in the camera coordinate system through inverse perspective projection transformation, and obtain the target's position in the camera coordinate system.

[0147] For example, terminal 100 calls a pre-stored camera intrinsic parameter matrix, inputs the center point image coordinates from the target's basic feature data, constructs an inverse perspective projection transformation model, and converts the two-dimensional coordinates in the image plane into three-dimensional coordinates in the camera coordinate system through the inverse perspective projection transformation model, eliminating the coordinate deviation caused by perspective projection during imaging, and calculates the x, y, and z axis coordinates of the target in the camera coordinate system, which is the target's position in the camera coordinate system. The intrinsic parameter matrix contains key parameters such as the camera's focal length and the coordinates of the principal point of the image, reflecting the optical characteristics and geometric relationships of the camera imaging.

[0148] Step 503: Obtain the three-dimensional position coordinate data of each long-range radio beacon deployed indoors in the global spatial reference frame to obtain the beacon spatial coordinate set.

[0149] For example, from the constructed global spatial reference framework, the terminal 100 extracts the three-dimensional position coordinate data of each long-distance radio beacon deployed indoors, verifies the consistency between the coordinate data of each beacon and the physical deployment location of the beacon, and ensures that the coordinate data can accurately reflect the actual location of the beacon in the indoor space. All the verified three-dimensional coordinates of the beacons are classified and organized according to the unique beacon identifier to form a beacon spatial coordinate set.

[0150] Step 504: Based on the beacon spatial coordinate set and the target camera coordinate system position, and combined with the long-distance radio signal time difference model, calculate the distance data from the target to each beacon to obtain the target-to-beacon distance set.

[0151] For example, based on the three-dimensional coordinates of each beacon in the beacon spatial coordinate set and the position of the target camera coordinate system, a long-distance radio signal time difference model is initiated. The signal acquisition module captures the propagation time difference of radio signals reflected or blocked by the target between each beacon. Combining the propagation speed of radio waves in the indoor environment, the linear correspondence between propagation time difference and distance is used to calculate the straight-line distance from the target to each beacon. The calculated distance data is verified for reasonableness, and abnormal distance values ​​that exceed the physical range of the indoor space are excluded. The valid distance values ​​are sorted in order according to the corresponding beacon identifier to obtain the set of distances from the target to the beacons.

[0152] Step 505: Using a spatial geometric orientation calculation algorithm, combined with the position coordinates of each beacon and the distance data from the target to the beacon, calculate the azimuth data of the target relative to each beacon, and obtain the target relative azimuth set.

[0153] For example, a spatial geometric orientation calculation algorithm is adopted. Each long-distance radio beacon is used as a calculation reference point. Based on the three-dimensional coordinates of the beacon in the beacon spatial coordinate set and the distance data from the target to the beacon, a spatial geometric model containing the target position is constructed with the beacon as the origin. The horizontal and vertical angles of the target relative to the beacon are calculated through the spatial geometric model. The horizontal angle is the angle between the target on the horizontal plane and the preset reference direction, and the vertical angle is the angle between the target in the vertical plane and the horizontal plane. The two together constitute the azimuth angle of the target relative to the beacon. The above calculation process is repeated for each beacon to ensure that the azimuth angle data matches the distance data of the corresponding beacon. The azimuth angles corresponding to all beacons are organized to form a set of target relative azimuth angles.

[0154] Step 506: Perform spatiotemporal alignment processing on the set of distances from the target to the beacon and the corresponding set of azimuth angles, and integrate them into spatial location association data of long-range radio beacons, thus obtaining spatial location association data of long-range radio beacons.

[0155] For example, spatiotemporal alignment processing is performed on the set of distances from targets to beacons and the corresponding set of relative azimuth angles of targets. Based on the timestamp information generated by the system during data acquisition, the distance data and azimuth angle data corresponding to the same beacon are matched in the time dimension to ensure that the two types of data are consistent in the acquisition time sequence, eliminating the spatiotemporal misalignment problem caused by data acquisition delay or transmission deviation. The spatiotemporally aligned distance data and azimuth angle data are bound with the unique identification information of the corresponding beacon to clarify the association between each data and the beacon. At the same time, the integrity and validity of the data are verified, and invalid data with missing association information or exceeding the reasonable range are eliminated. Finally, the spatial location association data of long-distance radio beacons is integrated to form the spatial location association data of long-distance radio beacons.

[0156] In one embodiment of the present invention, initial position information, size information, and spatial position association data of long-range radio beacons are combined with a global spatial reference frame for spatial registration and multi-source data fusion to obtain the target's position information, including:

[0157] Step 601: Based on the spatial coordinate mapping rules of the global spatial reference frame, the initial position information, size information and spatial location association data of long-distance radio beacons are unified in dimension through coordinate transformation algorithm to obtain multi-source data registration results.

[0158] For example, based on the spatial coordinate mapping rules preset by the global spatial reference frame, the conversion logic of image plane coordinates, beacon local coordinates and global coordinate system is clarified. The terminal 100 converts the initial position information, target size information and spatial position association data of long-distance radio beacons to the global coordinate system through coordinate transformation algorithm. The two-dimensional coordinates of the image plane are converted into three-dimensional coordinates by inverse perspective projection combined with the depth parameters of the global coordinate system. The initial position information, size information and spatial position association data of long-distance radio beacons are converted into preliminary three-dimensional data of the target relative to the global coordinate system, and the multi-source data registration result is obtained.

[0159] Step 602: For the registration results of multi-source data, a preset multi-source information fusion algorithm is used to perform data association and weight allocation, and the fusion accuracy is optimized by combining the spatiotemporal consistency constraint mechanism to obtain the fused target spatial coordinates.

[0160] For example, terminal 100 uses a preset weighted multi-source information fusion algorithm to associate the three-dimensional coordinates of the image source and the three-dimensional coordinates of the beacon source in the multi-source data registration result, and the fused target spatial coordinates P fusion The calculation formula is:

[0161] P fusion =w1P1+w2P2

[0162] Where w1 is the image data weight, which is obtained by mapping the LW-DETR target detection confidence value, and w2 is the beacon data weight that satisfies w1+w2=1. In this process, the spatiotemporal consistency constraint mechanism is combined to compare the temporal continuity of the current fused coordinates with the coordinates at adjacent time points and the spatial correlation with the global frame grid nodes, eliminate abnormal fusion results that exceed the reasonable spatiotemporal range, optimize the fusion accuracy, and obtain the fused target spatial coordinates.

[0163] Step 603: Perform confidence assessment and error compensation on the fused target spatial coordinates to obtain the corrected spatial coordinates.

[0164] For example, a confidence assessment method based on data dispersion is adopted to calculate the mean deviation between the coordinates of each independent data source and the fused target spatial coordinates in the multi-source data registration result. Based on the magnitude of the mean deviation, three confidence levels of high, medium and low are defined. When the confidence level is in the medium or low level, error compensation parameters pre-calibrated based on the indoor scene are called. The error compensation parameters cover the signal attenuation coefficient and image detection deviation value of different regions. By superimposing the fused coordinates with the error compensation parameters of the corresponding regions, the system deviation caused by environmental interference is corrected, and the corrected spatial coordinates are obtained.

[0165] Step 604: The corrected spatial coordinates are calibrated using a coordinate correction algorithm to correct the position deviation and obtain the target's position information.

[0166] For example, the corrected spatial coordinates are input into the coordinate correction algorithm. The coordinate correction algorithm uses the three-dimensional coordinate grid model nodes in the global spatial reference frame as a reference, calculates the Euclidean distance between the corrected coordinates and the nearest grid node, and calibrates the distance deviation according to the absolute coordinate accuracy of the grid node. If the deviation value exceeds the preset ratio of the grid node spacing, the corrected coordinates are adjusted to meet the range of grid spatial accuracy requirements to ensure that the target coordinates correspond accurately with the actual physical space position in the room and obtain the target's position information.

[0167] In one embodiment of the present invention, the motion state of a target is tracked and predicted based on its location coordinates and long-range radio positioning data to obtain the target's trajectory and speed information, including:

[0168] Step 701: Use the Kalman filter algorithm to fuse the position coordinates and long-range radio positioning data to obtain fused position data; wherein, the state vector of the Kalman filter algorithm contains three-dimensional position and velocity, and the observation vector contains position coordinates and long-range radio positioning results.

[0169] For example, the Kalman filter algorithm is used to fuse the target's position coordinates with long-range radio positioning data. The filtered state vector is defined as containing the target's three-dimensional position (x, y, z) and three-dimensional velocity (v). x ,v y ,v z The observation vector is a 6-dimensional vector containing the position coordinates obtained from machine vision and the results of long-range radio positioning. The filtering process first performs state prediction based on the state vector and state transition matrix of the previous time step. The core state prediction formula is:

[0170] =A +B

[0171] in, Let A be the prior state estimate at time k, and let A be a 6×6 state transition matrix. The position dimension coefficient and velocity dimension coefficient on the diagonal are both 1, and the cross-dimensional coefficient between position and velocity is the sampling time interval Δt. Let B be the posterior state estimate at time k-1, and let B be the control input matrix. To control the input vector, the prior state is updated by combining the observation matrix and Kalman gain to obtain the fused position data.

[0172] Step 702: Based on the fused position data, fit the motion trajectory using the least squares method to obtain the fitted trajectory curve.

[0173] For example, based on the fused position data, the target's three-dimensional coordinates (x, y, y) at different times are extracted. t ,y t ,z t Time series data is generated, with time t as the independent variable and three-dimensional coordinates as the dependent variables. The least squares method is used to construct a trajectory fitting function. By calculating the function coefficients that minimize the sum of squared residuals between the coordinate observations and the predicted values ​​of the fitting function, the fitting curves of each dimension are determined. The fitting curves of the three dimensions are integrated to obtain a fitting trajectory curve that can reflect the historical movement trend of the target.

[0174] Step 703: Construct a linear regression model by fitting the trajectory curve.

[0175] For example, a linear regression model is constructed by fitting a trajectory curve, with time t as the independent variable and the target coordinates at each moment on the fitted trajectory curve as the dependent variable. A univariate linear regression equation is established, and the slope and intercept of the regression equation are determined by calculating the covariance and variance of the independent and dependent variables. Similarly, linear regression models in the y and z directions are constructed to form a linear regression model that can characterize the motion laws of the target in various dimensions.

[0176] Step 704: Predict the future trajectory of the target using a long short-term memory network model based on a linear regression model, optimize the future trajectory using a particle filter algorithm, and obtain the target's motion trajectory and velocity information.

[0177] For example, the terminal 100 divides the historical fused location data into training and validation sets according to the time series based on the target motion trend parameters output by the linear regression model, and inputs them into the long short-term memory network model for training, so that the model learns the temporal dependence of the target motion. The trained model is used to predict the future three-dimensional coordinate sequence of the target within a preset time period to obtain a preliminary future trajectory. The particle filtering algorithm is used to optimize the preliminary future trajectory. By initializing multiple sets of particles to represent possible future trajectories, updating particle weights according to the matching degree between the trajectory and the actual motion law, and resampling to retain high-weight particles, a precise target motion trajectory is obtained. At the same time, the velocity information of the target at each moment is calculated based on the coordinate difference and time interval of adjacent moments on the trajectory.

[0178] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0179] In one embodiment of the present invention, such as Figure 3 Furthermore, this application also provides an indoor target localization system 800 based on machine vision and deep learning, comprising:

[0180] Module 801 is constructed to acquire real-time positioning data transmitted by multiple long-range radio beacons deployed indoors, and anchors the positioning data to nodes of the indoor three-dimensional coordinate grid model to obtain a global spatial reference frame.

[0181] Preprocessing module 802 is used to acquire real-time video images and preprocess them to obtain preprocessed video images;

[0182] The target detection module 803 is used to detect targets in the preprocessed video image using the LW-DETR algorithm to obtain the initial position and size information of the target in the image plane;

[0183] The spatial correlation calculation module 804 is used to calculate the distance and azimuth between the target and each long-range radio beacon based on the initial position information and size information and the signal characteristics of the long-range radio beacon, so as to obtain the spatial position correlation data of the long-range radio beacon;

[0184] The position calculation module 805 is used to combine the initial position information, size information and spatial position association data of long-distance radio beacons with the global spatial reference frame to perform spatial registration and multi-source data fusion to obtain the position information of the target.

[0185] The motion state detection module 806 is used to track and predict the motion state of the target based on the position coordinates and long-range radio positioning data, and obtain the target's motion trajectory and speed information within a preset time period;

[0186] The position integration module 807 is used to integrate position coordinates, motion trajectory and velocity information in a global spatial reference frame to obtain the final predicted position of the target.

[0187] In one embodiment of the present invention, the construction framework module 801 is further configured to:

[0188] The positioning data transmitted in real time by multiple long-range radio beacons deployed indoors is acquired, and the positioning data is denoised using a filtering algorithm to obtain the denoised positioning data.

[0189] Abnormal data in the noise-reduced positioning data is removed by using data validity verification rules to obtain preprocessed long-range radio beacon positioning data;

[0190] Based on the preprocessed long-range radio beacon positioning data, image acquisition and feature extraction instructions are generated using SLAM automatic calibration technology;

[0191] The image acquisition and feature extraction command is sent to the camera. The image acquisition and feature extraction command is used to instruct the camera to capture indoor environment images at a preset frequency and extract environmental feature points to obtain an environmental feature point set.

[0192] The spatial relationship between the set of environmental feature points and each beacon is established by using a feature point matching algorithm, thus obtaining spatial correlation data between feature points and beacons.

[0193] Based on the spatial correlation data of feature points and beacons, a coordinate transformation algorithm is used to calculate the mapping relationship between the camera and the beacon coordinate system, and the mapping parameters are obtained.

[0194] Based on the mapping parameters, the local coordinate systems corresponding to multiple beacons are fused and uniformly calibrated to obtain the unified spatial coordinate system parameters.

[0195] Obtain the actual physical dimensions of the indoor environment, and construct a uniform three-dimensional coordinate grid of the indoor space using a three-dimensional mesh modeling method based on the unified spatial coordinate system parameters to obtain the three-dimensional coordinate grid model of the indoor space.

[0196] The unified spatial coordinate system parameters are integrated into the three-dimensional coordinate grid model, and the beacon position is bound to the grid node through the coordinate anchoring algorithm to establish a global spatial reference frame.

[0197] In one embodiment of the present invention, the preprocessing module 802 is further configured to:

[0198] Acquire real-time video stream, extract continuous video frames at a preset frame rate, and timestamp each video frame to obtain a timestamped video frame.

[0199] Median filtering is applied to the timestamped video frames to denoise them, resulting in denoised video frames.

[0200] The denoised video frames are subjected to global histogram equalization to obtain the denoised and enhanced video frames.

[0201] Correcting radial distortion in the denoised and enhanced video frame yields a distortion-corrected image.

[0202] The original geometric shape of the image after distortion correction is restored by bilinear interpolation algorithm to obtain the preprocessed video image.

[0203] In one embodiment of the present invention, the target detection module 803 is further configured to:

[0204] The LW-DETR algorithm is used to detect targets in the preprocessed video image to obtain target candidate region images;

[0205] A lightweight convolutional backbone network is used to extract hierarchical features from the target candidate region image to obtain multi-scale feature maps.

[0206] Multi-scale feature maps are input into the Transformer encoder for context modeling, and long-distance dependencies are captured through a multi-head self-attention mechanism to obtain context-enhanced feature representations.

[0207] The context-enhanced feature representation is input into the Transformer decoder for target query interaction, and a dynamic anchor point allocation strategy is used to determine the target candidate box.

[0208] Multi-scale prediction processing is performed on the target candidate boxes to obtain the preliminary localization of the target candidate boxes;

[0209] The initial positioning is processed by bounding box regression to obtain initial position and size information.

[0210] In one embodiment of the present invention, the spatial association calculation module 804 is further configured to:

[0211] Based on the initial position and size information, the image coordinates of the target center point and the size parameters of the target's bounding rectangle are extracted to obtain the target's basic feature data;

[0212] Based on the center point image coordinates of the target's basic feature data and the camera's intrinsic parameter matrix, the initial estimate of the target's three-dimensional position in the camera coordinate system is calculated through inverse perspective projection transformation, thus obtaining the target's position in the camera coordinate system.

[0213] Obtain the three-dimensional position coordinate data of each long-range radio beacon deployed indoors within a global spatial reference frame to obtain the beacon spatial coordinate set;

[0214] Based on the beacon spatial coordinate set and the target camera coordinate system position, the distance data from the target to each beacon is calculated using the long-distance radio signal time difference model, thus obtaining the target-to-beacon distance set.

[0215] By using a spatial geometric orientation calculation algorithm, and combining the position coordinates of each beacon with the distance data from the target to the beacon, the azimuth angle data of the target relative to each beacon is calculated, and the set of target relative azimuth angles is obtained.

[0216] The set of distances from the target to the beacon and the corresponding set of azimuth angles are spatiotemporally aligned and integrated into spatial location association data of long-range radio beacons, thus obtaining spatial location association data of long-range radio beacons.

[0217] In one embodiment of the present invention, the position calculation module 805 is further configured to:

[0218] Based on the spatial coordinate mapping rules of the global spatial reference frame, the initial position information, size information and spatial position association data of long-distance radio beacons are unified in dimension through coordinate transformation algorithm to obtain multi-source data registration results;

[0219] For the registration results of multi-source data, a preset multi-source information fusion algorithm is used to perform data association and weight allocation, and the fusion accuracy is optimized by combining a spatiotemporal consistency constraint mechanism to obtain the fused target spatial coordinates;

[0220] The confidence level of the fused target spatial coordinates is evaluated and error compensation is performed to obtain the corrected spatial coordinates.

[0221] The corrected spatial coordinates are then used to calibrate the position deviation using a coordinate correction algorithm to obtain the target's position information.

[0222] In one embodiment of the present invention, the motion state detection module 806 is further configured to:

[0223] The Kalman filter algorithm is used to fuse position coordinates and long-range radio positioning data to obtain fused position data. The state vector of the Kalman filter algorithm contains three-dimensional position and velocity, and the observation vector contains position coordinates and long-range radio positioning results.

[0224] Based on the fused position data, the motion trajectory is fitted using the least squares method to obtain the fitted trajectory curve;

[0225] A linear regression model is constructed by fitting the trajectory curve;

[0226] The future trajectory of the target is predicted by a linear regression model and a long short-term memory network model. The future trajectory is then optimized using a particle filter algorithm to obtain the target's motion trajectory and velocity information.

[0227] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of a power supply safety management method as described above.

[0228] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0229] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0230] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the embodiments of this application, and these modifications and improvements all fall within the protection scope of the embodiments of this application.

Claims

1. An indoor target localization method based on machine vision and deep learning, characterized in that, The method includes: The positioning data transmitted in real time by multiple long-range radio beacons deployed indoors is obtained, and the positioning data is anchored to the nodes of the indoor three-dimensional coordinate grid model to obtain a global spatial reference frame. Acquire real-time video images and preprocess the real-time video images to obtain preprocessed video images; The LW-DETR algorithm is used to detect targets in the preprocessed video image to obtain the initial position and size information of the targets in the image plane; Based on the initial position information and the size information, the distance and azimuth between the target and each long-range radio beacon are calculated using the signal characteristics of the long-range radio beacon, thereby obtaining the spatial position association data of the long-range radio beacon; The initial location information, the size information, and the spatial location association data of the long-range radio beacon are combined with the global spatial reference frame to perform spatial registration and multi-source data fusion to obtain the location information of the target. Based on the location coordinates and the long-range radio positioning data, the motion state of the target is tracked and predicted to obtain the target's motion trajectory and speed information within a preset time period. The position coordinates, the motion trajectory, and the velocity information are integrated within the global spatial reference frame to obtain the final predicted position of the target.

2. The method according to claim 1, characterized in that, The process of acquiring real-time positioning data from multiple long-range radio beacons deployed indoors, anchoring the positioning data to nodes of an indoor three-dimensional coordinate grid model, and obtaining a global spatial reference frame includes: The positioning data transmitted in real time by multiple long-range radio beacons deployed indoors is acquired, and the positioning data is denoised using a filtering algorithm to obtain denoised positioning data. Abnormal data in the denoised positioning data is removed by data validity verification rules to obtain preprocessed long-range radio beacon positioning data. Based on the preprocessed long-range radio beacon positioning data, image acquisition and feature extraction instructions are generated using SLAM automatic calibration technology; The image acquisition and feature extraction command is sent to the camera. The image acquisition and feature extraction command is used to instruct the camera to capture indoor environment images at a preset frequency and extract environmental feature points to obtain an environmental feature point set. The spatial relationship between the set of environmental feature points and each beacon is established by a feature point matching algorithm to obtain feature point beacon spatial association data; Based on the spatial association data of the feature points and beacons, a coordinate transformation algorithm is used to calculate the mapping relationship between the camera and the beacon coordinate system, and the mapping parameters are obtained. Based on the mapping parameters, the local coordinate systems corresponding to multiple beacons are fused and uniformly calibrated to obtain the unified spatial coordinate system parameters. Obtain the actual physical dimensions of the indoor environment, and construct a uniform three-dimensional coordinate grid of the indoor space using the unified spatial coordinate system parameters to obtain the three-dimensional coordinate grid model of the indoor space. The unified spatial coordinate system parameters are integrated into the three-dimensional coordinate grid model, and the beacon position is bound to the grid node through the coordinate anchoring algorithm to establish a global spatial reference frame.

3. The method according to claim 1, characterized in that, The step of acquiring real-time video images and preprocessing the real-time video images to obtain preprocessed video images includes: The real-time video stream is acquired, and continuous video frames are extracted at a preset frame rate. Each video frame is timestamped to obtain a timestamped video frame. The timestamped video frames are subjected to median filtering for denoising to obtain denoised video frames. The denoised video frames are subjected to global histogram equalization to obtain denoised and enhanced video frames. Correct the radial distortion of the denoised and enhanced video frame to obtain a distortion-corrected image; The original geometric shape of the distortion-corrected image is restored using a bilinear interpolation algorithm to obtain a preprocessed video image.

4. The method according to claim 1, characterized in that, The LW-DETR algorithm is used to detect targets in the preprocessed video image to obtain the initial position and size information of the targets in the image plane, including: The LW-DETR algorithm is used to detect targets in the preprocessed video image to obtain target candidate region images; The lightweight convolutional backbone network is used to perform hierarchical feature extraction on the target candidate region image to obtain multi-scale feature maps; The multi-scale feature maps are input into the Transformer encoder for context modeling. Long-distance dependencies are captured through a multi-head self-attention mechanism to obtain context-enhanced feature representations. The context-enhanced feature representation is input into the Transformer decoder for target query interaction, and a dynamic anchor point allocation strategy is used to determine the target candidate box. Multi-scale prediction processing is performed on the target candidate boxes to obtain the preliminary localization of the target candidate boxes; The initial positioning is processed by bounding box regression to obtain the initial position and the size information.

5. The method according to claim 1, characterized in that, The step of calculating the distance and azimuth between the target and each long-range radio beacon based on the initial position information and the size information, using the signal characteristics of the long-range radio beacons, to obtain spatial position association data of the long-range radio beacons includes: Based on the initial position information and the size information, the image coordinates of the target center point and the size parameters of the target's bounding rectangle are extracted to obtain the target's basic feature data; Based on the center point image coordinates of the target's basic feature data and the camera's intrinsic parameter matrix, the initial estimate of the target's three-dimensional position in the camera coordinate system is calculated by inverse perspective projection transformation, thus obtaining the target's position in the camera coordinate system. Obtain the three-dimensional position coordinate data of each long-range radio beacon deployed indoors within a global spatial reference frame to obtain the beacon spatial coordinate set; Based on the beacon spatial coordinate set and the target camera coordinate system position, and combined with the long-distance radio signal time difference model, the distance data from the target to each beacon is calculated to obtain the target-to-beacon distance set; By using a spatial geometric orientation calculation algorithm, and combining the position coordinates of each beacon with the distance data from the target to the beacon, the azimuth angle data of the target relative to each beacon is calculated, and the set of target relative azimuth angles is obtained. The distance set from the target to the beacon and the corresponding azimuth angle set are spatiotemporally aligned and integrated into spatial location association data of long-range radio beacons, thus obtaining spatial location association data of long-range radio beacons.

6. The method according to claim 1, characterized in that, The step of combining the initial position information, the size information, and the spatial position association data of the long-range radio beacon with the global spatial reference frame for spatial registration and multi-source data fusion to obtain the target's position information includes: Based on the spatial coordinate mapping rules of the global spatial reference frame, the initial position information, the size information and the spatial position association data of the long-distance radio beacon are unified in dimension through a coordinate transformation algorithm to obtain multi-source data registration results. The multi-source data registration results are used to perform data association and weight allocation using a preset multi-source information fusion algorithm, and the fusion accuracy is optimized by combining a spatiotemporal consistency constraint mechanism to obtain the fused target spatial coordinates. The fused target spatial coordinates are subjected to confidence assessment and error compensation to obtain the corrected spatial coordinates. The corrected spatial coordinates are then used to calibrate the position deviation using a coordinate correction algorithm to obtain the target's position information.

7. The method according to claim 1, characterized in that, The step of tracking and predicting the target's motion state based on the location coordinates and the long-range radio positioning data to obtain the target's trajectory and speed information includes: The Kalman filter algorithm is used to fuse the position coordinates and the long-range radio positioning data to obtain fused position data; wherein, the state vector of the Kalman filter algorithm contains three-dimensional position and velocity, and the observation vector contains position coordinates and long-range radio positioning results; Based on the fused position data, the motion trajectory is fitted using the least squares method to obtain the fitted trajectory curve; A linear regression model is constructed using the fitted trajectory curve; The target's future trajectory is predicted using a long short-term memory network model based on the linear regression model, and the future trajectory is optimized using a particle filter algorithm to obtain the target's motion trajectory and velocity information.

8. An indoor target localization system based on machine vision and deep learning, characterized in that, The system includes: A framework module is constructed to acquire real-time positioning data transmitted by multiple long-range radio beacons deployed indoors, and to anchor the positioning data to nodes of an indoor three-dimensional coordinate grid model to obtain a global spatial reference frame. The preprocessing module is used to acquire real-time video images and preprocess the real-time video images to obtain preprocessed video images; The target detection module is used to detect targets in the preprocessed video image using the LW-DETR algorithm to obtain the initial position and size information of the target in the image plane. The spatial correlation calculation module is used to calculate the distance and azimuth between the target and each long-range radio beacon based on the initial position information and the size information, using the signal characteristics of the long-range radio beacon, to obtain the spatial position correlation data of the long-range radio beacon; The location calculation module is used to combine the initial location information, the size information, and the spatial location association data of the long-distance radio beacon with the global spatial reference frame to perform spatial registration and multi-source data fusion to obtain the location information of the target. The motion state detection module is used to track and predict the motion state of the target based on the location coordinates and the long-distance radio positioning data, and obtain the target's motion trajectory and speed information within a preset time period; The location integration module is used to track and predict the target's motion state based on the location coordinates and the long-range radio positioning data, and obtain the target's motion trajectory and speed information within a preset time period.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images