An indoor multi-radar fusion positioning method and related apparatus

Abstract

The application relates to an indoor multi-radar fusion positioning method and a related device. The method comprises the following steps: acquiring radar target positions detected by multiple radar devices in a current frame; acquiring fusion state information of a human target in a previous frame to predict tracking prediction positions of each human target in the current frame through a Kalman filtering algorithm; for each human target, the tracking prediction position is matched with each radar target position respectively to determine at least one radar target position associated with the human target; the weight coefficient of each associated radar target position is determined to fuse the radar target positions associated with the same human target to obtain a fusion observation position of the human target in the current frame; the fusion observation position is taken as an observation value and input into the Kalman filtering algorithm for updating to determine the fusion positioning information of the human target in the current frame. The method can realize accurate positioning and stable tracking of the human target in an indoor environment in a multi-radar deployment scene.

Description

Technical Field

[0001] This application relates to the field of indoor radar detection technology, and in particular to an indoor multi-radar fusion human body perception method and related device. Background Technology

[0002] In the field of indoor radar detection technology, the use of radar equipment to detect and track people indoors is involved, thereby achieving personnel positioning and tracking in the indoor environment.

[0003] In existing indoor radar detection methods, the prediction and updating of indoor personnel positions typically involves frame-by-frame matching of the radar-detected locations combined with traditional Kalman filtering algorithms. However, the inventors discovered that, on the one hand, single radar devices have limited coverage in practical applications, easily creating detection blind spots; on the other hand, in the collaborative application of multiple radar devices, different radar devices may repeatedly detect the same human target. Consequently, the aforementioned methods are highly prone to problems with poor accuracy and stability in human target localization. Summary of the Invention

[0004] Therefore, it is necessary to provide an indoor multi-radar fusion positioning method, intelligent sensing system, computer equipment, and computer-readable storage medium to address the aforementioned technical problems and solve the issues of poor accuracy and stability in human target positioning.

[0005] Firstly, this application provides an indoor multi-radar fusion positioning method, applied to an intelligent sensing system, the method comprising: The radar target positions detected by multiple radar devices deployed in a preset indoor environment in the current frame are obtained, and the radar target positions are converted to a unified spatial reference frame. The fusion state information of at least one human target determined in the previous frame is obtained, and based on the fusion state information, the tracking prediction position of each human target in the current frame is predicted by the Kalman filter algorithm. For each human target in the current frame, the tracking and prediction position is paired with the radar target position of each radar device to determine at least one radar target position associated with the human target. A weight coefficient is determined for each associated radar target location, and based on the weight coefficient, the radar target locations associated with the same human target are fused to obtain the fused observation location of the human target in the current frame. The weight coefficient is used to characterize the reliability of the radar target location. The fused observation position of the human target in the current frame is used as the observation value and input into the Kalman filter algorithm for updating, so as to determine the fused positioning information of the human target in the current frame. The fused positioning information is used for position tracking and prediction in the next frame.

[0006] Secondly, this application also provides an intelligent sensing system, comprising: The acquisition module is used to acquire the radar target positions detected by multiple radar devices deployed in a preset indoor environment in the current frame, and to convert each radar target position to a unified spatial reference system; The prediction module is used to obtain the fusion state information of at least one human target determined in the previous frame, and based on the fusion state information, predict the tracking prediction position of each human target in the current frame using a Kalman filter algorithm. A pairing module is used to pair the tracking and prediction position with the radar target position of each of the radar devices for each human target in the current frame, and determine at least one radar target position associated with the human target. The fusion module is used to determine the weight coefficient of each associated radar target position, and based on the weight coefficient, fuse the radar target positions associated with the same human target to obtain the fused observation position of the human target in the current frame. The weight coefficient is used to characterize the reliability of the radar target position. The update module is used to take the fused observation position of the human target in the current frame as the observation value and input it into the Kalman filter algorithm for updating, so as to determine the fused positioning information of the human target in the current frame. The fused positioning information is used for position tracking and prediction in the next frame.

[0007] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the above steps.

[0008] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the above steps.

[0009] The aforementioned indoor multi-radar fusion positioning method, intelligent sensing system, computer equipment, and computer-readable storage medium firstly transform the radar target positions detected by multiple radar devices to the same spatial reference frame, thus providing a consistent spatial representation basis for position data from different sources. Secondly, based on the fusion state information of the previous frame and the Kalman filter algorithm, the tracking prediction position of the human target in the current frame is predicted, providing a continuous reference for position association in the current frame. Thirdly, based on the pairing relationship between the tracking prediction position and the positions of each radar target, the observation source corresponding to the same human target is determined. Fourthly, based on the reliability of each associated radar target position, weight coefficients are determined and weighted fusion is performed to form a fused observation position for the same human target. Finally, the fused observation position is updated using the Kalman filter algorithm to obtain fused positioning information that can be used for continued tracking in the next frame. The entire technical solution forms a continuous processing procedure for multi-radar target positions, from unified representation, association determination, weighted fusion, to recursive updating, thereby achieving accurate positioning and stable tracking of human targets in indoor environments under multi-radar deployment scenarios. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a flowchart illustrating an indoor multi-radar fusion positioning method in one embodiment; Figure 2 This is a flowchart illustrating the multi-radar location matching and location fusion process in one embodiment; Figure 3 This is a structural block diagram of an intelligent sensing system in one embodiment; Figure 4 This is an internal structural diagram of a computer device that implements an indoor multi-radar fusion positioning method in one embodiment; Figure 5 This is an internal structural diagram of a computer device that implements an indoor multi-radar fusion positioning method in yet another embodiment; Figure 6 This is an internal structural diagram of a computer-readable storage medium for implementing an indoor multi-radar fusion positioning method in one embodiment. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0013] In one embodiment, such as Figure 1 As shown, an indoor multi-radar fusion positioning method is provided. This embodiment illustrates the method by applying it to a server. It is understood that the method can also be applied to a terminal, or to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method is applied to an intelligent sensing system, which includes the following steps S100 to S500.

[0014] In this context, an intelligent sensing system refers to a system that acquires sensing data through sensing devices and processes that data to form a description of the target's state. In this technical solution, multiple radar devices detect human targets in the same indoor space, acquire the sensing data from these devices, and perform pairing and fusion processing under a unified spatial reference frame to obtain the fused positioning information of the human target. Therefore, this technical solution embodies the process of acquiring, associating, and processing sensing data, and is directly related to the implementation method of the intelligent sensing system at the data processing level.

[0015] Furthermore, intelligent sensing systems can take the form of terminals, servers, or a combination of both. For example, on the terminal side, it can be a security monitoring terminal integrating radar equipment, which directly completes the acquisition, pairing, and fusion processing of multi-source sensor data; on the server side, it can be an edge computing node or a centralized processing server, which performs pairing and fusion processing on the incoming multi-source sensor data; in the case of a combination of both, the terminal completes the data acquisition, and the server completes the pairing and fusion processing, thus forming a complete sensing and processing flow.

[0016] Step S100: Obtain the radar target positions detected by multiple radar devices deployed in a preset indoor environment in the current frame, and convert each radar target position to a unified spatial reference frame.

[0017] For example, multiple radar devices are deployed in a pre-defined indoor environment, where the indoor environment refers to a closed or semi-closed space with clearly defined spatial boundaries, such as an office area, meeting room, or living space. Such environments contain walls, partitions, and fixed facilities, thus constraining the signal propagation path. Within this indoor environment, each radar device is distributed across different locations to detect targets within the environment in the current frame and output the corresponding radar target positions. Since the installation positions and orientations of different radar devices vary, their output radar target positions reside in different coordinate systems. Therefore, based on the actual installation positions of each radar device within the indoor environment, the radar target positions of each device in its respective coordinate system are mapped to a unified spatial reference system through translation and orientation adjustment. This results in a set of radar target positions under a unified spatial reference system, ensuring that the radar detection results from different radar devices have a consistent spatial representation.

[0018] Step S200: Obtain the fusion state information of at least one human target determined in the previous frame, and predict the tracking and prediction position of each human target in the current frame based on the fusion state information using the Kalman filter algorithm.

[0019] For example, the fused state information represents the state description result of the human target in a unified spatial reference frame, specifically including the position, velocity, and acceleration components of the human target in each direction, used to characterize the motion state of the human target at a specific frame time. Based on this, the fused state information corresponding to each human target is used as the state input of the Kalman filter algorithm. According to the state recursion relationship preset by the Kalman filter algorithm, the position, velocity, and acceleration components in each direction of the previous frame are calculated, thereby deriving the motion state of each human target at the current frame time, and thus determining the tracking prediction position of each human target in the current frame; wherein, the human target refers to the person object that needs to be located and tracked in the indoor environment, and the tracking prediction position refers to the predicted spatial position of the human target in the current frame obtained by recursive calculation based on the fused state information of the previous frame.

[0020] Optionally, based on the pre-defined state recursion relationship of the Kalman filter algorithm, the corresponding calculation method can be expressed as follows: within the time interval between two adjacent frames, the position components and corresponding velocity components of the previous frame in each direction are linearly combined according to the time interval to obtain the position components of the current frame, the velocity components and corresponding acceleration components of the previous frame are linearly combined according to the time interval to obtain the velocity components of the current frame, and the acceleration components of the previous frame are passed as the continuation of the current frame, thereby forming the motion state calculation results of the position, velocity and acceleration of the current frame in each direction.

[0021] Step S300: For each human target in the current frame, the tracking and predicted position is paired with the radar target position of each radar device to determine at least one radar target position associated with the human target.

[0022] For example, each tracking prediction position corresponding to a human target is treated as an independent processing object. This tracking prediction position is then paired one-to-one with all radar target positions output by each radar device in the current frame. Here, the tracking prediction position can be understood as the spatial position estimate of a known human target, while the radar target position can be understood as the spatial position detection value of an unknown human target. During the pairing process, each tracking prediction position is sequentially compared with all radar target positions in terms of spatial relationship. By calculating the degree of spatial difference between the two, all radar target positions are filtered, thereby determining a set of radar target positions corresponding to the tracking prediction position from all radar target positions. Through this one-to-one pairing process, each tracking prediction position can select at least one radar target position corresponding to its spatial position from all radar target positions, thus forming a set of radar target positions centered on a single human target.

[0023] For example, in a unified spatial reference frame, the predicted tracking position of a human target is (x1, y1, z1), while the radar target positions detected by multiple radar devices in the current frame include (x2, y2, z2), (x3, y3, z3), and (x4, y4, z4), etc. By calculating the coordinate differences between each of the above radar target positions and the predicted tracking position in three-dimensional space, and comparing the differences, radar target positions with differences within a preset range are selected as candidate positions, thereby obtaining a set of radar target positions that spatially correspond to the predicted tracking position.

[0024] Furthermore, in addition to calculating the coordinate difference between the radar target position and the tracked predicted position in three-dimensional space, it can also calculate the radial velocity difference, signal-to-noise ratio difference, and echo energy difference corresponding to the Doppler frequency shift characteristics. It can also calculate the trajectory direction and displacement difference based on the position changes of adjacent frames. It can also calculate the change difference based on human physiological characteristics such as breathing and heartbeat obtained by radar observation and algorithm detection, thereby filtering the radar target position under a unified multi-dimensional difference metric.

[0025] Furthermore, the above scheme adopts a many-to-one matching method, that is, all tracking and predicted positions are matched with each radar device. In other words, when each radar device detects multiple radar target positions in the same frame, this scheme actually adopts a many-to-many matching method. That is, each tracking and predicted position is taken as the center and analyzed in correspondence with all radar target positions in each radar device. Thus, when traversing all radar devices, at least one radar target position belonging to the same human target is grouped into the same set. In contrast, the many-to-many matching method in conventional technology is to perform a large-scale many-to-many matching of all tracking and predicted positions with all radar target positions of all radar devices. The many-to-many matching method in this scheme uses known human targets and known radar devices as constraints, so that each group of radar target positions associated with the same human target is formed by traversing each human target under the premise of a single radar device. This reduces the complexity of the association relationship and gives each radar target position in the same radar device a clear direction of belonging during the processing.

[0026] Step S400: Determine the weight coefficient of each associated radar target position, and based on the weight coefficient, fuse the radar target positions associated with the same human target to obtain the fused observation position of the human target in the current frame. The weight coefficient is used to characterize the reliability of the radar target position.

[0027] For example, after associating each human target with at least one radar target location, weighting coefficients are determined for each radar target location corresponding to the same human target. During this process, the predicted tracking location of the human target is used as a reference, and each associated radar target location is compared with that predicted tracking location. Three-dimensional spatial difference The system performs heterogeneous calculations to obtain the deviation of each radar target position in each direction. Based on the magnitude of this deviation, the radar target positions are quantified, with radar target positions with smaller deviations corresponding to larger weight coefficients, and vice versa. After obtaining the weight coefficients, the positions of multiple radar targets associated with the same human target are weighted according to their respective weight coefficients. By weighted summing of the components of each position in each direction, the fused observation position of the human target in the current frame is obtained. The fused observation position represents a single position expression formed by integrating the detection results of multiple radars under a unified spatial reference frame, reflecting the spatial position observation result of the human target in the current frame.

[0028] Optionally, the weighting coefficients can also be determined based on the spatial consistency between the radar target positions within the current frame. For example, for a group of radar target positions associated with the same human target, the pairwise distances between each radar target position in three-dimensional space are calculated, and radar target positions that are closer to most positions and located at the spatial cluster center are given higher weights, while positions that deviate significantly from other positions are given lower weights. Alternatively, the weighting coefficients can be determined based on the installation position relationship of the radar equipment, giving higher weights to radar target positions that are more consistent with the tracking and prediction position in spatial direction, while giving lower weights to positions with larger directional deviations.

[0029] In step S500, the fused observation position of the human target in the current frame is taken as the observation value and input into the Kalman filter algorithm for updating, so as to determine the fused positioning information of the human target in the current frame. The fused positioning information is used for position tracking and prediction in the next frame.

[0030] For example, after obtaining the fused observation positions of each human target in the current frame, the fused observation positions are used as the observation values ​​of the current frame and introduced into the Kalman filter algorithm, and combined with the tracking prediction positions obtained in step S200. During this process, based on the difference between the fused observation positions and the tracking prediction positions, the motion state of the human target in the current frame is corrected and calculated to obtain the updated motion state, which includes the position, velocity, and acceleration components of the human target in the current frame. This updated motion state is output as the fused positioning information of the current frame, where the fused positioning information represents a comprehensive expression of the human target's position, velocity, and acceleration in the current frame under a unified spatial reference frame. Furthermore, the fused positioning information is saved as the fused state information of the current frame and used as the input basis in the next frame processing, enabling subsequent frames to perform new tracking prediction position calculations based on the new fused state information. This forms a state recursion relationship between consecutive frames, allowing the motion state of the human target in different frames to be connected through a unified state update process.

[0031] Optionally, the fused positioning information of consecutive frames provides a continuous and stable representation of human target positions under a unified spatial reference system, enabling each human target to form an associative position sequence between different frames. This allows for the counting of targets within a specified time period to achieve personnel counting. Simultaneously, by continuously associating the positions of each frame, a complete spatial movement path is formed, thereby achieving personnel trajectory tracking. Furthermore, based on this, the movement state of personnel is analyzed by combining the direction, speed, and spatial distribution of position changes, thereby achieving personnel behavior analysis.

[0032] In the aforementioned indoor multi-radar fusion positioning method, in step S100, the radar target positions detected by multiple radar devices are uniformly converted to the same spatial reference frame, thereby providing a consistent spatial representation basis for position data from different sources. In step S200, the tracking prediction position of the human target in the current frame is predicted based on the fusion state information of the previous frame and the Kalman filter algorithm, thus providing a continuous reference for position association in the current frame. In step S300, the observation source corresponding to the same human target is determined based on the pairing relationship between the tracking prediction position and the positions of each radar target. In step S400, weight coefficients are determined based on the reliability of each associated radar target position and weighted fusion is performed to form a fused observation position for the same human target. In step S500, the fused observation position is updated according to the Kalman filter algorithm to obtain fused positioning information that can be used for continued tracking in the next frame. The entire technical solution forms a continuous processing flow from unified representation, association determination, weighted fusion to recursive updating of multi-radar target positions, thereby achieving accurate positioning and stable tracking of human targets in indoor environments under multi-radar deployment scenarios.

[0033] In an exemplary embodiment, step S200, "predicting the tracking and prediction position of each human target in the current frame based on the fused state information using the Kalman filter algorithm", includes step S201.

[0034] Step S201: The position component, velocity component and acceleration component of the human target in the fused state information of the previous frame are used as the state vector of the Kalman filter algorithm, and the tracking prediction position of the human target in the current frame is predicted according to the preset motion model; wherein, the motion model represents the state transition model based on the motion state enhancement expression of Doppler frequency shift features.

[0035] For example, the fused state information of the human target in the previous frame is obtained, and the position, velocity, and acceleration components of the human target in each direction under a unified spatial reference frame are extracted from it. These components are combined in a predetermined order to form a state vector, which is used to represent the complete motion state of the human target in the previous frame. The state vector is used as the input basis of the Kalman filter algorithm, and the state vector is recursively calculated according to a preset motion model. The motion model represents the introduction of Doppler frequency shift information reflected by radar signals on the basis of the conventional recursive relationship of position, velocity, and acceleration, to enhance the expression of the changes of the human target in the direction of motion, so that the state recursion process can reflect the motion trend of the human target in space.

[0036] Specifically, during the calculation process, the position components and corresponding velocity components of the previous frame in each direction are first superimposed according to the time interval between two adjacent frames, and the increment generated by the acceleration component at time intervals is also superimposed to obtain the position components of the current frame in each direction. Then, the velocity components and acceleration components of the previous frame are linearly combined at time intervals to obtain the velocity components of the current frame in each direction, while the acceleration components in each direction are transmitted as the continuation of the current frame.

[0037] Based on this, the motion information corresponding to the Doppler frequency shift characteristics obtained from the radar signal is mapped to a component consistent with the target's motion direction. The velocity component in the corresponding direction and the resulting position increment are corrected to ensure that the change in that direction reflects the target's actual motion trend, while the components in other directions maintain their original recursive relationship. Through the above step-by-step calculations, the motion state results of the current frame in each direction are formed, including position, velocity, and acceleration. The predicted tracking position of the human target in the current frame is then extracted from the position component.

[0038] Optionally, the radial motion information of the human target along the radar observation direction can be obtained based on the Doppler frequency shift characteristics, which reflects the approach or departure trend of the human target in the radial direction and its velocity magnitude. On this basis, the radial motion information is mapped to the component consistent with its direction, and the velocity component in the radial direction is incrementally corrected accordingly to make its value consistent with the radial motion information. At the same time, the position increment caused by the velocity component is proportionally adjusted so that the position change in the radial direction can reflect the actual displacement change of the human target in the radial direction, while the original recursive calculation results remain unchanged for other directional components that are inconsistent with the radial direction.

[0039] In this embodiment, in step S201, on the one hand, the position component, velocity component, and acceleration component in the fused state information of the previous frame are used as the state vector of the Kalman filter algorithm, so that the motion state of the human target in the previous frame can be used as the basis for the recursive calculation of the prediction of the current frame; on the other hand, the motion information corresponding to the Doppler frequency shift feature is fused in the state recursion process of the Kalman filter algorithm according to the preset motion model, so as to realize the enhanced expression of motion state in the recursive calculation process of the Kalman filter algorithm; based on this, in the whole technical solution, by simultaneously using the state vector and the Doppler frequency shift feature in the Kalman filter algorithm, a tracking prediction position that takes into account both the existing motion state and the motion trend is obtained, providing a more targeted prediction basis for subsequent processing.

[0040] In an exemplary embodiment, step S300, "for each human target in the current frame, the tracking and predicted position is paired with the radar target position of each radar device to determine at least one radar target position associated with the human target", includes steps S301 to S303.

[0041] Step S301: Calculate the feature difference between the tracking and predicted position of each human target and the position of each radar target in the same radar device, and construct the cost matrix between each human target and the same radar device based on each feature difference.

[0042] For example, taking each human target as the processing object, the feature difference between its corresponding tracking and predicted position and the position of each radar target in the current frame of the same radar device is calculated. The feature difference may include a distance value, which represents the three-dimensional spatial difference between the two targets in a unified spatial reference frame, characterizing their spatial proximity. Further, the feature difference may also include signal-to-noise ratio difference, echo energy difference, and the degree of difference in features such as Doppler frequency shift, trajectory observation features, and human physiological features. Essentially, various feature differences can be normalized in a unified order and represented as vectors. For example, the distance difference, signal-to-noise ratio difference, echo energy difference, Doppler frequency shift difference, trajectory observation feature difference, and human physiological feature difference are respectively dimensionally unified and numerically scaled to make them comparable, and arranged in a preset order to form a set of feature vectors. Each dimension corresponds to a type of difference, thus mapping multi-dimensional feature differences into a vector form for subsequent unified calculation and comparison.

[0043] Among them, trajectory observation features represent the changes in the position of people between adjacent frames predicted by the Kalman filter algorithm or detected by radar equipment, while human physiological features represent the human physiological activities predicted by the Kalman filter algorithm or detected by radar equipment.

[0044] For example, in radar equipment detection, the radar equipment obtains radial velocity information by frequency changes in the echo signal, and combines it with time changes to obtain radial position and acceleration, thereby forming Doppler frequency shift features. At the same time, it obtains trajectory direction and trajectory contour by detecting position changes in adjacent frames, thereby forming trajectory observation features. Furthermore, it extracts breathing and heartbeat information by periodic micro-movements in the echo signal, thereby forming corresponding human physiological characteristics.

[0045] For example, in terms of Kalman filter algorithm prediction, when the Kalman filter algorithm performs recursive calculations on state components such as position, velocity and acceleration, it can extract the position, velocity and acceleration in the radial direction from each directional component, thereby forming Doppler frequency shift features. At the same time, it determines the trajectory direction and trajectory contour by the change relationship of position components in adjacent frames, thereby forming trajectory observation features. Furthermore, it forms human physiological characteristics by reflecting breathing and heartbeat information through subtle periodic changes in state components.

[0046] Specifically, when calculating the degree of difference in features such as Doppler frequency shift characteristics, trajectory observation characteristics, and human physiological characteristics: On the one hand, for Doppler frequency shift characteristics, the radial motion information corresponding to each human target in the current frame is compared with the radial motion information reflected by the radar target position. The difference in Doppler frequency shift characteristics is obtained by calculating the difference in radial direction consistency and velocity change amplitude between the two. On the other hand, for trajectory observation characteristics, the trajectory changes formed by each human target between adjacent frames are compared with the trajectory changes of the radar target position in adjacent frames. The difference in trajectory observation characteristics is obtained by calculating the trajectory direction deviation and trajectory contour difference. Furthermore, for human physiological characteristics, the respiratory rhythm and heart rate changes exhibited by each human target in the current state are compared with the respiratory rhythm and heart rate changes corresponding to the radar target position. The difference in human physiological characteristics is obtained by calculating the differences in their intensity and periodicity.

[0047] After calculating the feature differences, the feature differences between each human target and the location of each radar target in the same radar device are organized according to the correspondence. Thus, a cost matrix is ​​constructed based on the feature differences between each human target and the location of all radar targets in the radar device, which involves the overall mapping relationship between the location of all radar targets in the radar device and all human targets. Each element in the cost matrix represents the magnitude of the feature difference between a certain human target in the entire set of human targets and a certain radar target location in the radar device.

[0048] Step S302: In each radar device, for a cost matrix constructed by the same radar device based on each human target, the Hungarian algorithm is used to solve the cost matrix until the solution of all cost matrices is completed, and at least one radar target position with a distance value less than a preset threshold is assigned to each human target.

[0049] For example, a cost matrix constructed based on each human target for the same radar device is used as input to the Hungarian algorithm for solution. During the solution process, the correspondence between each human target and the radar target position in the same radar device is optimized based on the overall distribution of the feature differences in the cost matrix, so that the combination with smaller feature differences is preferentially determined in the global scope. Specifically, for each human target, the distance value between it and all radar target positions is read from the cost matrix, and each distance value is compared with a preset threshold. Only radar target positions with distance values ​​less than the preset threshold are retained as candidates for allocation in the Hungarian algorithm. The preset threshold is set based on the spatial variation range of the human target between adjacent frames. That is, based on the time interval between the current frame and the previous frame and the maximum movement range of the human target within the time interval, the allowable spatial deviation is limited, thereby ensuring that the radar target positions participating in the matching maintain a reasonable spatial proximity relationship with the corresponding human target.

[0050] Based on this, the Hungarian algorithm is used to solve the filtered cost matrix. This allows for the allocation of suitable human targets to each radar target location within the overall mapping relationship reflected in the cost matrix, until all cost matrices are solved. This ensures that each human target receives at least one radar target location with a distance that meets the requirements, thus transforming the overall mapping relationship in the cost matrix into a specific allocation result. For example, given human targets A and B, radar target locations R1 and R2 in the first radar device, and radar target locations R3 and R4 in the second radar device, a cost matrix is ​​constructed for the first radar device based on the feature differences between A / B and R1 / R2. Similarly, a cost matrix is ​​constructed for the second radar device based on the feature differences between A / B and R3 / R4. Each element in the cost matrix represents the feature difference between a specific human target and a specific radar target location within the specified radar device. After distance threshold filtering, feature differences that meet the distance threshold requirements are filled into the cost matrix to sparsify it and improve computational and storage efficiency. The cost matrix after filtering is used as input for solving the problem. During the solution process, the cost matrix is ​​gradually adjusted based on rows and columns to retain candidate positions that can represent small feature differences in each row and column, and gradually eliminates allocation relationships with large feature differences. Thus, among all the available allocation relationships, the allocation relationship that minimizes the sum of feature differences between each human target and the assigned radar target position is selected. Under the one-to-many allocation constraint between radar equipment and human targets, for the first radar equipment, R1 is assigned to A and R2 is assigned to B, and for the second radar equipment, R3 is assigned to A and R4 is assigned to B, thus forming the allocation result with the minimum overall cost.

[0051] Furthermore, when the distance between two human targets is less than a preset threshold, and their respective associated radar target positions originate from different radar devices, it is determined that the same human target is repeatedly detected by different radar devices due to differences in the position of the reflection point. Based on this, by merging the above multiple human targets into a single human target, the multiple detection caused by differences in the reflection position of the same human target by different radar devices is eliminated.

[0052] In this embodiment, in step S301, the feature difference between the predicted tracking position of the human target and the position of each radar target is calculated, and a cost matrix is ​​constructed based on each feature difference, thereby transforming the feature proximity relationship between the human target and the radar target position into a uniformly processed correspondence data; in step S302, the cost matrix is ​​solved using the Hungarian algorithm until all cost matrices are solved, thereby assigning radar target positions with distance values ​​less than a preset threshold to each human target, and obtaining an allocation result based on spatial distance constraints; based on this, in the entire technical solution, the processing flow from feature difference modeling to overall solution and allocation of the human target and radar target positions is realized, thereby improving the clarity of the current frame association result.

[0053] In an exemplary embodiment, before "constructing the cost matrix between each human target and the same radar device based on the feature differences" in step S301, the method further includes steps S304 to S305.

[0054] Step S304: Using the tracking and prediction position of each human target as the center and a preset threshold as the radius, define an effective association area for the tracking and prediction position of each human target.

[0055] Step S305: Only the locations of radar targets falling within the effective association area are used to construct the cost matrix corresponding to the corresponding human targets.

[0056] For example, for each human target, its corresponding tracking and prediction position is used as the center point, and a preset threshold is used as the radius to define a three-dimensional spatial range under a unified spatial reference system, thereby forming a corresponding effective association region for each human target. After the effective association region is determined, the positions of all radar targets detected by each radar device in the current frame are judged one by one. The spatial position relationship between each radar target position and the effective association region corresponding to each human target is compared. By calculating the distance between the radar target position and the corresponding tracking and prediction position, it is determined whether the radar target position falls within any effective association region. For radar target positions that fall within the region, they are used as candidate positions to participate in the matching calculation of the corresponding human target and are used to construct the corresponding cost matrix in the future. For radar target positions that do not fall within the region, they do not participate in the matching calculation of the corresponding human target.

[0057] For example, suppose the number of human targets is M and the number of radar target locations is N. Without area restrictions, all pairing relationships need to be calculated, resulting in a computational load of M×N, increasing quadratically. However, after introducing an effective association area, suppose the number of candidate radar target locations corresponding to each human target is k (k If the total computational cost is N, then the overall computational cost is transformed into M×k, thereby reducing the computational scale that originally depended on the global number N to a linear scale that depends on the local number of candidates k; in this way, the data scale involved in the construction and solution of the cost matrix is ​​significantly reduced.

[0058] Optionally, the effective association region can also be set based on data scale constraints. That is, under the premise of meeting the matching requirements, the calculation range can be limited by controlling the number of radar target positions participating in the calculation. For example, the maximum number of radar target positions participating in the matching calculation for each human target can be preset to k, and all radar target positions can be sorted in ascending order of distance from the tracking and prediction position. Only the first k radar target positions with the smallest distance are selected as candidate data in the effective association region, thereby replacing the simple spatial radius constraint with a quantity constraint. Furthermore, the upper limit of this quantity can be dynamically adjusted according to the total number of radar target positions in the current frame, so that the number of candidates is tightened when the number of targets is large and appropriately relaxed when the number of targets is small. This ensures matching coverage while keeping the data scale participating in the subsequent cost matrix construction and calculation within a manageable range.

[0059] In this embodiment, in step S304, an effective association region is determined based on the tracking and prediction position of the human target as the center and a preset threshold as the radius, thereby limiting the spatial range of the radar target positions participating in the matching; in step S305, only the radar target positions falling within the effective association region are used to construct the cost matrix corresponding to the corresponding human target, so that the construction and solution of the cost matrix are based only on candidate data within a local range; based on this, in the entire technical solution, by introducing a data filtering process based on spatial range limitation before matching, the scale of data involved in the calculation is reduced and a matching processing structure centered on the human target is formed.

[0060] In an exemplary embodiment, the reliability of the radar target position is characterized by the covariance matrix. The step S400, "based on the weight coefficient, fuse the radar target positions associated with the same human target to obtain the fused observation position of the human target in the current frame", includes steps S401 to S403.

[0061] Step S401: Based on the motion state of each human target in the indoor environment and the positional distribution characteristics of the radar target positions associated with the same human target, determine the covariance matrix corresponding to the radar target positions associated with the same human target.

[0062] For example, the motion state of a human target in an indoor environment includes position components, velocity components, and acceleration components; the positional distribution characteristics of the radar target positions associated with the human target represent the degree of aggregation or dispersion of each radar target position in three-dimensional space, for example, multiple positions are close to each other in space and concentrated in a certain area, or they are stretched in one direction and relatively concentrated in other directions. Based on this, the motion state and position distribution characteristics of the human target are quantified. In the specific process, the position change trend and corresponding change amplitude of the human target in each direction are determined according to the position component, velocity component and acceleration component in each direction, and a first evaluation value reflecting the degree of motion activity is formed accordingly. At the same time, according to the degree of clustering or dispersion of each radar target position in three-dimensional space, the distribution range in each direction is statistically analyzed to form a second evaluation value reflecting the degree of spatial distribution dispersion. Subsequently, the first evaluation value and the second evaluation value are combined in each direction. By weighting or superimposing the two types of evaluation values, the uncertainty values ​​in each direction are obtained, and these uncertainty values ​​are filled into the corresponding positions of the matrix to form a covariance matrix corresponding to the position of each radar target, so as to represent the uncertainty distribution of a radar target position in each direction.

[0063] For example, if a human target has large velocity and acceleration components in the x-direction but small changes in the y and z-directions in the current frame, a first evaluation value corresponding to a larger value in the x-direction and a smaller value in the y and z-directions can be obtained based on the position, velocity, and acceleration components. Simultaneously, if the radar target positions associated with this human target are concentrated in the x-direction, discrete in the y-direction, and moderately distributed in the z-direction, a second evaluation value corresponding to a larger value in the y-direction, a smaller value in the x-direction, and a moderate value in the z-direction can be obtained based on their degree of clustering or dispersion in three-dimensional space. Furthermore, the first and second evaluation values ​​in each direction are combined and calculated accordingly, for example, by weighted superposition, to obtain the two types of evaluation values ​​in each direction, thus forming the corresponding covariance matrix.

[0064] Optionally, the weighted coefficients mentioned above are used to balance the influence of motion state and position distribution characteristics on uncertainty. Their values ​​are set according to the stability and reliability of the two types of information. For example, when the velocity and acceleration components of the human target change relatively stably, the weight of the first evaluation value can be increased. When the radar target position is relatively concentrated and consistent in space, the weight of the second evaluation value can be increased, thereby achieving a reasonable combination of the two types of evaluation values ​​in different scenarios.

[0065] Step S402: Determine the weighting coefficients corresponding to the radar target position based on the covariance matrix corresponding to the radar target position.

[0066] For example, referring to equation (1), the covariance matrix D corresponding to a certain radar target position is inverted to obtain the corresponding inverse matrix, which is used to reflect the inverse vector degree relationship of uncertainty in each direction. That is, the smaller the uncertainty, the larger the corresponding value in the inverse matrix. Then, the determinant of the inverse matrix is ​​calculated to obtain a scalar value, which represents the overall measurement result under the combined effect of uncertainty in each direction. Based on this, the reciprocal of the square root of the scalar value is calculated to obtain the weight coefficient corresponding to the radar target position. .

[0067] (1) It can be seen that in equation (1), the uncertainties in each direction, which were originally represented in matrix form, are compressed into a single weight coefficient, so that the positions of each radar target can participate in the subsequent fusion calculation in a unified scalar form, and the radar target positions with smaller uncertainties correspond to larger weight coefficients, thereby completing the conversion process from covariance matrix to weight coefficient.

[0068] Step S403: Based on the weighting coefficients corresponding to the radar target positions, fuse the radar target positions associated with the same human target to obtain the fused observation position of the human target in the current frame.

[0069] For example, referring to equation (2), there are n radar target locations associated with a certain human target, where the k-th radar target location... Corresponding to weighting coefficients During the calculation, the position of each radar target is multiplied by its corresponding weighting coefficient, and all product results are accumulated. Simultaneously, the weighting coefficients are summed, and the sum of these weighting coefficients is used to normalize the accumulated result, yielding the fused coordinates. That is, the fusion observation position corresponding to the human target.

[0070] (2) It can be seen that in Equation (2), the radar target position with smaller uncertainty and larger weight coefficient occupies a higher proportion in the fusion process, while the radar target position with larger uncertainty and smaller weight coefficient has a correspondingly reduced impact on the final result. Finally, the fused observation position under the unified spatial reference system is obtained. This position is used as a single position expression of the human target in the current frame to reflect the spatial position observation result of the human target in the current frame.

[0071] In this embodiment, in step S401, the covariance matrix corresponding to each radar target position is jointly determined based on the motion state of the human target and the positional distribution characteristics of the radar target position, thereby quantifying the uncertainty in multiple directions in matrix form; in step S402, the covariance matrix is ​​converted into weight coefficients, thereby compressing the uncertainty in multiple directions into scalar weights that can be used for fusion calculation; in step S403, the positions of each radar target are weighted and fused according to the weight coefficients corresponding to each radar target position, thereby obtaining a single fused observation position corresponding to the human target; based on this, in the entire technical solution, a continuous processing process from uncertainty modeling, weight mapping to weighted fusion is realized, enabling the fusion expression of multi-source radar target positions on a unified metric.

[0072] In an exemplary embodiment, the reliability of the radar target position is characterized by the covariance matrix. The step S400, "based on the weight coefficient, fuse the radar target positions associated with the same human target to obtain the fused observation position of the human target in the current frame", includes steps S404 to S406.

[0073] Step S404: Based on the motion state of each human target in the indoor environment and the positional distribution characteristics of the radar target positions associated with the same human target, determine the covariance matrix corresponding to the radar target positions associated with the same human target.

[0074] Step S405: Based on the covariance matrix corresponding to the radar target position, determine the weight coefficient corresponding to the radar target position, and construct a probability distribution model corresponding to the radar target position associated with the same human target.

[0075] For example, after obtaining the covariance matrix corresponding to the radar target position associated with the same human target according to the same implementation process as step S401, and obtaining the weight coefficients corresponding to the radar target positions according to the same implementation process as step S402, the coordinates of each radar target position in a unified spatial reference system are used as reference points in space, and the corresponding weight coefficients are mapped to the probability contribution of the corresponding reference points in space. This ensures that radar target positions with larger weight coefficients correspond to higher probability values ​​in their neighborhoods, while radar target positions with smaller weight coefficients correspond to lower probability values ​​in their neighborhoods. Then, taking each radar target position as the center, its weight coefficients are extended in space, so that the probability value of any spatial point is determined by its distance relationship with each radar target position and its corresponding weight, thereby forming multiple probability distribution expressions centered on the radar target positions.

[0076] For example, if a radar target location has a weighting coefficient of 0.6, then within the area centered on that radar target location, spatial points near that location will be assigned a higher probability value. As the distance between the spatial point and the radar target location increases, the probability value gradually decreases with distance, resulting in a probability distribution of the radar target location that continuously changes from high to low within its neighborhood. For another radar target location with a weighting coefficient of 0.3, the overall probability within its neighborhood is lower than that of the former, but it also shows a trend of gradually decreasing with increasing distance.

[0077] Based on this, the above probability distributions are superimposed in the entire continuous space to obtain the probability distribution model corresponding to the same human target. The probability distribution model represents the probability distribution of each position in the space as the true position of the human target. For example, in the region where multiple radar target positions with higher weight coefficients are close to each other, the corresponding probability value is higher, while the probability value in the region far away from these positions gradually decreases, thus completing the process of constructing the probability distribution model based on weight coefficients.

[0078] Step S406: Based on the probability distribution of the probability distribution model in the continuous space, evaluate the joint probability of each spatial point in the continuous space, and take the spatial point with the highest joint probability as the fusion observation position of the corresponding human target in the current frame.

[0079] For example, for any spatial point, the probability value corresponding to that spatial point is directly obtained from the probability distribution model, where the probability value represents the likelihood of that spatial point being the true location of a human target. Since the probability distribution model is constructed by multiple radar target locations and their weight coefficients, the probability value of that spatial point comprehensively reflects the influence relationship of each radar target location on that spatial point. Based on this, the probability value corresponding to that spatial point is taken as its joint probability to represent the comprehensive likelihood of that spatial point being the true location of a human target under the combined effect of multiple source radar target locations.

[0080] For example, at a certain spatial point P, the probability distribution model has given a joint probability of 0.82. This value is formed by the combined effect of the positions of three radar targets, with corresponding weights of 0.5, 0.3 and 0.2, respectively. In the process of model construction, the contribution of each radar target position to the spatial point has been attenuated according to the distance relationship from each radar target position to the spatial point and weighted and superimposed according to the corresponding weight coefficients, so as to obtain the final probability value of the spatial point.

[0081] The joint probabilities corresponding to each spatial point in the continuous space are compared, and the spatial point with the highest joint probability is selected as the fused observation position of the corresponding human target in the current frame. Furthermore, the fused observation position is not limited to existing radar target positions, but is the optimal spatial point obtained through joint probability evaluation in the continuous space. When multiple radar target positions are highly consistent in space, the optimal spatial point may coincide with the position of one of the radar targets. However, in general, its position is determined by the overall probability distribution and is the result of integrating multi-source information.

[0082] In this embodiment, in step S404, the covariance matrix corresponding to each radar target position is jointly determined based on the motion state of the human target and the positional distribution characteristics of the radar target position, thereby quantifying the uncertainty in multiple directions in matrix form; in step S405, the weight coefficients are determined based on the covariance matrix and a probability distribution model is constructed, thereby uniformly mapping the reliability of each radar target position and its spatial influence relationship to a probability distribution in continuous space; in step S406, the joint probability assessment of each spatial point in continuous space is performed based on the probability distribution model, and the position corresponding to the maximum value is selected, thereby obtaining the fused observation position; based on this, in the entire technical solution, a continuous processing process from uncertainty modeling, weight mapping to probability assessment is realized, so that the fusion result is determined after multi-source information is uniformly expressed in space.

[0083] In an exemplary embodiment, the method further includes steps S600 to S800.

[0084] Step S600: If there are abnormal tracking prediction positions in each tracking prediction position of the current frame that are not matched with any radar target position, then according to the Kalman filter algorithm, predict the target tracking prediction position of the human target to which the abnormal tracking prediction position belongs in the next frame, and match the target tracking prediction position of the next frame with the radar target position obtained in the next frame.

[0085] For example, after completing the pairing process between the human target and the radar target position in the current frame, the tracking prediction position corresponding to each human target is checked one by one. When there is a tracking prediction position that has not established a pairing relationship with any radar target position, it is marked as an abnormal tracking prediction position. The human target to which the abnormal tracking prediction position belongs is taken as the processing object, and the motion state of the human target in the current frame is recursively calculated based on the Kalman filter algorithm. Specifically, the position component, velocity component, and acceleration component of the human target in the current frame are used as inputs, and the state recursion calculation is performed according to the time interval between adjacent frames to obtain the target tracking prediction position of the human target in the next frame. Based on this, after obtaining the radar target position in the next frame, the target tracking prediction position and the radar target position in the next frame are paired according to the spatial position relationship to determine whether the human target is re-paired with the radar target position in subsequent frames, so that unpaired human targets can continue to participate in the pairing process of subsequent frames through state recursion.

[0086] Step S700: If there is a radar target position that is paired with the target tracking prediction position in the next frame, then the corresponding human target's fused observation position in the next frame is determined based on the paired radar target position.

[0087] For example, if there is a radar target position that is paired with the target tracking prediction position in the next frame, the paired radar target positions are fused according to the corresponding weight coefficients to obtain the fused observation position of the human target in the next frame; then the fused observation position of the human target in the next frame is used as the observation value and input into the Kalman filter algorithm for updating, so as to determine the fused positioning information of the human target in the next frame.

[0088] In step S800, if there is no radar target position that matches the target tracking prediction position in the next frame, the target tracking prediction position in the next frame is iteratively predicted again according to the Kalman filter algorithm until the radar target position of the corresponding frame is matched in subsequent frames. If the radar target position is not matched within a preset number of frames, the abnormal tracking prediction position and the positions obtained by iterative prediction of the abnormal tracking prediction position in subsequent frames are discarded.

[0089] For example, if no radar target position matches the target tracking prediction position in the next frame, the state recursion calculation for the human target continues based on the Kalman algorithm. This involves using the position, velocity, and acceleration components of the human target reflected in the target tracking prediction position in the next frame as input, and performing state recursion calculation based on the time interval between adjacent frames to obtain the target tracking prediction position of the human target in the next-next frame. Based on this, after obtaining the radar target position in the next-next frame, the target tracking prediction position and the radar target position in the next-next frame are paired according to their spatial relationship until a successful pairing is achieved. Furthermore, the number of iteration frames is counted during the recursion process. If no radar target position corresponding to the human target is obtained within a preset number of frames, the abnormal tracking prediction position is determined to no longer have effective observation support. Therefore, the abnormal tracking prediction position and all positions recursively obtained in subsequent frames are discarded, thus ending the tracking process for the human target.

[0090] Optionally, the preset frame range is set based on the continuity of human target movement in an indoor environment and the intermittent nature of radar detection. That is, even if the human target is briefly obscured, the signal attenuates, or detection is temporarily lost, it still maintains a predictable trajectory within several frames. Therefore, a certain number of frames needs to be reserved for recursive continuation. Simultaneously, the upper limit of this frame range must be less than the time range within which the human target's state remains stable under no observation constraints, to avoid the continuous accumulation of recursive errors leading to positional deviation. Therefore, by balancing the "recoverable observation time window" and the "controllable range of recursive errors," the preset frame range is determined, ensuring coverage of short-term undetected situations while also allowing timely termination of tracking in cases of prolonged lack of matching.

[0091] In this embodiment, in step S600, the abnormal tracking prediction position that is not paired with any radar target position in the current frame continues to be recursively processed, and the target tracking prediction position in the next frame is re-paired with the radar target position in the next frame, so that the human target that is mismatched in the current frame can still enter the association processing of subsequent frames; in step S700, if the target tracking prediction position is successfully paired, the fusion observation position of the corresponding human target in the next frame is determined, so that the human target that has been restored to pairing can obtain the observation result again; in step S800, if the target tracking prediction position pairing fails, iterative prediction continues, and if it is still not successfully paired within a preset number of frames, the corresponding position is discarded, so as to terminate the abnormal prediction result that has no observation support; based on this, in the whole technical solution, a continuous processing process of unpaired human targets from continued prediction, restoration and update to timeout removal is realized.

[0092] In one exemplary embodiment, Figure 2The flowchart illustrates the multi-radar position matching and fusion process. Specifically: First, during position matching, all radar target positions from the first radar device in the list are loaded. Then, the relative coordinates of all radar target positions from that device are converted to absolute coordinates to align different radar data in a unified spatial reference system. Next, all radar target positions from that device are paired with the tracking prediction position of each known human target, i.e., a cost matrix is ​​constructed based on the radar target positions from that device and the tracking prediction positions of all human targets. After pairing the radar target positions of the current radar device, the process continues by loading all radar target positions from the next radar device in the list and repeating the above process. Furthermore, it is determined whether all radar devices in the list have been loaded. If not, the loading and pairing process continues. If loaded, the Hungarian algorithm is used to globally optimize each matched radar target position and tracking prediction position within the same radar device, i.e., the cost matrix corresponding to that radar device is solved, until the cost matrices corresponding to all radar devices are solved sequentially. This ensures that each tracking prediction position corresponds to at least one radar target position, and each radar target position does not repeatedly correspond to different tracking prediction positions.

[0093] After completing the location matching process, the location fusion process begins: First, the first tracking prediction location in the list is loaded. Then, the radar target locations associated with this tracking prediction location are loaded, and the association status is judged. If no associated radar target location exists within a preset number of frames, the tracking prediction location is discarded; if an associated radar target location exists, subsequent processing continues. Next, the weight coefficients of the associated radar target locations are determined, and fusion calculations are performed on the associated radar target locations based on the weight coefficients to obtain the fused observation location. This fused observation location is then used as the observation value and input into the Kalman algorithm for updating to obtain the fused localization information of the corresponding human target. Finally, it is judged whether all tracking prediction locations in the list have been loaded. If not, the next tracking prediction location is loaded and the above process is repeated; if all locations have been loaded, it is further judged whether there are duplicate detections of nearby human targets, and duplicate detections of human targets are merged to finally obtain the multi-radar fusion result.

[0094] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0095] Based on the same inventive concept, this application also provides an intelligent sensing system for implementing the aforementioned indoor multi-radar fusion positioning method. The solution provided by this intelligent sensing system is similar to the implementation described in the above method; therefore, the specific limitations in one or more intelligent sensing system embodiments provided below can be found in the limitations of the indoor multi-radar fusion positioning method described above, and will not be repeated here.

[0096] In one exemplary embodiment, such as Figure 3 As shown, an intelligent sensing system is provided, including: an acquisition module 301, a prediction module 302, a pairing module 303, a fusion module 304, and an update module 305, wherein: The acquisition module 301 is used to acquire the radar target positions detected by multiple radar devices deployed in a preset indoor environment in the current frame, and convert each radar target position to a unified spatial reference system; The prediction module 302 is used to obtain the fusion state information of at least one human target determined in the previous frame, and based on the fusion state information, predict the tracking and prediction position of each human target in the current frame using the Kalman filter algorithm. The pairing module 303 is used to pair the tracking and predicted position with the radar target position of each radar device for each human target in the current frame, and determine at least one radar target position associated with the human target. The fusion module 304 is used to determine the weight coefficient of each associated radar target position, and based on the weight coefficient, fuse the radar target positions associated with the same human target to obtain the fused observation position of the human target in the current frame. The weight coefficient is used to characterize the reliability of the radar target position. The update module 305 is used to take the fused observation position of the human target in the current frame as the observation value and input it into the Kalman filter algorithm for updating, so as to determine the fused positioning information of the human target in the current frame. The fused positioning information is used for position tracking prediction in the next frame.

[0097] In an exemplary embodiment, the prediction module 302 is further configured to: use the position component, velocity component and acceleration component of the human target in the fused state information of the previous frame as the state vector of the Kalman filter algorithm, and predict the tracking prediction position of the human target in the current frame according to the preset motion model; wherein, the motion model represents the state transition model based on the motion state enhancement expression of Doppler frequency shift features.

[0098] In an exemplary embodiment, the pairing module 303 is further configured to: calculate the feature difference between the tracking and predicted position of each human target and the position of each radar target in the same radar device, and construct a cost matrix between each human target and the same radar device based on each feature difference; in each radar device, for a cost matrix constructed by the same radar device based on each human target, use the Hungarian algorithm to solve a cost matrix until the solution of all cost matrices is completed, and assign at least one radar target position with a distance value less than a preset threshold to each human target.

[0099] In an exemplary embodiment, the pairing module 303 is further configured to: define an effective association region for the tracking prediction position of each human target, with the tracking prediction position of each human target as the center and a preset threshold as the radius; and use only the radar target positions that fall within the effective association region to construct a cost matrix corresponding to the corresponding human target.

[0100] In an exemplary embodiment, the fusion module 304 is further configured to: determine the covariance matrix corresponding to the radar target position associated with the same human target based on the motion state of each human target in the indoor environment and the positional distribution characteristics of the radar target positions associated with the same human target; determine the weight coefficient corresponding to the radar target position based on the covariance matrix corresponding to the radar target position; and fuse the radar target positions associated with the same human target based on the weight coefficient corresponding to the radar target position to obtain the fused observation position of the human target in the current frame.

[0101] In an exemplary embodiment, the fusion module 304 is further configured to: determine the covariance matrix corresponding to the radar target position associated with the same human target based on the motion state of each human target in the indoor environment and the positional distribution characteristics of the radar target positions associated with the same human target; determine the weight coefficients corresponding to the radar target positions based on the covariance matrix corresponding to the radar target positions; and construct a probability distribution model corresponding to the radar target positions associated with the same human target; evaluate the joint probability of each spatial point in the continuous space based on the probability distribution model in the continuous space; and take the spatial point with the highest joint probability as the fusion observation position of the corresponding human target in the current frame.

[0102] In an exemplary embodiment, the intelligent sensing system further includes an anomaly handling module, which is configured to: if there is an abnormal tracking prediction position in each tracking prediction position of the current frame that is not paired with any radar target position, then predict the target tracking prediction position of the human target to which the abnormal tracking prediction position belongs in the next frame according to the Kalman filter algorithm, and pair the target tracking prediction position of the next frame with the radar target position obtained in the next frame; if there is a radar target position that is paired with the target tracking prediction position of the next frame, then determine the corresponding fused observation position of the human target in the next frame according to the paired radar target position; if there is no radar target position that is paired with the target tracking prediction position of the next frame, then iteratively predict the target tracking prediction position of the next frame again according to the Kalman filter algorithm until the radar target position of the corresponding frame is paired in subsequent frames; if the radar target position is still not paired within a preset number of frames, then discard the abnormal tracking prediction position and the positions obtained by iterative prediction of the abnormal tracking prediction position in subsequent frames.

[0103] Each module in the aforementioned intelligent sensing system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0104] In one exemplary embodiment, a computer device is provided, the computer device including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps in the above-described method embodiments.

[0105] This computer device can be a server, and its internal structure diagram can be as follows: Figure 4As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs in the non-volatile storage media to run. The database stores data related to the aforementioned indoor multi-radar fusion positioning method. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network.

[0106] This computer device can be a terminal, and its internal structure diagram can be as follows: Figure 5 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, and a display unit. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface and display unit are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements the steps in the aforementioned indoor multi-radar fusion positioning method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display (LCD) or an e-ink display.

[0107] Those skilled in the art will understand that Figure 4 or Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0108] In one exemplary embodiment, such as Figure 6The diagram shows the internal structure of a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the above-described method embodiments.

[0109] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods.

[0110] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0111] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A positioning method using indoor multi-radar fusion, characterized in that, The method, applied to intelligent sensing systems, includes: The radar target positions detected by multiple radar devices deployed in a preset indoor environment in the current frame are obtained, and the radar target positions are converted to a unified spatial reference frame. The fusion state information of at least one human target determined in the previous frame is obtained, and based on the fusion state information, the tracking prediction position of each human target in the current frame is predicted by the Kalman filter algorithm. For each human target in the current frame, the tracking and prediction position is paired with the radar target position of each radar device to determine at least one radar target position associated with the human target. A weight coefficient is determined for each associated radar target location, and based on the weight coefficient, the radar target locations associated with the same human target are fused to obtain the fused observation location of the human target in the current frame. The weight coefficient is used to characterize the reliability of the radar target location. The fused observation position of the human target in the current frame is used as the observation value and input into the Kalman filter algorithm for updating, so as to determine the fused positioning information of the human target in the current frame. The fused positioning information is used for position tracking and prediction in the next frame.

2. The method according to claim 1, characterized in that, The step of predicting the tracking and prediction position of each human target in the current frame using a Kalman filter algorithm based on the fused state information includes: The position, velocity, and acceleration components of the human target in the fused state information of the previous frame are used as the state vector of the Kalman filter algorithm, and the tracking prediction position of the human target in the current frame is predicted according to the preset motion model; wherein, the motion model represents a state transition model based on Doppler frequency shift features to enhance the expression of motion state.

3. The method according to claim 1, characterized in that, For each human target in the current frame, pairing the tracking and predicted position with the radar target position of each radar device to determine at least one radar target position associated with the human target includes: Calculate the feature difference between the tracking and predicted position of each human target and the position of each radar target in the same radar device, and construct the cost matrix between each human target and the same radar device based on each feature difference; In each radar device, for a cost matrix constructed by the same radar device based on each human target, the Hungarian algorithm is used to solve the cost matrix until all cost matrices are solved, and each human target is assigned at least one radar target position with a distance value less than a preset threshold.

4. The method according to claim 3, characterized in that, Before constructing the cost matrix between each human target and the same radar device based on the feature differences, the method further includes: With the predicted tracking position of each human target as the center and a preset threshold as the radius, an effective association region is defined for the predicted tracking position of each human target. Only the locations of radar targets falling within the effective association area are used to construct the cost matrix corresponding to the corresponding human targets.

5. The method according to claim 1, characterized in that, The reliability of the radar target location is characterized by the covariance matrix; The step of fusing the radar target positions associated with the same human target based on the weighting coefficients to obtain the fused observation position of the human target in the current frame includes: Based on the motion state of each human target in the indoor environment and the positional distribution characteristics of the radar target positions associated with the same human target, the covariance matrix corresponding to the radar target positions associated with the same human target is determined. Based on the covariance matrix corresponding to the radar target position, determine the weighting coefficients corresponding to the radar target position; Based on the weighting coefficients corresponding to the radar target positions, the radar target positions associated with the same human target are fused to obtain the fused observation position of the human target in the current frame.

6. The method according to claim 1, characterized in that, The reliability of the radar target location is characterized by the covariance matrix; The step of fusing the radar target positions associated with the same human target based on the weighting coefficients to obtain the fused observation position of the human target in the current frame includes: Based on the motion state of each human target in the indoor environment and the positional distribution characteristics of the radar target positions associated with the same human target, the covariance matrix corresponding to the radar target positions associated with the same human target is determined. Based on the covariance matrix corresponding to the radar target position, determine the weight coefficient corresponding to the radar target position, and construct a probability distribution model corresponding to the radar target position associated with the same human target. Based on the probability distribution model in continuous space, the joint probability of each spatial point in continuous space is evaluated, and the spatial point with the highest joint probability is taken as the fusion observation position of the corresponding human target in the current frame.

7. The method according to claim 1, characterized in that, The method further includes: If there is an abnormal tracking prediction position in each tracking prediction position of the current frame that is not paired with any radar target position, then according to the Kalman filter algorithm, the target tracking prediction position of the human target to which the abnormal tracking prediction position belongs in the next frame is predicted, and the target tracking prediction position in the next frame is paired with the radar target position obtained in the next frame. If there is a radar target position that is paired with the target tracking prediction position in the next frame, then the corresponding human target's fused observation position in the next frame is determined based on the paired radar target position. If no radar target position is found that matches the target tracking prediction position in the next frame, the target tracking prediction position in the next frame is re-predicted using the Kalman filter algorithm until a radar target position is found in the subsequent frames. If no radar target position is found within a preset number of frames, the abnormal tracking prediction position and the positions obtained by iterative prediction of the abnormal tracking prediction position in the subsequent frames are discarded.

8. An intelligent sensing system, characterized in that, The system includes: The acquisition module is used to acquire the radar target positions detected by multiple radar devices deployed in a preset indoor environment in the current frame, and to convert each radar target position to a unified spatial reference system; The prediction module is used to obtain the fusion state information of at least one human target determined in the previous frame, and based on the fusion state information, predict the tracking prediction position of each human target in the current frame using a Kalman filter algorithm. A pairing module is used to pair the tracking and prediction position with the radar target position of each of the radar devices for each human target in the current frame, and determine at least one radar target position associated with the human target. The fusion module is used to determine the weight coefficient of each associated radar target position, and based on the weight coefficient, fuse the radar target positions associated with the same human target to obtain the fused observation position of the human target in the current frame. The weight coefficient is used to characterize the reliability of the radar target position. The update module is used to take the fused observation position of the human target in the current frame as the observation value and input it into the Kalman filter algorithm for updating, so as to determine the fused positioning information of the human target in the current frame. The fused positioning information is used for position tracking and prediction in the next frame.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images