Indoor personnel active positioning perception method and system based on millimeter wave bio-radar

Abstract

The present application relates to the field of millimeter wave bio-radar, and specifically to a method and system for actively positioning and sensing indoor personnel based on millimeter wave bio-radar. The method collects personnel detection echo data through millimeter wave radar, and after processing the personnel detection echo data using a digital wave speed synthesis algorithm, the obtained target point cloud information is input into a data processing and analysis algorithm based on artificial intelligence and deep learning for processing, thereby capturing and depicting the local significant association semantics and key geometric structure feature information of the target point cloud, so as to perform personnel target semantic segmentation to determine the spatial coordinates of the personnel target, and can perform intelligent analysis on the target point cloud information to distinguish between personnel and other objects, so as to identify which points belong to the personnel target, and determine the spatial coordinates of the personnel target based on the target point cloud information corresponding to the personnel target, thereby achieving more accurate positioning perception of indoor personnel and reducing the possibility of missed reports.

Description

Technical Field

[0001] The present application relates to the field of millimeter-wave bio-radar, and specifically to a method and system for actively positioning and sensing indoor personnel based on millimeter-wave bio-radar. Background Art

[0002] With the rapid development of smart home and Internet of Things technologies, indoor occupant positioning and perception are becoming increasingly important in security monitoring, smart home management, health monitoring, etc. Although traditional indoor positioning methods such as Bluetooth, Wi-Fi, and ultra-wideband (UWB) can meet positioning requirements in some scenarios, they have their own limitations in terms of accuracy, cost, power consumption, etc.

[0003] Millimeter-wave radar has the advantages of strong penetration, high anti-interference ability and good real-time performance. It can effectively detect the presence of people in complex indoor environments and realize the perception and positioning of indoor people. Chinese patent CN116165650A discloses an indoor active positioning perception system and method based on millimeter-wave bio-radar, which can use millimeter-wave bio-radar sensor equipment to obtain human detection echo data within the coverage range, and perform data identification, aggregation, analysis and processing on these echo data, and then obtain the spatial coordinates, speed and angle data of the human target, and further visualize this information on an electronic map to complete the active positioning perception of indoor people.

[0004] Although the above-mentioned indoor personnel active positioning perception solution can locate and perceive indoor personnel, it has limitations in the accuracy of personnel target recognition and positioning. Specifically, the above-mentioned solution only performs simple processing on the echo data to extract information such as the spatial coordinates of the personnel, which may not be able to effectively distinguish personnel from other objects in a complex environment, resulting in inaccurate positioning results. For example, if there are objects in the environment that are similar in shape to personnel, such as backpacks, piles of clothes, etc., the system may not be able to accurately distinguish them, and thus mistake them for personnel, resulting in errors in the positioning perception of indoor personnel and false alarms.

[0005] Therefore, an optimized indoor personnel active positioning and perception solution based on millimeter-wave bioradar is needed to solve the above technical problems. Summary of the invention

[0006] This application is made in consideration of the above problems. One purpose of this application is to provide a method and system for actively locating and sensing indoor personnel based on millimeter wave bio-radar.

[0007] The embodiment of the present application provides a method for actively locating and sensing indoor personnel based on millimeter-wave bio-radar, which includes:

[0008] Obtaining personnel detection echo data within the coverage area of ​​the millimeter-wave radar;

[0009] Processing the personnel detection echo data using a digital wave velocity synthesis algorithm to obtain target point cloud information;

[0010] Extracting local point cloud correlation features from the target point cloud information to obtain a local correlation feature map of the target point cloud;

[0011] Inputting the target point cloud local correlation feature map into a feature distribution gradient mask salient device to obtain a target point cloud local salient correlation feature map;

[0012] Based on the local significant association feature map of the target point cloud, a semantic segmentation result of the person target is generated, and the spatial coordinates of the person target are determined.

[0013] For example, according to the embodiment of the present application, the indoor personnel active positioning and perception method based on millimeter-wave bioradar, wherein the local point cloud association feature extraction is performed on the target point cloud information to obtain the target point cloud local association feature map, including: inputting the target point cloud information into a local point cloud association feature extractor based on a three-dimensional convolutional neural network model to obtain the target point cloud local association feature map.

[0014] For example, according to the indoor personnel active positioning perception method based on millimeter wave bio-radar according to the embodiment of the present application, the local correlation feature map of the target point cloud is input into the feature distribution gradient mask salient device to obtain the local significant correlation feature map of the target point cloud, including:

[0015] Calculating the multidirectional gradient value distribution of each position in the local correlation feature map of the target point cloud, and determining the gradient amplitude value of each position in the local correlation feature map of the target point cloud based on the multidirectional gradient value distribution of each position to obtain a local correlation feature gradient amplitude distribution feature map of the target point cloud;

[0016] Calculating the local description operator of the gradient amplitude of each position in the local correlation feature gradient amplitude distribution feature map of the target point cloud to obtain a local significant distribution feature map of the local correlation feature gradient amplitude of the target point cloud;

[0017] Inputting the target point cloud local correlation feature gradient amplitude local significant distribution feature map into a gated masker based on a GELU function to obtain a target point cloud local correlation feature gradient amplitude local significant gated mask feature map;

[0018] The local significant correlation feature map of the target point cloud is obtained by calculating the position point multiplication between the local significant gated mask feature map of the local correlation feature gradient amplitude of the target point cloud and the local correlation feature map of the target point cloud.

[0019] For example, according to the indoor personnel active positioning perception method based on millimeter wave bio-radar according to the embodiment of the present application, the multi-directional gradient value distribution of each position in the local correlation feature map of the target point cloud is calculated, and the gradient amplitude value of each position in the local correlation feature map of the target point cloud is determined based on the multi-directional gradient value distribution of each position to obtain the target point cloud local correlation feature gradient amplitude distribution feature map, including:

[0020] Calculate the gradient amplitude value of each position feature value in the local correlation feature map of the target point cloud along the height direction to obtain a local correlation feature height gradient distribution feature map of the target point cloud;

[0021] Calculate the gradient amplitude value of each position feature value in the local correlation feature map of the target point cloud along the width direction to obtain a local correlation feature width gradient distribution feature map of the target point cloud;

[0022] Calculate the gradient amplitude value of each position feature value in the target point cloud local correlation feature map along the channel direction to obtain the target point cloud local correlation feature channel gradient distribution feature map;

[0023] The square root of the sum of squares of feature values ​​at corresponding positions in the target point cloud local correlation feature height gradient distribution feature map, the target point cloud local correlation feature width gradient distribution feature map and the target point cloud local correlation feature channel gradient distribution feature map is calculated to obtain the target point cloud local correlation feature gradient amplitude distribution feature map.

[0024] For example, according to the embodiment of the present application, an indoor personnel active positioning and perception method based on millimeter-wave bioradar is provided, wherein the gradient amplitude value of the feature value of each position in the local correlation feature map of the target point cloud is calculated along the channel direction to obtain the target point cloud local correlation feature channel gradient distribution feature map, including: calculating the difference between the feature value of the (k+1)th position and the feature value of the (k-1)th position along the channel dimension direction in the local correlation feature map of the target point cloud to obtain the channel gradient amplitude value of the kth position in the target point cloud local correlation feature channel gradient distribution feature map.

[0025] For example, according to the indoor personnel active positioning perception method based on millimeter wave bio-radar according to the embodiment of the present application, the local description operator of the gradient amplitude of each position in the local correlation feature gradient amplitude distribution feature map of the target point cloud is calculated to obtain the local significant distribution feature map of the local correlation feature gradient amplitude of the target point cloud, including:

[0026] Determine a local neighborhood range of the gradient amplitude, and calculate a sequence of target point cloud local correlation feature gradient amplitude distribution neighborhood feature maps based on the local neighborhood range of the gradient amplitude and the target point cloud local correlation feature gradient amplitude distribution feature map;

[0027] Counting the number of feature maps in the sequence of the target point cloud local correlation feature gradient amplitude distribution neighborhood feature map to obtain the target point cloud local correlation feature gradient amplitude distribution neighborhood number statistics;

[0028] Calculating the position difference between each target point cloud local correlation feature gradient amplitude distribution neighborhood feature map in the sequence of the target point cloud local correlation feature gradient amplitude distribution feature map and the target point cloud local correlation feature gradient amplitude distribution neighborhood feature map, and dividing the target point cloud local correlation feature gradient amplitude distribution feature map by position point to obtain a sequence of target point cloud local correlation feature gradient relative amplitude distribution neighborhood feature maps;

[0029] The local significant distribution feature map of the local associated feature gradient amplitude of the target point cloud is obtained by adding each target point cloud local associated feature gradient relative amplitude distribution neighborhood feature map in the sequence of the target point cloud local associated feature gradient relative amplitude distribution neighborhood feature map at the position point and dividing it by the target point cloud local associated feature gradient amplitude distribution neighborhood statistical value at the position point.

[0030] For example, according to the indoor personnel active positioning perception method based on millimeter wave bio-radar of the embodiment of the present application, based on the local significant association feature map of the target point cloud, the personnel target semantic segmentation result is generated, and the spatial coordinates of the personnel target are determined, including:

[0031] Inputting the local significant correlation feature map of the target point cloud into a classifier-based semantic segmentation network to obtain the semantic segmentation result of the person target;

[0032] Based on the target point cloud information corresponding to the semantic segmentation result of the personnel target, the spatial coordinates of the personnel target are determined.

[0033] The embodiment of the present application also provides an indoor personnel active positioning perception system based on millimeter wave bio-radar, which includes:

[0034] An echo data acquisition module is used to acquire the echo data of personnel detection within the coverage area of ​​the millimeter wave radar;

[0035] A synthesis processing module, used for processing the personnel detection echo data using a digital wave velocity synthesis algorithm to obtain target point cloud information;

[0036] A local point cloud correlation feature extraction module is used to extract local point cloud correlation features from the target point cloud information to obtain a local correlation feature map of the target point cloud;

[0037] A gradient mask module, used for inputting the target point cloud local correlation feature map into a feature distribution gradient mask salient device to obtain a target point cloud local salient correlation feature map;

[0038] The spatial coordinate determination module is used to generate a semantic segmentation result of a person target based on the local significant association feature map of the target point cloud, and determine the spatial coordinates of the person target.

[0039] For example, according to the indoor personnel active positioning and perception system based on millimeter-wave bioradar according to the embodiment of the present application, the local point cloud association feature extraction module is used to: input the target point cloud information into a local point cloud association feature extractor based on a three-dimensional convolutional neural network model to obtain the target point cloud local association feature map.

[0040] For example, according to the indoor personnel active positioning perception system based on millimeter wave bio-radar according to an embodiment of the present application, the gradient mask module is used to:

[0041] Calculating the multidirectional gradient value distribution of each position in the local correlation feature map of the target point cloud, and determining the gradient amplitude value of each position in the local correlation feature map of the target point cloud based on the multidirectional gradient value distribution of each position to obtain a local correlation feature gradient amplitude distribution feature map of the target point cloud;

[0042] Calculating the local description operator of the gradient amplitude of each position in the local correlation feature gradient amplitude distribution feature map of the target point cloud to obtain a local significant distribution feature map of the local correlation feature gradient amplitude of the target point cloud;

[0043] Inputting the target point cloud local correlation feature gradient amplitude local significant distribution feature map into a gated masker based on a GELU function to obtain a target point cloud local correlation feature gradient amplitude local significant gated mask feature map;

[0044] The local significant correlation feature map of the target point cloud is obtained by calculating the position point multiplication between the local significant gated mask feature map of the local correlation feature gradient amplitude of the target point cloud and the local correlation feature map of the target point cloud.

[0045] The present application has at least the following technical effects: Compared with the prior art, the method and system for actively positioning and sensing indoor personnel based on millimeter-wave bio-radar according to the embodiment of the present application collects personnel detection echo data through millimeter-wave radar, and after processing the personnel detection echo data using a digital wave speed synthesis algorithm, the obtained target point cloud information is input into a data processing and analysis algorithm based on artificial intelligence and deep learning for processing, thereby capturing and depicting the local salient association semantics and key geometric structure feature information of the target point cloud, so as to perform semantic segmentation of personnel targets to determine the spatial coordinates of the personnel targets. The present application can perform intelligent analysis of the target point cloud information to distinguish between personnel and other objects, to identify which points belong to the personnel target, and to determine the spatial coordinates of the personnel target based on the target point cloud information corresponding to the personnel target, thereby achieving more accurate positioning perception of indoor personnel and reducing the possibility of missed reports. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings of the embodiments of the present application are briefly introduced below. Obviously, the drawings described below only relate to some embodiments of the present application, and are not intended to limit the present application.

[0047] Figure 1 A schematic diagram of the application architecture of the method for active indoor personnel positioning and sensing based on millimeter-wave bio-radar in an embodiment of the present application is shown;

[0048] Figure 2 A flow chart of an indoor personnel active positioning and sensing method based on millimeter-wave bio-radar in an embodiment of the present application is shown;

[0049] Figure 3 A flowchart of sub-step S540 of the method for actively locating and sensing indoor personnel based on millimeter-wave bio-radar in an embodiment of the present application is shown;

[0050] Figure 4 A structural schematic diagram of an indoor personnel active positioning perception system based on millimeter-wave bio-radar in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work also fall within the scope of protection of the present application.

[0052] The terms used in this specification are those common terms currently widely used in the art in consideration of the functions of the present application, but these terms may vary according to the intentions of those skilled in the art, precedents, or new technologies in the art. In addition, specific terms may be selected, and in this case, their detailed meanings will be described in the detailed description of the present application. Therefore, the terms used in the specification should not be understood as simple names, but rather as a general description based on the meaning of the terms and the present application.

[0053] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0054] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. At the same time, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0055] Figure 1 A schematic diagram of the application architecture of an active indoor personnel positioning and sensing method based on millimeter-wave bio-radar in an embodiment of the present application is shown, including a server 100 and a terminal device 200.

[0056] The terminal device 200 and the server 100 can be connected via the Internet to achieve mutual communication. Optionally, the above-mentioned Internet uses standard communication technology and / or protocol. The Internet is usually the Internet, but it can also be any network, including but not limited to any combination of a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a private network or a virtual private network. In some embodiments, the data exchanged through the network can be represented by technologies and / or formats including Hyper Text Markup Language (HTML), Extensible Markup Language (XML), etc. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec), etc. can be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above-mentioned data communication technologies.

[0057] The server 100 can provide various network services for the terminal device 200, wherein the server 100 can be a single server, a server cluster consisting of several servers, or a cloud computing center. Specifically, the server 100 may include a processor 110 (Center Processing Unit, CPU), a memory 120, an input device 130, and an output device 140, etc. The input device 130 may include a keyboard, a mouse, a touch screen, etc., and the output device 140 may include a display device, such as a liquid crystal display (Liquid Crystal Display, LCD), a cathode ray tube (Cathode Ray Tube, CRT), etc.

[0058] The memory 120 may include a read-only memory (ROM) and a random access memory (RAM), and provides the processor 110 with program instructions and data stored in the memory 120. In the embodiment of the present application, the memory 120 may be used to store the program of the indoor personnel active positioning perception method based on millimeter wave bio-radar in the embodiment of the present application.

[0059] The processor 110 calls the program instructions stored in the memory 120, and the processor 110 is used to execute the steps of any one of the indoor personnel active positioning and perception methods based on millimeter-wave bioradar in the embodiments of the present application according to the obtained program instructions.

[0060] In addition, the application architecture diagram in the embodiment of the present application is intended to more clearly illustrate the technical solution in the embodiment of the present application, and does not constitute a limitation on the technical solution provided in the embodiment of the present application. Of course, for other application architectures and business applications, the technical solution provided in the embodiment of the present application is also applicable to similar problems.

[0061] The following is a non-restrictive explanation of the indoor personnel active positioning and perception method based on millimeter-wave bioradar provided according to at least one embodiment of the present application through several examples or embodiments. As described below, different features in these specific examples or embodiments can be combined with each other without conflicting with each other to obtain new examples or embodiments, and these new examples or embodiments also fall within the scope of protection of the present application.

[0062] In response to the above technical problems, the technical concept of this application is to collect personnel detection echo data through millimeter wave radar, and after processing the personnel detection echo data using a digital wave speed synthesis algorithm, the obtained target point cloud information is input into a data processing and analysis algorithm based on artificial intelligence and deep learning for processing, thereby capturing and characterizing the local significant association semantics and key geometric structure feature information of the target point cloud, so as to perform personnel target semantic segmentation to determine the spatial coordinates of the personnel target, and to perform intelligent analysis of the target point cloud information to distinguish between personnel and other objects, so as to identify which points belong to the personnel target, and determine the spatial coordinates of the personnel target based on the target point cloud information corresponding to the personnel target. In this way, it is possible to achieve more accurate positioning perception of indoor personnel and reduce the possibility of missed reports.

[0063] Figure 2 The flowchart of the method for actively locating and sensing indoor personnel based on millimeter wave bio-radar in the embodiment of the present application is shown. For example, the method for actively locating and sensing indoor personnel based on millimeter wave bio-radar can be executed by a server, which can be Figure 1 The server 100 shown in FIG. Figure 2As shown, according to the embodiment of the present application, the indoor personnel active positioning and perception method based on millimeter-wave bio-radar includes the following steps: S510, acquiring personnel detection echo data within the coverage range of the millimeter-wave radar; S520, processing the personnel detection echo data using a digital wave velocity synthesis algorithm to obtain target point cloud information; S530, performing local point cloud association feature extraction on the target point cloud information to obtain a target point cloud local association feature map; S540, inputting the target point cloud local association feature map into a feature distribution gradient mask salient device to obtain a target point cloud local significant association feature map; S550, generating a personnel target semantic segmentation result based on the target point cloud local significant association feature map, and determining the spatial coordinates of the personnel target.

[0064] Specifically, in the technical solution of the present application, first, the personnel detection echo data within the coverage range of the millimeter wave radar is obtained. Then, the personnel detection echo data is processed using a digital wave velocity synthesis algorithm to obtain target point cloud information. Specifically, the digital wave velocity synthesis algorithm can effectively filter out noise and interference and enhance the clarity of the signal. This is particularly important for millimeter wave radars, because there may be a variety of interference sources in indoor environments (such as other electronic devices, building materials, etc.), which affect the quality of the echo signal. The digital wave velocity synthesis algorithm can help the system extract useful information from the original radar echo data and construct the three-dimensional position information of the target to obtain the target point cloud, thereby improving the angular resolution and distance resolution of the radar, which is very important for identifying small targets or close targets and subsequent personnel perception and positioning tasks.

[0065] Then, in order to accurately identify the human target in the future and obtain a more accurate result of semantic segmentation of the human target, it is necessary to effectively extract and analyze the target point cloud information. Based on this, in the technical solution of the present application, the target point cloud information is further input into a local point cloud association feature extractor based on a three-dimensional convolutional neural network model for feature mining, so as to extract the local association distribution features and semantic information of the target point cloud information, thereby obtaining a local association feature map of the target point cloud.

[0066] Correspondingly, in step S530, local point cloud association features are extracted from the target point cloud information to obtain a target point cloud local association feature map, including: inputting the target point cloud information into a local point cloud association feature extractor based on a three-dimensional convolutional neural network model to obtain the target point cloud local association feature map.

[0067] Furthermore, since the local correlation feature map of the target point cloud contains the local correlation feature distribution information about the target point cloud, some of the local correlation features of the target point cloud are crucial for the target semantic segmentation, while some are irrelevant interference features and redundant information. Therefore, in order to enhance the expression ability of the local correlation semantics of the key point cloud in the local correlation feature map of the target point cloud, so as to help more accurately understand and perceive the distribution of the local correlation features of the target point cloud, and provide a basis for the target semantic segmentation, in the technical solution of the present application, the local correlation feature map of the target point cloud is further input into the feature distribution gradient mask salient device to obtain the local significant correlation feature map of the target point cloud. It is worth mentioning that the main function of the feature distribution gradient mask salient device is to automatically identify and highlight the key features related to the target semantic segmentation task by analyzing the gradient information in the local correlation feature map of the target point cloud. In other words, the important information in the local correlation feature map of the target point cloud is highlighted by the feature gradient amplitude measurement and the gated mask mechanism, which helps the model focus on the most important part, thereby improving the accuracy of personnel target recognition and semantic segmentation.

[0068] Specifically, the feature distribution gradient mask salient device first calculates the gradient amplitude value of each position in the local correlation feature map of the target point cloud in multiple directions, and can detect which areas in the feature map have more obvious local changes in the local correlation semantic gradient amplitude, that is, which areas contain more information related to the local correlation feature distribution of the target point cloud. Then, based on the result of the gradient calculation, the local description operator of each position in the local correlation feature gradient amplitude distribution map of the target point cloud is calculated to analyze which local correlation distribution features of the point cloud are more critical and important for target recognition, so as to highlight the significant areas of the gradient amplitude. Then, the nonlinear characteristics of the GELU (Gaussian Error Linear Unit) function are used to enhance or suppress certain areas in the local significant distribution map of the gradient amplitude of the local correlation feature of the target point cloud, so as to ensure that the key features related to the subsequent personnel target recognition and semantic segmentation tasks are retained, while the non-key features are weakened, thereby enhancing the response of the model to the local correlation distribution features of the salient point cloud. Finally, the enhanced local correlation features of the target point cloud are combined with the original features through the weighted point multiplication operation to obtain the optimized local significant correlation feature map of the target point cloud. This feature enhancement method can better distinguish between people and other objects to identify which points belong to human targets. At the same time, it can help the model to stably identify targets when facing complex backgrounds or interference, thereby more accurately identifying and semantically segmenting human target objects, which helps with indoor personnel perception and positioning tasks and improves the robustness and accuracy of the system.

[0069] Accordingly, in step S540, if Figure 3As shown, the target point cloud local correlation feature map is input into the feature distribution gradient mask salient device to obtain the target point cloud local significant correlation feature map, including: S541, calculating the multi-directional gradient value distribution of each position in the target point cloud local correlation feature map, and determining the gradient amplitude value of each position in the target point cloud local correlation feature map based on the multi-directional gradient value distribution of each position to obtain the target point cloud local correlation feature gradient amplitude distribution feature map; S542, calculating the gradient amplitude local description operator of each position in the target point cloud local correlation feature gradient amplitude distribution feature map to obtain the target point cloud local correlation feature gradient amplitude local significant distribution feature map; S543, inputting the target point cloud local correlation feature gradient amplitude local significant distribution feature map into the gated masker based on the GELU function to obtain the target point cloud local correlation feature gradient amplitude local significant gated mask feature map; S544, calculating the position point multiplication between the target point cloud local correlation feature gradient amplitude local significant gated mask feature map and the target point cloud local correlation feature map to obtain the target point cloud local significant correlation feature map.

[0070] Among them, in step S541, the multi-directional gradient value distribution of each position in the local association feature map of the target point cloud is calculated, and the gradient amplitude value of each position in the local association feature map of the target point cloud is determined based on the multi-directional gradient value distribution of each position to obtain the gradient amplitude distribution feature map of the local association feature of the target point cloud, including: calculating the gradient amplitude value of each position feature value in the local association feature map of the target point cloud along the height direction to obtain the target point cloud local association feature height gradient distribution feature map; calculating the gradient amplitude value of each position feature value in the local association feature map of the target point cloud along the width direction to obtain the target point cloud local association feature width gradient distribution feature map; calculating the gradient amplitude value of each position feature value in the local association feature map of the target point cloud along the channel direction to obtain the target point cloud local association feature channel gradient distribution feature map; calculating the square root of the sum of the squares of the feature values ​​of the corresponding positions in the local association feature height gradient distribution feature map of the target point cloud, the local association feature width gradient distribution feature map of the target point cloud, and the local association feature channel gradient distribution feature map of the target point cloud to obtain the target point cloud local association feature gradient amplitude distribution feature map.

[0071] Specifically, the gradient amplitude value of the feature value at each position in the local correlation feature map of the target point cloud is calculated along the channel direction to obtain the target point cloud local correlation feature channel gradient distribution feature map, including: calculating the difference between the feature value at the (k+1)th position and the feature value at the (k-1)th position along the channel dimension direction in the local correlation feature map of the target point cloud to obtain the channel gradient amplitude value of the kth position in the channel gradient distribution feature map of the local correlation feature of the target point cloud, where k is the number of local correlation feature maps of the target point cloud.

[0072] Wherein, in step S542, the local description operator of the gradient amplitude of each position in the target point cloud local correlation feature gradient amplitude distribution feature map is calculated to obtain the target point cloud local correlation feature gradient amplitude local significant distribution feature map, including: determining the local neighborhood range of the gradient amplitude, and calculating the sequence of the target point cloud local correlation feature gradient amplitude distribution neighborhood feature map based on the gradient amplitude local neighborhood range and the target point cloud local correlation feature gradient amplitude distribution feature map; counting the number of feature maps in the sequence of the target point cloud local correlation feature gradient amplitude distribution neighborhood feature map to obtain the target point cloud local correlation feature gradient amplitude distribution neighborhood number statistics; calculating the target point cloud local correlation feature gradient amplitude distribution feature map and The position differences between each target point cloud local correlation feature gradient amplitude distribution neighborhood feature map in the sequence of target point cloud local correlation feature gradient amplitude distribution neighborhood feature maps are divided by position point with the target point cloud local correlation feature gradient amplitude distribution feature map to obtain a sequence of target point cloud local correlation feature gradient relative amplitude distribution neighborhood feature maps; each target point cloud local correlation feature gradient relative amplitude distribution neighborhood feature map in the sequence of target point cloud local correlation feature gradient relative amplitude distribution neighborhood feature maps is added by position point and divided by the target point cloud local correlation feature gradient amplitude distribution neighborhood number statistics value to obtain the target point cloud local correlation feature gradient amplitude local significant distribution feature map.

[0073] In a specific example, the target point cloud local correlation feature map is input into a feature distribution gradient mask salient device to obtain a target point cloud local significant correlation feature map, including: inputting the target point cloud local correlation feature map into the feature distribution gradient mask salient device and processing it with the following gradient significant feature enhancement formula to obtain the target point cloud local significant correlation feature map; wherein the gradient significant feature enhancement formula is:

[0074]

[0075]

[0076]

[0077]

[0078]

[0079]

[0080]

[0081] in, , , , , and They are respectively the first , , , , and The eigenvalues ​​of the positions, is the first The gradient amplitude value at each position, is the first The gradient amplitude value at each position, is the first The gradient amplitude value at each position, is the first The eigenvalues ​​at the positions, is the first The eigenvalues ​​at the positions, The range is Neighborhood of express The number of elements in , Indicates the local significant distribution feature map of the local correlation feature gradient amplitude of the target point cloud. The eigenvalues ​​at the positions, represents the Gaussian error linear gating function, Represents the local correlation feature gradient amplitude of the target point cloud in the local significant gated mask feature map The eigenvalues ​​at the positions, is the local significant gated mask feature map of the local correlation feature gradient amplitude of the target point cloud, is the local correlation feature map of the target point cloud, It means point multiplication by position. It is the local significant correlation feature map of the target point cloud.

[0082] Then, the local significant association feature map of the target point cloud is input into the semantic segmentation network based on the classifier to obtain the semantic segmentation result of the person target. In other words, the local significant association semantic representation information and geometric structure feature information of the target point cloud are used for classification processing to perform semantic segmentation of the person target, and the spatial coordinates of the person target are determined based on the target point cloud information corresponding to the semantic segmentation result of the person target. In this way, the target point cloud information can be intelligently analyzed to distinguish between people and other objects, so as to identify which points belong to the person target, and determine the spatial coordinates of the person target based on the target point cloud information corresponding to the person target. In this way, more accurate positioning perception of indoor personnel can be achieved, reducing the possibility of underreporting.

[0083] Accordingly, in step S550, based on the local significant association feature map of the target point cloud, a semantic segmentation result of the personnel target is generated, and the spatial coordinates of the personnel target are determined, including: inputting the local significant association feature map of the target point cloud into a classifier-based semantic segmentation network to obtain the semantic segmentation result of the personnel target; and determining the spatial coordinates of the personnel target based on the target point cloud information corresponding to the semantic segmentation result of the personnel target.

[0084] Specifically, the role of the classifier is to use the given categories and known training data to learn classification rules and classifiers, and then classify (or predict) unknown data. Logistic regression, SVM, etc. are often used to solve binary classification problems. For multi-class classification problems, logistic regression or SVM can also be used, but multiple binary classifications are required to form a multi-classification. However, this is prone to errors and is inefficient. Commonly used multi-classification methods include the Softmax classification function.

[0085] Preferably, the local significant association feature map of the target point cloud is input into a semantic segmentation network based on a classifier to obtain the semantic segmentation result of the person target, including: expanding the local significant association feature map of the target point cloud into a local significant association feature vector of the target point cloud; subtracting the zero norm of the local significant association feature vector of the target point cloud from the length of the local significant association feature vector of the target point cloud to obtain the local significant association isolated representation value of the target point cloud; calculating the logarithm of the sum of the square of the local significant association isolated representation value of the target point cloud and the local significant association isolated representation value of the target point cloud with base 2 to obtain the local significant association information order value of the target point cloud; calculating a power function with each eigenvalue of the local significant association feature vector of the target point cloud as the base and the difference of the local significant association isolated representation value of the target point cloud minus one as the exponential, and summing it with the local significant association information of the target point cloud The order value is dot-multiplied to obtain the target point cloud local significant association leading vector; the target point cloud local significant association feature vector is dot-multiplied with the difference of the target point cloud local significant association isolated representation value minus one, and then the difference is dot-multiplied with the inverse of the target point cloud local significant association isolated representation value to obtain the target point cloud local significant association field constraint vector; an exponential function with each eigenvalue of the target point cloud local significant association field constraint vector as the base is calculated to obtain the target point cloud local significant association field bias vector; the target point cloud local significant association leading vector is dot-added with the target point cloud local significant association field bias vector to obtain the optimized target point cloud local significant association feature vector; and the optimized target point cloud local significant association feature vector is input into the classifier-based semantic segmentation network to obtain the semantic segmentation result of the person target.

[0086] Specifically, the local significant correlation feature vector of the target point cloud is, for example, denoted as The optimization process is expressed as follows:

[0087]

[0088]

[0089] in, represents the local significant correlation feature vector of the target point cloud, represents the length of the local significant correlation feature vector of the target point cloud, represents the zero norm of a vector, represents the local significant associated isolated representation value of the target point cloud, represents the logarithm operation with base 2, represents the exponential operation with base e, represents a power function that calculates the eigenvalues ​​of each position in the eigenvector using the difference between the local significant associated isolated representation value of the target point cloud and one as an exponent, Indicates point addition, represents dot product, represents the optimized local significant correlation feature vector of the target point cloud.

[0090] Among them, the target point cloud local correlation feature map is used to represent the three-dimensional local semantic correlation features of the point cloud information determined based on the three-dimensional convolutional coding of the target point cloud information. After the target point cloud local correlation feature map is input into the feature distribution gradient mask salient device, the feature distribution gradient mask salient device uses the gradient values ​​of each position in the target point cloud local correlation feature map to construct a quasi-attention gating mechanism to improve the feature expression significance of the target point cloud local correlation feature map in the spatial feature. Therefore, it is expected to further improve the comprehensibility of the spatial dimension feature distribution difference aggregation regression of the target point cloud local correlation feature map, thereby improving the accuracy of the semantic segmentation results of the personnel target obtained by inputting it into the classifier-based semantic segmentation network.

[0091] Therefore, the high-dimensional feature manifold of the local significant associated feature vector of the target point cloud obtained by expanding the local significant associated feature map of the target point cloud is represented by the feature value of the feature set as the vector field of the aggregation dimension, and the local position of the feature value of the feature set is fixed by using the superposition value of the vector field of the local significant associated feature vector of the target point cloud at the isolated zero position as the order information, and the bias of the reversibility of the feature regression distribution field of the local significant associated feature vector of the target point cloud is added as a reward to achieve the mapping target tracking of the regression distribution of the local significant associated feature vector of the target point cloud for the feature value position, so that the feature set of the local significant associated feature vector of the target point cloud perceives the mapping migration to the aggregate distribution, thereby improving the accuracy of the semantic segmentation result of the person target obtained by inputting the local significant associated feature map of the target point cloud into the semantic segmentation network based on the classifier by improving the feature difference aggregation regression understandability of the local significant associated feature map of the target point cloud. In this way, the person and other objects in the target point cloud information can be more accurately identified and distinguished, so as to identify which points belong to the person target, and determine the spatial coordinates of the person target based on the target point cloud information corresponding to the person target.

[0092] Based on the above embodiments, see Figure 4As shown, it is a structural schematic diagram of an indoor personnel active positioning perception system 800 based on millimeter wave bio-radar in an embodiment of the present application. The indoor personnel active positioning perception system 800 based on millimeter wave bio-radar includes: an echo data acquisition module 810, which is used to acquire personnel detection echo data within the coverage range of the millimeter wave radar; a synthesis processing module 820, which is used to process the personnel detection echo data using a digital wave speed synthesis algorithm to obtain target point cloud information; a local point cloud association feature extraction module 830, which is used to extract local point cloud association features from the target point cloud information to obtain a target point cloud local association feature map; a gradient mask module 840, which is used to input the target point cloud local association feature map into a feature distribution gradient mask salient device to obtain a target point cloud local significant association feature map; a spatial coordinate determination module 850, which is used to generate a personnel target semantic segmentation result based on the target point cloud local significant association feature map, and determine the spatial coordinates of the personnel target.

[0093] In one example, in the above-mentioned millimeter-wave bioradar-based indoor personnel active positioning perception system 800, the local point cloud association feature extraction module 830 is used to: input the target point cloud information into a local point cloud association feature extractor based on a three-dimensional convolutional neural network model to obtain the target point cloud local association feature map.

[0094] In one example, in the above-mentioned indoor personnel active positioning perception system 800 based on millimeter wave bioradar, the gradient mask module 840 is used to: calculate the multi-directional gradient value distribution of each position in the local correlation feature map of the target point cloud, and determine the gradient amplitude value of each position in the local correlation feature map of the target point cloud based on the multi-directional gradient value distribution of each position to obtain the target point cloud local correlation feature gradient amplitude distribution feature map; calculate the gradient amplitude local description operator of each position in the local correlation feature gradient amplitude distribution feature map of the target point cloud to obtain the target point cloud local correlation feature gradient amplitude local significant distribution feature map; input the target point cloud local correlation feature gradient amplitude local significant distribution feature map into the gated masker based on the GELU function to obtain the target point cloud local correlation feature gradient amplitude local significant gated mask feature map; calculate the position point multiplication between the target point cloud local correlation feature gradient amplitude local significant gated mask feature map and the target point cloud local correlation feature map to obtain the target point cloud local significant correlation feature map.

[0095] Here, those skilled in the art can understand that the specific functions and operations of each module in the above-mentioned indoor personnel active positioning perception system 800 based on millimeter wave bio-radar have been referred to above. Figure 2 to Figure 3 The invention has been introduced in detail in the description of the indoor personnel active positioning perception method based on millimeter-wave bioradar, and therefore, its repeated description will be omitted.

[0096] Based on the above embodiments, another exemplary embodiment of an electronic device is also provided in the embodiments of the present application. In some possible implementations, the electronic device in the embodiments of the present application may include a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor can implement the steps of the indoor personnel active positioning perception method based on millimeter wave bio-radar in the above embodiments when executing the program.

[0097] For example, in the case of electronic equipment Figure 1 Taking the server 100 in the example as an example, the processor in the electronic device is the processor 110 in the server 100, and the memory in the electronic device is the memory 120 in the server 100.

[0098] The embodiment of the present application also provides a computer-readable storage medium, on which computer-executable instructions are stored. When the computer-executable instructions are executed by the processor, the method for actively positioning and sensing indoor personnel based on millimeter-wave bio-radar according to the embodiment of the present application described with reference to the above figures can be executed. The computer-readable storage medium includes, but is not limited to, for example, volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, etc.

[0099] The embodiments of the present application also provide a computer program product or a computer program, which includes computer executable instructions, and the computer executable instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer executable instructions from the computer-readable storage medium, and the processor executes the computer executable instructions, so that the computer device executes the indoor personnel active positioning perception method based on millimeter-wave bio-radar according to the embodiments of the present application.

[0100] Those skilled in the art will appreciate that the contents disclosed in this application may be subject to various variations and improvements. For example, the various devices or components described above may be implemented by hardware, or by software, firmware, or a combination of some or all of the three.

[0101] In addition, although the present application makes various references to certain units in the system according to embodiments of the present application, any number of different units can be used and run on the client and / or server. The units are only illustrative, and different aspects of the system and method can use different units.

[0102] Those skilled in the art will appreciate that all or part of the steps in the above method can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, such as a read-only memory, a disk or an optical disk. Optionally, all or part of the steps in the above embodiment can also be implemented using one or more integrated circuits. Accordingly, each module / unit in the above embodiment can be implemented in the form of hardware or in the form of software function modules. The present application is not limited to any particular form of combination of hardware and software.

[0103] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art to which this application belongs. It should also be understood that terms such as those defined in common dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an idealized or extremely formal sense, unless explicitly defined as such herein.

[0104] The above is an explanation of the present application and should not be considered as limiting thereof. Although several exemplary embodiments of the present application have been described, those skilled in the art will readily appreciate that many modifications may be made to the exemplary embodiments without departing from the novel teachings and advantages of the present application.

Claims

1. Active positioning and perception method of indoor personnel based on millimeter-wave bio-radar, It is characterized in that include: Obtaining personnel detection echo data within the coverage area of ​​the millimeter-wave radar; Processing the personnel detection echo data using a digital wave velocity synthesis algorithm to obtain target point cloud information; Extracting local point cloud correlation features from the target point cloud information to obtain a local correlation feature map of the target point cloud; Inputting the target point cloud local correlation feature map into a feature distribution gradient mask salient device to obtain a target point cloud local salient correlation feature map; Based on the local significant correlation feature map of the target point cloud, a semantic segmentation result of the person target is generated, and the spatial coordinates of the person target are determined; The method of inputting the target point cloud local correlation feature map into a feature distribution gradient mask salient device to obtain a target point cloud local salient correlation feature map comprises: Calculating the multidirectional gradient value distribution of each position in the local correlation feature map of the target point cloud, and determining the gradient amplitude value of each position in the local correlation feature map of the target point cloud based on the multidirectional gradient value distribution of each position to obtain a local correlation feature gradient amplitude distribution feature map of the target point cloud; Calculating the local description operator of the gradient amplitude of each position in the local correlation feature gradient amplitude distribution feature map of the target point cloud to obtain a local significant distribution feature map of the local correlation feature gradient amplitude of the target point cloud; Inputting the target point cloud local correlation feature gradient amplitude local significant distribution feature map into a gated masker based on a GELU function to obtain a target point cloud local correlation feature gradient amplitude local significant gated mask feature map; The local significant correlation feature map of the target point cloud is obtained by calculating the position point multiplication between the local significant gated mask feature map of the local correlation feature gradient amplitude of the target point cloud and the local correlation feature map of the target point cloud.

2. According to claim 1, the method for actively positioning and sensing indoor personnel based on millimeter-wave bio-radar, It is characterized in that Performing local point cloud correlation feature extraction on the target point cloud information to obtain a target point cloud local correlation feature map, including: inputting the target point cloud information into a local point cloud correlation feature extractor based on a three-dimensional convolutional neural network model to obtain the target point cloud local correlation feature map.

3. According to claim 2, the method for actively locating and sensing indoor personnel based on millimeter-wave bio-radar, It is characterized in that Calculating the multidirectional gradient value distribution of each position in the local correlation feature map of the target point cloud, and determining the gradient amplitude value of each position in the local correlation feature map of the target point cloud based on the multidirectional gradient value distribution of each position to obtain a local correlation feature gradient amplitude distribution feature map of the target point cloud, including: Calculate the gradient amplitude value of each position feature value in the local correlation feature map of the target point cloud along the height direction to obtain a local correlation feature height gradient distribution feature map of the target point cloud; Calculate the gradient amplitude value of each position feature value in the local correlation feature map of the target point cloud along the width direction to obtain a local correlation feature width gradient distribution feature map of the target point cloud; Calculate the gradient amplitude value of each position feature value in the local correlation feature map of the target point cloud along the channel direction to obtain a channel gradient distribution feature map of the local correlation feature of the target point cloud; The square root of the sum of squares of feature values ​​at corresponding positions in the target point cloud local correlation feature height gradient distribution feature map, the target point cloud local correlation feature width gradient distribution feature map and the target point cloud local correlation feature channel gradient distribution feature map is calculated to obtain the target point cloud local correlation feature gradient amplitude distribution feature map.

4. According to claim 3, the method for actively locating and sensing indoor personnel based on millimeter-wave bio-radar, It is characterized in that The gradient amplitude value of the feature value at each position in the local correlation feature map of the target point cloud is calculated along the channel direction to obtain the channel gradient distribution feature map of the local correlation feature of the target point cloud, including: calculating the difference between the feature value at the (k+1)th position and the feature value at the (k-1)th position along the channel dimension direction in the local correlation feature map of the target point cloud to obtain the channel gradient amplitude value of the kth position in the channel gradient distribution feature map of the local correlation feature of the target point cloud, wherein k is the number of local correlation feature maps of the target point cloud.

5. According to claim 4, the method for actively locating and sensing indoor personnel based on millimeter-wave bio-radar, It is characterized in that Calculating the local description operator of the gradient amplitude of each position in the local correlation feature gradient amplitude distribution feature map of the target point cloud to obtain the local significant distribution feature map of the local correlation feature gradient amplitude of the target point cloud, including: Determine a local neighborhood range of the gradient amplitude, and calculate a sequence of target point cloud local correlation feature gradient amplitude distribution neighborhood feature maps based on the local neighborhood range of the gradient amplitude and the target point cloud local correlation feature gradient amplitude distribution feature map; Counting the number of feature maps in the sequence of the target point cloud local correlation feature gradient amplitude distribution neighborhood feature map to obtain the target point cloud local correlation feature gradient amplitude distribution neighborhood number statistics; Calculating the position difference between each target point cloud local correlation feature gradient amplitude distribution neighborhood feature map in the sequence of the target point cloud local correlation feature gradient amplitude distribution feature map and the target point cloud local correlation feature gradient amplitude distribution neighborhood feature map, and dividing the target point cloud local correlation feature gradient amplitude distribution feature map by position point to obtain a sequence of target point cloud local correlation feature gradient relative amplitude distribution neighborhood feature maps; The local significant distribution feature map of the local associated feature gradient amplitude of the target point cloud is obtained by adding each target point cloud local associated feature gradient relative amplitude distribution neighborhood feature map in the sequence of the target point cloud local associated feature gradient relative amplitude distribution neighborhood feature map at the position point and dividing it by the target point cloud local associated feature gradient amplitude distribution neighborhood statistical value at the position point.

6. The method for actively positioning and sensing indoor personnel based on millimeter-wave bio-radar according to claim 5, It is characterized in that Based on the local significant correlation feature map of the target point cloud, a semantic segmentation result of the person target is generated, and the spatial coordinates of the person target are determined, including: Inputting the local significant correlation feature map of the target point cloud into a classifier-based semantic segmentation network to obtain the semantic segmentation result of the person target; Based on the target point cloud information corresponding to the semantic segmentation result of the personnel target, the spatial coordinates of the personnel target are determined.

7. Indoor personnel active positioning perception system based on millimeter wave bio-radar, It is characterized in that include: An echo data acquisition module is used to acquire the echo data of personnel detection within the coverage area of ​​the millimeter wave radar; A synthesis processing module, used for processing the personnel detection echo data using a digital wave velocity synthesis algorithm to obtain target point cloud information; A local point cloud correlation feature extraction module is used to extract local point cloud correlation features from the target point cloud information to obtain a local correlation feature map of the target point cloud; A gradient mask module, used for inputting the target point cloud local correlation feature map into a feature distribution gradient mask salient device to obtain a target point cloud local salient correlation feature map; A spatial coordinate determination module, used to generate a semantic segmentation result of a person target based on the local significant association feature map of the target point cloud, and determine the spatial coordinates of the person target; Wherein, the gradient mask module is used for: Calculating the multidirectional gradient value distribution of each position in the local correlation feature map of the target point cloud, and determining the gradient amplitude value of each position in the local correlation feature map of the target point cloud based on the multidirectional gradient value distribution of each position to obtain a local correlation feature gradient amplitude distribution feature map of the target point cloud; Calculating the local description operator of the gradient amplitude of each position in the local correlation feature gradient amplitude distribution feature map of the target point cloud to obtain a local significant distribution feature map of the local correlation feature gradient amplitude of the target point cloud; Inputting the target point cloud local correlation feature gradient amplitude local significant distribution feature map into a gated masker based on a GELU function to obtain a target point cloud local correlation feature gradient amplitude local significant gated mask feature map; The local significant correlation feature map of the target point cloud is obtained by calculating the position point multiplication between the local significant gated mask feature map of the local correlation feature gradient amplitude of the target point cloud and the local correlation feature map of the target point cloud.

8. The indoor personnel active positioning and sensing system based on millimeter-wave bio-radar according to claim 7, It is characterized in that The local point cloud association feature extraction module is used to: input the target point cloud information into a local point cloud association feature extractor based on a three-dimensional convolutional neural network model to obtain the target point cloud local association feature map.

Images