Driver status point cloud detection method based on millimeter wave radar

Abstract

The present invention provides a driver status point cloud detection method based on millimeter wave radar, which relates to the field of millimeter wave radar technology, including: obtaining a target data matrix, which is composed of target vital sign signals of a target driver dynamically measured by a multi-transmitter and multi-receiver MIMO-FMCW millimeter wave radar based on a predetermined measurement distance; determining the distance unit in combination with the position of the target part and the radar distance resolution, performing Fourier transform on the target data matrix, and obtaining a target motion three-dimensional point cloud; analyzing and constructing a target space map; performing latent space normal estimation between consecutive frames on the target space map to obtain a target continuous frame point cloud; and evaluating the state of the target driver based on a target dense point cloud obtained by multi-frame fusion processing of the target continuous frame point cloud. The present invention solves the technical problems that traditional driver status monitoring methods often rely on vision or sensor data, and have limited vision, large environmental interference, and difficulty in identifying complex scenes.

Description

Technical Field

[0001] The present invention relates to the technical field of millimeter wave radar, and in particular to a driver status point cloud detection method based on millimeter wave radar. Background Art

[0002] Traditional driver status monitoring technology mainly relies on cameras, infrared sensors and other equipment. These methods have technical problems in some cases. Specifically, traditional methods mainly rely on cameras to obtain driver status information, but at night, in bad weather or poor lighting, vision is limited and the driver's status cannot be accurately obtained; in high-speed driving or complex traffic scenes, interference from the surrounding environment will affect the accuracy of the sensor, resulting in unreliable monitoring results; traditional methods often find it difficult to accurately identify complex driver conditions, such as fatigue, distraction, emotions, etc., and require more dimensional and accurate information. Summary of the invention

[0003] This application provides a driver status point cloud detection method based on millimeter-wave radar, aiming to solve the technical problems that traditional driver status monitoring methods often rely on vision or sensor data, and have limited vision, large environmental interference, and difficulty in identifying complex scenes.

[0004] In view of the above problems, the present application provides a driver status point cloud detection method based on millimeter wave radar.

[0005] The first aspect disclosed in the present application provides a driver status point cloud detection method based on millimeter wave radar, the method comprising: obtaining a target data matrix, the target data matrix being composed of target vital sign signals of a target driver dynamically measured by a multi-transmitter and multi-receiver MIMO-FMCW millimeter wave radar based on a predetermined measurement distance; determining a distance unit in combination with the position of the target part and the radar distance resolution of the multi-transmitter and multi-receiver MIMO-FMCW millimeter wave radar, and performing Fourier transform on the target data matrix based on the distance unit to obtain a target motion three-dimensional point cloud; analyzing the target motion three-dimensional point cloud and constructing a target space map; performing latent space normal estimation on the target space map between consecutive frames to obtain a target continuous frame point cloud; and evaluating the status of the target driver based on a target dense point cloud obtained by performing multi-frame fusion processing on the target continuous frame point cloud.

[0006] The second aspect disclosed in the present application provides a driver status point cloud detection system based on millimeter wave radar, and the system is used for the above-mentioned driver status point cloud detection method based on millimeter wave radar, and the system includes: a target data matrix acquisition module, used to acquire a target data matrix, and the target data matrix is ​​composed of target vital sign signals of the target driver dynamically measured by the multi-transmitter and multi-receiver MIMO-FMCW millimeter wave radar based on a predetermined measurement distance; a Fourier transform module, used to determine the distance unit in combination with the position of the target part and the radar distance resolution of the multi-transmitter and multi-receiver MIMO-FMCW millimeter wave radar, and perform Fourier transform on the target data matrix based on the distance unit to obtain a target motion three-dimensional point cloud; a target space map construction module, used to analyze the target motion three-dimensional point cloud and construct a target space map; a latent space normal estimation module, used to perform latent space normal estimation on the target space map between consecutive frames to obtain a target continuous frame point cloud; a state evaluation module, used to evaluate the state of the target driver based on the target dense point cloud obtained by multi-frame fusion processing of the target continuous frame point cloud.

[0007] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0008] Through the multi-transmit and multi-receive MIMO-FMCW millimeter wave radar, the target driver's target vital sign signals can be accurately measured to obtain an accurate target data matrix; combined with the radar range resolution and Fourier transform, the target motion three-dimensional point cloud is obtained to achieve a more accurate description of the target position and motion; the target space map is estimated between consecutive frames in the latent space normal, and based on this, the target continuous frame point cloud is obtained, which improves the ability to analyze the continuity of target motion and changes; the target dense point cloud obtained through multi-frame fusion processing improves the ability to analyze and evaluate the target details and status, and further improves the accuracy and comprehensiveness of the driver status assessment. In summary, the driver status point cloud detection method based on millimeter wave radar solves the problems of traditional methods in limited vision, environmental interference, and difficult recognition in complex scenes, and improves the accuracy and reliability of monitoring and evaluation of the target driver status.

[0009] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 A schematic diagram of a process flow of a driver status point cloud detection method based on millimeter wave radar provided in an embodiment of the present application;

[0011] Figure 2A schematic diagram of the structure of a driver status point cloud detection system based on millimeter-wave radar provided in an embodiment of the present application.

[0012] Explanation of the accompanying drawings: target data matrix acquisition module 10, Fourier transform module 20, target space map construction module 30, latent space normal estimation module 40, state assessment module 50. DETAILED DESCRIPTION

[0013] The embodiments of the present application provide a driver status point cloud detection method based on millimeter-wave radar, which solves the technical problems that traditional driver status monitoring methods often rely on vision or sensor data, have limited vision, large environmental interference, and difficulty in identifying complex scenes.

[0014] After introducing the basic principles of the present application, various non-limiting implementation methods of the present application will be specifically introduced below in conjunction with the accompanying drawings of the specification. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0015] like Figure 1 As shown, the embodiment of the present application provides a driver status point cloud detection method based on millimeter wave radar, and the method includes:

[0016] A target data matrix is ​​obtained, where the target data matrix is ​​composed of target vital sign signals of the target driver dynamically measured by the multi-transmit multi-receive MIMO-FMCW millimeter wave radar based on a predetermined measurement distance.

[0017] MIMO-FMCW millimeter wave radar is a frequency modulated continuous wave radar technology that measures distance by changing the frequency of the transmitted signal, and can send and receive multiple signals at the same time, increasing the sensitivity and accuracy of the system. Set the radar's predetermined measurement distance, which refers to the distance between the target and the radar, for example, set the subject (driver) to sit about 0.3m in front of the radar.

[0018] The radar sends millimeter wave signals and receives signals returned by the target. These signals are processed by the radar system to extract the target's vital signs, such as breathing and heartbeat. Dynamic measurement means that the radar can continuously collect changes in the target's vital signs in real time. The dynamically measured target vital signs signals are organized into a data matrix according to the time series. For example, each row of the data matrix represents a time point, and each column represents the signal strength of a vital sign signal, which is used to describe the changes of the target's vital signs over time.

[0019] The distance unit is determined in combination with the position of the target part and the radar distance resolution of the multi-transmit multi-receive MIMO-FMCW millimeter wave radar, and the target data matrix is ​​Fourier transformed based on the distance unit to obtain a three-dimensional point cloud of the target motion.

[0020] The location information of the target part is determined by the relative position between the radar position and the target. The radar range resolution of MIMO-FMCW millimeter wave radar refers to the smallest unit that the radar system can distinguish in distance. Combining the location information of the target and the range resolution of the radar, the range unit of the target in the radar system is determined. The range unit can be understood as the smallest interval divided by the radar in distance.

[0021] Performing Fourier transform on the target data matrix can convert the time domain data into the frequency domain. In radar applications, Fourier transform is used to extract the motion characteristics of the target, such as velocity, acceleration, etc. The result of Fourier transform is expressed as the distribution of the target motion in terms of position-velocity-signal intensity, forming a three-dimensional point cloud of the target motion.

[0022] The target motion three-dimensional point cloud is analyzed and a target space map is constructed.

[0023] Furthermore, analyzing the target motion three-dimensional point cloud and constructing a target space map includes:

[0024] The target motion three-dimensional point cloud includes N point clouds; preprocessing the first point cloud to obtain a first preprocessed point cloud, wherein the first point cloud refers to any one of the N point clouds; performing feature fusion embedding on the first preprocessed point cloud through an embedding space to obtain a first target point cloud, wherein the first target point cloud has an identification of a first target attribute; obtaining a target three-dimensional space map according to a target node feature matrix constructed based on the first target point cloud;

[0025] A predetermined edge connection strategy is read, and the predetermined edge connection strategy is combined with the first target attribute to construct the target three-dimensional space graph by edge connection to obtain the target space graph.

[0026] The target motion three-dimensional point cloud includes N point clouds, where N is the number of point clouds, and N point clouds are defined as a set ,in , Represents the three-dimensional spatial position characteristics, Represents the speed characteristics, Indicates the signal strength characteristics.

[0027] The inner product between adjacent nodes is used to calculate their feature similarity, that is, one point cloud is randomly selected from N point clouds as the first point cloud, and the adjacent point cloud of the first point cloud is obtained as the second point cloud, and the first inner product of the second point cloud and the first point cloud is calculated. When the first inner product reaches a predetermined inner product threshold, the first point cloud and the second point cloud are node-fused to obtain the first preprocessed point cloud.

[0028] Align feature representations in embedding space using two fully connected layers and location attributes , and then fuse them to obtain the attribute embedding of the node, and obtain the first target attribute of the first preprocessed point cloud, which is the result of feature fusion. At this point, the comprehensive representation of the node features can be completed and the first target point cloud can be obtained.

[0029] A target node feature matrix is ​​constructed based on the first target point cloud. The target node feature matrix refers to converting the point cloud into a node containing spatial features by combining the spatial position, speed information, and signal strength of the 4D point cloud.

[0030] Then, the target three-dimensional space map can be constructed ,by For a node, connect the node to other nodes in the radius neighbor area, Represents the connection relationship between neighbor nodes. Given an edge set ,After obtaining node features, the establishment of edges is based on the affinity of node attributes.,The correlation between two node attributes is defined as ,for , each node selects ( ) neighbor node set and through the edge connected, if the adjacency matrix ,Right now and There is a dependency relationship between them. The correlation calculation process is as follows:

[0031] ;

[0032] in, and Represents two linear transformation functions and can be expressed as , and are the learnable parameters in the two transformation functions, and the adjacency matrix is ​​obtained by multiplying the two matrices Finally, for each row in the matrix This method allows adaptive selection and establishment of topological dependencies, where the edges describe the spatial geometric associations within the same frame. After completing the construction of nodes and edges, the established adaptive spatial graph topology is obtained, namely the target spatial graph.

[0033] Furthermore, preprocessing the first point cloud to obtain a first preprocessed point cloud includes:

[0034] Acquire a second point cloud, and calculate a first inner product between the second point cloud and the first point cloud, where the second point cloud is a point cloud adjacent to the first point cloud among the N point clouds; when the first inner product reaches a predetermined inner product threshold, perform node fusion on the first point cloud and the second point cloud to obtain the first preprocessed point cloud.

[0035] From the given N point clouds, a point cloud adjacent to the first point cloud is selected as the second point cloud, and the first inner product of the second point cloud and the first point cloud is calculated. The first inner product is the inner product operation between two points in the point cloud, which is used to measure the similarity between them.

[0036] A predetermined inner product threshold is set. This threshold can be determined according to the application scenario and specific needs to measure the similarity between the two point clouds. The first inner product is compared with the predetermined inner product threshold. If the first inner product reaches the predetermined inner product threshold, it means that the first point cloud and the second point cloud have sufficient similarity. In this case, the first point cloud and the second point cloud are subjected to node fusion operation. Node fusion can be understood as merging two adjacent point clouds. It is a process of data dimensionality reduction to reduce the complexity and amount of calculation of the data while retaining important information. Node fusion includes operations such as feature merging and point cloud splicing, and finally the first preprocessed point cloud is obtained.

[0037] Specifically, the process of obtaining the first target attribute is as follows:

[0038] ;

[0039] in, is the first target attribute of the first preprocessed point cloud, , and are learnable weights, [.] represents the concatenation operation. is the LeakyReLU activation function, , and Is a collection The first pre-processed point cloud corresponds to the three-dimensional spatial position feature, speed feature and signal strength feature of the point cloud, wherein the set is used to characterize the N point clouds, and , Represents the three-dimensional spatial position characteristics, Represents the speed characteristics, Indicates the signal strength characteristics.

[0040] Specifically, the process of obtaining the first target attribute is as follows:

[0041] ;

[0042] Among them, the collection Includes N point clouds, each point cloud It consists of three parts: 3D spatial position features , speed characteristics and signal strength characteristics , the point cloud corresponding to the first preprocessed point cloud is , the corresponding feature is , and , , and are learnable weights that adjust the contribution of each feature to the final target attribute The influence of each feature is LeakyReLU activation function This activation function is used to introduce nonlinearity to help the network better learn complex feature relationships. The formula concatenates the three features after weight adjustment and activation function processing to form the final first target attribute.

[0043] Perform latent space normal estimation between consecutive frames on the target space map to obtain a point cloud of consecutive frames of the target.

[0044] Furthermore, performing latent space normal estimation between consecutive frames on the target space map to obtain a point cloud of consecutive frames of the target includes:

[0045] Extract target image pairs in the target space graph, wherein the target image pairs are adjacent front and rear frame images in the target space graph; perform latent space normal estimation on the target image pairs to obtain target posture correlation, and construct a target space-time graph of the target space graph based on the target posture correlation; read a predetermined space-time edge decision, and perform edge optimization processing on the target space-time graph based on the predetermined space-time edge decision to obtain the target continuous frame point cloud.

[0046] In the target space graph, adjacent front and back frame images are extracted as a pair of target image pairs, and the image pair is represented as , corresponding to the current frame and the previous frame respectively, where Total number of acquired frames represents the image acquisition moment, and I represents the image frame.

[0047] Latent space normal estimation is a technique for estimating the relationship between consecutive frame images. Corresponding to a normal estimate of the latent space , It represents the posture correlation between two consecutive frames. By performing latent space normal estimation, the target posture correlation is obtained, that is, the posture change relationship of the target in time.

[0048] According to the target posture correlation, the target space-time graph of the target space graph is constructed. The space-time graph adds the time dimension on the basis of the space graph to represent the changes of the target in time and space. Specifically, the space graph structure is established. Finally, the dynamic image is unwrapped by optimizing the connection edges and converted into a static image to better characterize the spatial domain features, thereby constructing a spatiotemporal image that can describe the nonlinear anatomical differences of the 4D point cloud before and after different frames. Specifically, for the spatial graphs before and after adjacent frames ,use calculate and Two-by-two nodes The similarity of Once the space-time graph structure is established After that, the time domain features can be characterized by adding spatiotemporal connecting edges. In general, the spatiotemporal edges It can be constructed by the similarity of two node features.

[0049] For each In ,from Select the most similar nodes , the association between visually similar and semantically aligned nodes is established through spatiotemporal edges, and the relationship between the same structure in different frames is modeled. The process of calculating semantic similarity is as follows:

[0050] ;

[0051] ;

[0052]

[0053] in, Represents node characteristics, is a learnable parameter, is the calculated feature. In the process of millimeter wave radar, two key points are worth paying attention to. First, since the spatial point cloud obtained by the millimeter wave radar is random, the shape and position have changed. The connections that only exist in the current graph structure will not be able to effectively represent the potential features of the point cloud at different times. In the process, this method also considers the situation of re-establishing and eliminating node connections. If a certain area of ​​the 4D point cloud of the current and subsequent frames is in a point cloud dense state and the node features of the area are highly similar, the spatiotemporal edges will be processed by adding edge connections.

[0054] Furthermore, performing edge optimization processing on the target spatiotemporal graph based on the predetermined spatiotemporal edge decision to obtain the target continuous frame point cloud includes:

[0055] Obtain a first spatiotemporal edge similarity in the target spatiotemporal graph, wherein the first spatiotemporal edge similarity refers to an attribute similarity between the first target attribute and the second target attribute in the target spatiotemporal graph; when the first spatiotemporal edge similarity reaches a predetermined similarity threshold, issue a first add edge instruction; based on the first add edge instruction, activate the predetermined spatiotemporal edge decision to perform edge optimization processing on the target spatiotemporal graph to obtain the target continuous frame point cloud.

[0056] The similarity between the first target attribute and the second target attribute in the target space-time graph is measured. The similarity can be calculated based on attribute characteristics, motion patterns, spatial positions and other factors to obtain the first space-time edge similarity to reflect the attribute relevance of the target.

[0057] When the first spatiotemporal edge similarity reaches a predetermined similarity threshold, it indicates that there is a certain similarity between the targets and edges need to be added to optimize the representation of the spatiotemporal graph. In this case, a first edge addition instruction is issued.

[0058] Based on the first add edge instruction, the predetermined spatiotemporal edge decision is activated. The spatiotemporal edge decision can include operations such as adding new edges and adjusting edge weights to optimize the structure of the spatiotemporal graph. The purpose of edge optimization processing is to ensure that the spatiotemporal graph more accurately reflects the movement and changes of the target, improve the quality of continuous frame point cloud data, and ultimately obtain optimized target continuous frame point cloud data.

[0059] The state of the target driver is evaluated according to the target dense point cloud obtained by performing multi-frame fusion processing on the target continuous frame point cloud.

[0060] Furthermore, the state of the target driver is evaluated according to the target dense point cloud obtained by multi-frame fusion processing of the target continuous frame point cloud, including:

[0061] Obtain a predetermined sparse adjacency matrix; perform iterative pose tuning on a first frame point cloud in the target continuous frame point cloud using the predetermined sparse adjacency matrix to obtain a first tuning result; when the value of the predetermined sparse adjacency matrix obtained based on the first tuning result is 0, obtain the target dense point cloud.

[0062] The ultimate goal of this method is to use the multi-frame fusion method to enhance the point cloud data under the premise of ensuring confidence. Therefore, after obtaining the latent space normal estimation of each frame image, it is necessary to perform multi-frame fusion and point cloud splicing based on the estimation results. The latent space normal estimation of the image only evaluates the normal estimation between the current frame and the previous frame, and for multi-frame images, it is necessary to perform global discreteness analysis in the form of a posture matrix to ensure the smoothness of the overall fusion.

[0063] Specifically, we first create a sparse adjacency matrix , which stores the latent space estimation results and the key vertices of each frame of the point cloud. In each iterative tuning process, the discreteness between the current frame and the overall point cloud will change by fine-tuning the single-frame pose, thereby updating the weights of the matrix. In the matrix, a negative number is taken as the initial value of the relative adjacency matrix, the connection relationship between vertices in different frames is represented as a positive number, and the number of edges is represented as a positive number as the relative adjacency matrix. When the distance between two points is less than the threshold, the edge is no longer established. By gradually iterating the number of adjacent edges connecting the points, the value in the matrix eventually approaches , at this time, the target dense point cloud is obtained.

[0064] Relevant features are extracted from the target dense point cloud, such as the target's position, speed, acceleration, motion trajectory, shape features, etc. Based on the extracted features, the target driver's status is modeled, such as the classification of driving behavior (such as normal driving, fatigue driving, abnormal driving, etc.), the driver's emotional state (such as anxiety, excitement, calmness, etc.), physical condition assessment (such as health status, fatigue level, etc.), etc. According to the assessment results, the assessment results of the target driver's status are output, such as normal / abnormal, fatigue level, etc.

[0065] In summary, the driver status point cloud detection method based on millimeter wave radar provided in the embodiment of the present application has the following technical effects:

[0066] 1. Through the multi-transmit and multi-receive MIMO-FMCW millimeter wave radar, the target vital sign signals of the target driver can be accurately measured to obtain an accurate target data matrix;

[0067] 2. Combining radar range resolution and Fourier transform, a three-dimensional point cloud of target motion is obtained to achieve a more accurate description of target position and motion;

[0068] 3. Perform latent space normal estimation between consecutive frames of the target space map, and obtain the target continuous frame point cloud based on this, which improves the continuity analysis capability of target motion and change;

[0069] 4. The dense point cloud of the target obtained through multi-frame fusion processing improves the ability to analyze and evaluate the target details and status, and further improves the accuracy and comprehensiveness of the driver status assessment.

[0070] In summary, the driver status point cloud detection method based on millimeter-wave radar solves the problems of traditional methods in limited vision, environmental interference, and difficulty in recognition under complex scenes, and improves the accuracy and reliability of monitoring and evaluation of the target driver status.

[0071] Based on the same inventive concept as the driver status point cloud detection method based on millimeter wave radar in the aforementioned embodiment, Figure 2 As shown, the present application provides a driver status point cloud detection system based on millimeter wave radar, the system comprising:

[0072] A target data matrix acquisition module 10 is used to acquire a target data matrix, wherein the target data matrix is ​​composed of target vital sign signals of the target driver dynamically measured by the multi-transmit multi-receive MIMO-FMCW millimeter wave radar based on a predetermined measurement distance;

[0073] A Fourier transform module 20 is used to determine a distance unit based on the position of the target part and the radar distance resolution of the MIMO-FMCW millimeter wave radar, and perform Fourier transform on the target data matrix based on the distance unit to obtain a three-dimensional point cloud of the target motion;

[0074] A target space map construction module 30, used for analyzing the target motion three-dimensional point cloud and constructing a target space map;

[0075] A latent space normal estimation module 40 is used to perform latent space normal estimation between consecutive frames of the target space map to obtain a target consecutive frame point cloud;

[0076] The state evaluation module 50 is used to evaluate the state of the target driver according to the target dense point cloud obtained by performing multi-frame fusion processing on the target continuous frame point cloud.

[0077] Furthermore, the system further includes a target space map acquisition module to perform the following operation steps:

[0078] The target motion three-dimensional point cloud includes N point clouds;

[0079] Preprocessing the first point cloud to obtain a first preprocessed point cloud, wherein the first point cloud refers to any one point cloud among the N point clouds;

[0080] Performing feature fusion embedding on the first preprocessed point cloud through an embedding space to obtain a first target point cloud, wherein the first target point cloud has an identification of a first target attribute;

[0081] Obtaining a target three-dimensional space graph according to a target node feature matrix constructed based on the first target point cloud;

[0082] A predetermined edge connection strategy is read, and the predetermined edge connection strategy is combined with the first target attribute to construct the target three-dimensional space graph by edge connection to obtain the target space graph.

[0083] Furthermore, the system further includes a first pre-processing point cloud acquisition module to perform the following operation steps:

[0084] Acquire a second point cloud, and calculate a first inner product between the second point cloud and the first point cloud, where the second point cloud is a point cloud adjacent to the first point cloud among the N point clouds;

[0085] When the first inner product reaches a predetermined inner product threshold, node fusion is performed on the first point cloud and the second point cloud to obtain the first preprocessed point cloud.

[0086] Specifically, the process of obtaining the first target attribute is as follows:

[0087] ;

[0088] in, is the first target attribute of the first preprocessed point cloud, , and are learnable weights, [.] represents the concatenation operation. is the LeakyReLU activation function, , and Is a collection The first pre-processed point cloud corresponds to the three-dimensional spatial position feature, speed feature and signal strength feature of the point cloud, wherein the set is used to characterize the N point clouds, and , Represents the three-dimensional spatial position characteristics, Represents the speed characteristics, Indicates the signal strength characteristics.

[0089] Furthermore, the system further includes a target continuous frame point cloud acquisition module to perform the following operation steps:

[0090] Extracting a target image pair from the target space map, wherein the target image pair is adjacent front and back frame images in the target space map;

[0091] Performing latent space normal estimation on the target image pair to obtain target posture correlation, and constructing a target spatiotemporal graph of the target space graph according to the target posture correlation;

[0092] The predetermined spatiotemporal edge decision is read, and the target spatiotemporal graph is edge optimized based on the predetermined spatiotemporal edge decision to obtain the target continuous frame point cloud.

[0093] Furthermore, the system further includes a target continuous frame point cloud acquisition module to perform the following operation steps:

[0094] Obtain a first spatiotemporal edge similarity in the target spatiotemporal graph, where the first spatiotemporal edge similarity refers to an attribute similarity between the first target attribute and the second target attribute in the target spatiotemporal graph;

[0095] When the first spatiotemporal edge similarity reaches a predetermined similarity threshold, issuing a first edge adding instruction;

[0096] Based on the first add edge instruction, the predetermined spatiotemporal edge decision is activated to perform edge optimization processing on the target spatiotemporal graph to obtain the target continuous frame point cloud.

[0097] Furthermore, the system further includes a target dense point cloud acquisition module to perform the following operation steps:

[0098] Obtain a predetermined sparse adjacency matrix;

[0099] Iteratively optimizing the pose of a first frame point cloud in the target continuous frame point cloud by using the predetermined sparse adjacency matrix to obtain a first optimization result;

[0100] When the value of the predetermined sparse adjacency matrix obtained based on the first tuning result is 0, the target dense point cloud is obtained.

[0101] Through the above-mentioned detailed description of the driver status point cloud detection method based on millimeter-wave radar in this specification, those skilled in the art can clearly understand the driver status point cloud detection system based on millimeter-wave radar in this embodiment. Since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0102] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A driver status point cloud detection method based on millimeter wave radar, characterized in that: The method comprises: Acquire a target data matrix, wherein the target data matrix is ​​composed of target vital sign signals of the target driver dynamically measured by a multi-transmit multi-receive MIMO-FMCW millimeter wave radar based on a predetermined measurement distance; Determine a distance unit in combination with the position of the target part and the radar distance resolution of the multi-transmit multi-receive MIMO-FMCW millimeter wave radar, and perform Fourier transform on the target data matrix based on the distance unit to obtain a three-dimensional point cloud of the target motion; Analyzing the target motion three-dimensional point cloud and constructing a target space map; Performing latent space normal estimation between consecutive frames on the target space map to obtain a point cloud of consecutive frames of the target; Performing a state assessment of the target driver according to a target dense point cloud obtained by performing multi-frame fusion processing on the target continuous frame point cloud; Analyzing the target motion three-dimensional point cloud and constructing a target space map, including: The target motion three-dimensional point cloud includes N point clouds; Preprocessing the first point cloud to obtain a first preprocessed point cloud, wherein the first point cloud refers to any one point cloud among the N point clouds; Performing feature fusion embedding on the first preprocessed point cloud through an embedding space to obtain a first target point cloud, wherein the first target point cloud has an identification of a first target attribute; Obtaining a target three-dimensional space graph according to a target node feature matrix constructed based on the first target point cloud; A predetermined edge connection strategy is read, and the predetermined edge connection strategy is combined with the first target attribute to construct the target three-dimensional space graph by edge connection to obtain the target space graph.

2. The driver status point cloud detection method based on millimeter wave radar according to claim 1 is characterized in that: Preprocessing the first point cloud to obtain a first preprocessed point cloud includes: Acquire a second point cloud, and calculate a first inner product between the second point cloud and the first point cloud, where the second point cloud is a point cloud adjacent to the first point cloud among the N point clouds; When the first inner product reaches a predetermined inner product threshold, node fusion is performed on the first point cloud and the second point cloud to obtain the first preprocessed point cloud.

3. The driver status point cloud detection method based on millimeter wave radar according to claim 1 is characterized in that: The process of obtaining the first target attribute is as follows: ； in, is the first target attribute of the first preprocessed point cloud, , and are learnable weights, Represents the concatenation operation, is the LeakyReLU activation function, , and Is a collection The first pre-processed point cloud corresponds to the three-dimensional spatial position feature, speed feature and signal strength feature of the point cloud, wherein the set is used to characterize the N point clouds, and , Represents the three-dimensional spatial position characteristics, Represents the speed characteristics, Indicates the signal strength characteristics.

4. The driver status point cloud detection method based on millimeter wave radar according to claim 1 is characterized in that: Performing hidden space normal estimation between consecutive frames on the target space map to obtain a point cloud of consecutive frames of the target, including: Extracting a target image pair from the target space map, wherein the target image pair is adjacent front and back frame images in the target space map; Performing latent space normal estimation on the target image pair to obtain target posture correlation, and constructing a target spatiotemporal graph of the target space graph according to the target posture correlation; The predetermined spatiotemporal edge decision is read, and the target spatiotemporal graph is edge optimized based on the predetermined spatiotemporal edge decision to obtain the target continuous frame point cloud.

5. The driver status point cloud detection method based on millimeter wave radar according to claim 4 is characterized in that: The target spatiotemporal graph is edge-optimized based on the predetermined spatiotemporal edge decision to obtain the target continuous frame point cloud, including: Obtain a first spatiotemporal edge similarity in the target spatiotemporal graph, where the first spatiotemporal edge similarity refers to an attribute similarity between the first target attribute and the second target attribute in the target spatiotemporal graph; When the first spatiotemporal edge similarity reaches a predetermined similarity threshold, issuing a first edge adding instruction; Based on the first add edge instruction, the predetermined spatiotemporal edge decision is activated to perform edge optimization processing on the target spatiotemporal graph to obtain the target continuous frame point cloud.

6. The driver status point cloud detection method based on millimeter wave radar according to claim 1 is characterized in that: The state of the target driver is evaluated according to the target dense point cloud obtained by multi-frame fusion processing of the target continuous frame point cloud, including: Obtain a predetermined sparse adjacency matrix; Iteratively optimizing the pose of a first frame point cloud in the target continuous frame point cloud by using the predetermined sparse adjacency matrix to obtain a first optimization result; When the value of the predetermined sparse adjacency matrix obtained based on the first tuning result is 0, the target dense point cloud is obtained.

7. The driver status point cloud detection system based on millimeter wave radar is characterized by: The system for implementing the driver status point cloud detection method based on millimeter wave radar according to any one of claims 1 to 6 comprises: A target data matrix acquisition module, used to acquire a target data matrix, wherein the target data matrix is ​​composed of target vital sign signals of the target driver dynamically measured by the multi-transmit multi-receive MIMO-FMCW millimeter wave radar based on a predetermined measurement distance; A Fourier transform module is used to determine a distance unit based on the position of the target part and the radar distance resolution of the MIMO-FMCW millimeter wave radar, and to perform Fourier transform on the target data matrix based on the distance unit to obtain a three-dimensional point cloud of the target motion; A target space map construction module, used for analyzing the target motion three-dimensional point cloud and constructing a target space map; A latent space normal estimation module is used to perform latent space normal estimation between consecutive frames of the target space map to obtain a point cloud of consecutive frames of the target; The state evaluation module is used to evaluate the state of the target driver according to the target dense point cloud obtained by performing multi-frame fusion processing on the target continuous frame point cloud.

Images