Concentration degree detection method based on millimeter wave radar, program product and electronic equipment

Abstract

The invention discloses a concentration degree detection method based on a millimeter wave radar, a program product and electronic equipment, and the method comprises the steps: controlling a millimeter wave radar sensor to transmit an electromagnetic wave signal to a target space region, and receiving an echo signal reflected by the target space region; judging whether a human body exists in the target space area or not according to the echo signal; when determining that a human body exists in the target space area according to the echo signal, extracting concentration degree detection data of the human body based on the echo signal; and calculating current concentration degree data of the human body in the current time period based on the concentration degree detection data.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a concentration detection method based on millimeter wave radar, a computer program product and an electronic device. Background Art

[0002] In an educational environment, students' concentration directly affects learning outcomes. By testing concentration, teachers can understand students' learning status and take corresponding measures to improve teaching quality. Therefore, by monitoring users' concentration in real time during their learning process, users can be reminded in real time when they are not focused, helping them improve their learning efficiency.

[0003] In the existing technology, the user's concentration can be detected through brain wave signals, but the user needs to wear equipment to cooperate with the detection of concentration, which requires additional cost of wearing equipment, and also requires the user to actively cooperate with wearing equipment, but some users may not cooperate with wearing, so real-time monitoring is impossible. Others collect the user's image through a camera, and analyze the user's facial expressions and body movements through image data to judge the user's concentration. However, due to the large number of background factors that interfere with the image, the recognition results are prone to contradictions or unstable fluctuations in the previous and next frames, affecting the accuracy of the final determination of concentration. Summary of the invention

[0004] In order to solve the existing technical problems, the present invention provides a concentration detection method based on millimeter wave radar, a computer program product and an electronic device, which can improve the accuracy of concentration detection.

[0005] In a first aspect, a concentration detection method based on millimeter-wave radar is provided, comprising: controlling the millimeter-wave radar sensor to send an electromagnetic wave signal to a target space area, and receiving an echo signal reflected by the target space area; judging whether there is a human body in the target space area according to the echo signal; when it is determined that there is a human body in the target space area according to the echo signal, extracting concentration detection data of the human body based on the echo signal; and calculating current concentration data of the human body in a current time period based on the concentration detection data.

[0006] In a second aspect, a computer program product is provided, including a computer program, which, when executed by a processor, implements the concentration detection method based on millimeter-wave radar as described in any embodiment of the present application.

[0007] In a third aspect, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the concentration detection method based on millimeter-wave radar as described in any embodiment of the present application.

[0008] The embodiment of the present application utilizes a millimeter-wave radar sensor to transmit an electromagnetic wave signal to a target space area and receive a reflected echo signal. It first determines whether there is a person based on the echo signal, and detects whether there is a person in the target space area. If there is a person, it continues to extract concentration detection data based on the echo signal, and monitors the current concentration data of the human body in real time based on the concentration detection data. This does not require users to wear special equipment, which can save costs, and the concentration detection data can be collected unconsciously. The concentration detection data collected by the millimeter-wave radar sensor can be of multiple types, and the current concentration can be obtained based on a combination of multiple data, which can improve the accuracy of concentration detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 This is a diagram of an application environment of a concentration detection method based on millimeter-wave radar in an embodiment;

[0010] Figure 2 is a schematic diagram of an electronic device in an embodiment being a smart desk lamp;

[0011] Figure 3 This is a diagram of an application environment of a concentration detection method based on millimeter-wave radar in another embodiment;

[0012] Figure 4 is a flow chart of a concentration detection method based on millimeter wave radar in one embodiment;

[0013] Figure 5 is a flow chart of a concentration detection method based on millimeter wave radar in another embodiment;

[0014] Figure 6 is a schematic diagram of a concentration detection device based on millimeter wave radar in one embodiment;

[0015] Figure 7 FIG. 1 is a schematic diagram of an electronic device in an embodiment. DETAILED DESCRIPTION

[0016] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0017] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art of the technical field of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the scope of protection of the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0018] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it should be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0019] See also Figure 1 , is an application environment diagram of a concentration detection method based on millimeter wave radar in an embodiment. The concentration detection method based on millimeter wave radar is applied to an electronic device 10, and the electronic device 10 includes a millimeter wave radar sensor 12 and a processor 13. The millimeter wave radar sensor 12 is used to send electromagnetic wave signals to the target space area. Since there are multiple targets such as human bodies and other objects in the target space area, these multiple targets will reflect electromagnetic wave signals, and the millimeter wave radar sensor 12 can receive echo signals reflected by multiple targets. The processor 13 detects the concentration of the human body in front of the electronic device according to the echo signals detected by the millimeter wave radar sensor 12.

[0020] The millimeter wave radar sensor 12 is a radar system that uses electromagnetic waves in the millimeter wave band to detect targets. The wavelength of the millimeter wave ranges between 1 mm and 10 mm, which is between microwaves and terahertz waves. The millimeter wave radar sensor acquires information about the target by emitting millimeter wave signals and receiving signals reflected by the target. The target information includes but is not limited to the target's body movement data, the target's breathing data and heart rate data in its vital signs.

[0021] The processor 13 may be one or more processors. When there are multiple processors 13, the multiple processors may be integrated on one chip or independently arranged on each chip. The electronic device 10 is a device equipped with a millimeter wave radar sensor, for example, it may include a computing device (for example, a desktop computer, a laptop computer, a tablet computer, a handheld computer, a smart speaker, a server, etc.), a terminal device (for example, a mobile phone, etc.), a wearable device (for example, a pair of smart glasses or a smart watch), various infrared imaging devices, a smart desk lamp or similar devices.

[0022] like Figure 2 As shown, Figure 2Schematic diagram of an electronic device in an embodiment; the electronic device 10 is a smart desk lamp, in which a millimeter wave radar sensor 12 is provided. The smart desk lamp can be placed in the area in front of the user, and can detect the user's concentration while providing the user with a lighting function, so that the user's concentration can be unconsciously monitored without adding wearable devices. When the smart desk lamp is in the on state, that is, when providing the lighting function, the millimeter wave radar sensor 12 sends an electromagnetic wave signal to the target space area corresponding to its installation position, and according to the echo signal reflected by the human body, the body movement data, breathing data and heart rate data of the human body can be obtained, and the user's concentration can be automatically detected based on the acquired data.

[0023] like Figure 3 As shown, it is an application environment diagram of the concentration detection method based on millimeter wave radar in another embodiment. The electronic device 10 can also be connected to the terminal device 20 for communication, and the electronic device 10 can send the detected user's concentration to the terminal device 20. For example, when a child is doing homework, he turns on the smart desk lamp and uses it. The smart desk lamp automatically detects the child's concentration during the homework process, and sends the report to the parent's terminal device 20 in the form of a report based on the detected concentration data. In this way, parents do not need to keep an eye on their children all the time, and can understand their children's learning situation in real time.

[0024] See also Figure 4 , is a flow chart of a concentration detection method based on millimeter wave radar provided in an embodiment of the present application. The concentration detection method based on millimeter wave radar is applied to electronic devices, and the concentration detection method based on millimeter wave radar includes the following steps:

[0025] S11. Control the millimeter-wave radar sensor to send an electromagnetic wave signal to the target space area, and receive an echo signal reflected by the target space area.

[0026] In this embodiment, the target space area indicates the area where the signal of the millimeter wave radar sensor can be transmitted. Due to the installation position of the millimeter wave radar sensor in the electronic device, the target space area is related to the installation position and the parameters of the millimeter wave radar sensor itself. When the electronic device is turned on, the millimeter wave radar sensor is controlled to send electromagnetic wave signals.

[0027] S12. Determine whether there is a human body in the target space area according to the echo signal.

[0028] In this embodiment, the echo signal is first preprocessed to obtain a preprocessed signal, wherein the preprocessing operation includes but is not limited to static clutter elimination, a processing operation based on a mixer, a sampling operation based on an analog-to-digital converter, a transformation operation based on a fast-time Fourier transform, and the like. The static clutter elimination is performed on the echo signal, and the static clutter elimination process can be performed to reduce the influence of clutter signals generated by large static objects indoors on the target echo signal. The transmission signal is mixed with the echo signal based on the processing operation of the mixer to obtain a signal with a new frequency, which is an intermediate frequency signal. The frequency and phase of the echo signal and the transmission signal are different, so data such as distance, speed and angle can be extracted and analyzed from the superimposed intermediate frequency signal. The sampling operation based on the analog-to-digital converter is used to sample the processed echo signal, and the sampled signal is Fourier transformed to obtain a discrete echo signal x(m,n) containing distance dimension and slow time dimension information, wherein m represents the slow time dimension, which is the mth pulse echo, and n represents the distance dimension, which is the nth distance unit.

[0029] In some embodiments, it is determined whether there is a human body in the target space area based on the preprocessed signal. Since the radar continuously sends a transmission signal, if a preset number of point cloud data is continuously obtained from the preprocessed signal within a preset time period, it means that there is a person in the detection scene. In other embodiments, the preprocessed signal can also be input into a human classifier, and the human classifier is used to identify whether there is a human body in the preprocessed signal. If the human classifier outputs that there is a person, it means that there is a human body, otherwise, there is no human body. This application does not impose any limitation on the method of determining whether there is a human body in the target space area based on the echo signal.

[0030] S13. When it is determined according to the echo signal that a human body exists in the target space area, extracting concentration detection data of the human body based on the echo signal.

[0031] In this embodiment, the concentration detection data is used to detect the data required for concentration, including but not limited to body movement data, breathing data and heart rate data, wherein the body movement data is used to represent the information of the movement and posture change of the human body, such as the body posture change data when the human body uses the smart desk lamp. The breathing data is used to indicate the breathing frequency of the human body in a certain period of time, such as the breathing frequency of the human body when the human body uses the smart desk lamp. The heart rate data is used to represent the heart rate data of the human body in a certain period of time, such as the heart rate data of the human body when the human body uses the smart desk lamp.

[0032] In some application scenarios, after a user turns on an electronic device, the user may not appear in the target space area. For example, the user turns on a smart desk lamp but does not sit in front of the smart desk lamp. In order to reduce false detection or reduce resource waste, human concentration detection data is extracted only after a human body is detected in the target space area. When no human body is detected in the target space area, the process returns to continue executing S12.

[0033] In some embodiments, phase data is extracted based on the echo signal, and a phase unwrapping operation is performed on the extracted phase data. By subtracting continuous phase values ​​and performing phase difference operations on the unfolded phase, the heartbeat signal can be enhanced and any phase drift can be eliminated. According to the difference between the heartbeat and respiratory frequencies, the phase value is filtered using a bandpass filter to distinguish them, so that respiratory data and heart rate data can be obtained. Specifically, the phase signal is Fourier transformed, and the corresponding respiratory frequency within N frame times is obtained according to the peak value and its harmonic characteristics. The respiratory frequency within a period of time is recorded, and the respiratory frequency at this time is judged according to different confidence indicators, and the relationship between the respiratory frequency and time is output. After filtering the phase, the purpose here is to reduce the impact of the relative position movement of the human body on the heart rate measurement. Because the measurement of heart rate is based on the distance difference caused by the tiny movement of heart contraction and diastole, and the phase change caused, according to the micro-Doppler principle, when the human body swings greatly, it will affect its accuracy. Here, by segmenting the sample and setting the threshold to determine whether it meets the range of heart rate changes, the vital sign signal is finally obtained.

[0034] In some embodiments, the Doppler frequency shift in the echo signal may be extracted based on the echo signal by using methods such as Fourier transform, thereby acquiring the body motion data of the human body.

[0035] S14. Calculate the current concentration data of the human body in the current time period based on the concentration detection data.

[0036] In this embodiment, the current time period can be a time interval, for example, the time within 1 minute before the current time constitutes a current time period, and the current time period is related to a preset time interval. In this way, the concentration data of the human body is calculated once for each preset time interval. The shorter the time interval, the more times the calculation is performed.

[0037] In the above embodiment, a millimeter-wave radar sensor is used to transmit an electromagnetic wave signal to the target space area, and a reflected echo signal is received. First, whether there is a person is determined based on the echo signal, and whether there is a person in the target space area is detected. If there is a person, concentration detection data is extracted based on the echo signal. The current concentration data of the human body is monitored in real time based on the concentration detection data. In this way, there is no need for users to wear special equipment, which can save costs, and the concentration detection data can be collected unconsciously. The concentration detection data collected by the millimeter-wave radar sensor can be of multiple types. The current concentration can be obtained based on a combination of multiple data, which can improve the accuracy of concentration detection.

[0038] In some embodiments, Figure 5 As shown, Figure 5 : is a flow chart of a concentration detection method based on millimeter wave radar in another embodiment. The concentration detection method based on millimeter wave radar is applied to an electronic device. The concentration detection method based on millimeter wave radar includes the following steps:

[0039] S11. Control the millimeter-wave radar sensor to send an electromagnetic wave signal to a target space region, and receive an echo signal reflected by the target space region.

[0040] S12. Determine whether there is a human body in the target space area according to the echo signal.

[0041] S13. When it is determined according to the echo signal that a human body exists in the target space area, extracting concentration detection data of the human body based on the echo signal.

[0042] S14. Based on the concentration detection data and using a first concentration calculation method, calculate a current concentration value of the human body in a current time period.

[0043] S15. Based on the concentration detection data and using a second concentration calculation method, calculate the current concentration level of the human body in the current time period.

[0044] S16: Determine whether the current concentration value matches the current concentration level.

[0045] When the current concentration value does not match the current concentration level, and the mismatch satisfies a preset condition, S17 is executed; when the current concentration value matches the current concentration level, S18 is executed.

[0046] S17: Sending a prompt message to a terminal device connected to the electronic device to request the terminal device to confirm the concentration in the current time period.

[0047] S18: Using the current concentration value and the current concentration level as the current concentration data.

[0048] In this embodiment, S11-S13 are the same as the steps of the above-described embodiment, and will not be repeated here. The first concentration calculation method is used to calculate the current concentration value in the current time period, and the current concentration of the user is measured by a specific score. The second concentration calculation method is used to calculate the current concentration level in the current time period, and the current concentration of the user is measured by the level. For example, the concentration value can be set at 0 to 100 points, and the higher the score, the higher the concentration. The concentration level can be divided into high, medium and low levels. Of course, more levels can be set for needs. The concentration value can be pre-set to different levels according to the different scores, for example, 0 to 60 is a low level, 60-85 is a medium level, and 85 to 100 is a high level. It can be determined whether the current concentration level matches the current concentration value based on the calculated current concentration value and the pre-set level. If the current concentration level is the same as the configuration level corresponding to the current concentration value, it means that the current concentration level matches the current concentration value. If the configuration level corresponding to the current concentration level is different from the current concentration value, it means that the current concentration level does not match the current concentration value. At this time, a prompt message can be sent to the terminal device to request the terminal device to confirm the concentration in the current time period to reduce the probability of misjudgment.

[0049] In the above embodiment, the current concentration value and the current concentration level are calculated respectively by two different methods, and the calculated current concentration value and current concentration level are matched to determine whether the concentrations calculated by the two different methods are consistent, thereby reducing misjudgment. When the calculated current concentration value does not match the current concentration level, information is sent to the terminal device to allow the user of the terminal device to confirm whether the concentration is accurate, thereby improving the accuracy of concentration detection.

[0050] In some embodiments, the calculating the current concentration value of the human body based on the concentration feature data and using the first concentration calculation method includes:

[0051] Extracting body motion feature data, breathing feature data and heart rate feature data based on the concentration detection data, wherein the body motion feature data is used to indicate data of changes in the posture of a human body in the current time period; the breathing feature data is used to indicate data of fluctuations in breathing in the current time period; and the heart rate feature data is used to indicate heart rate data of a human body in the current time period;

[0052] Based on the body motion characteristic data, the breathing characteristic data and the heart rate characteristic data, the current concentration value is calculated using a concentration calculation formula, wherein the concentration calculation formula is as follows:

[0053]

[0054] Where V represents the current concentration value, α, β and γ are the first weight coefficient, the second weight parameter and the third weight parameter respectively, wherein the first weight coefficient represents the contribution of body motion feature data to the calculation of concentration, the second weight coefficient represents the contribution of heart rate feature data to the calculation of concentration, and the third weight coefficient represents the contribution of breathing feature data to the calculation of concentration, d i represents the body movement value at the i-th second in the current time period, n represents the total number of seconds in the current time period, h(i) represents the heart rate value at the i-th second, N represents the number of breaths in the current time period, x(i) represents the number of breaths at the i-th second, and μ represents the average number of breaths per minute.

[0055] Optionally, the calculating the current concentration value of the human body based on the concentration detection data and using a first concentration calculation method further includes:

[0056] Calculating the first weight coefficient, the second weight parameter and the third weight parameter;

[0057] The calculating of the first weight coefficient, the second weight parameter and the third weight parameter comprises:

[0058] Acquire multiple sets of different sample feature data, and form a feature sample matrix with the multiple sets of sample feature data; each set of sample feature data includes body movement sample feature data, breathing sample feature data and heart rate sample feature data, where Represents the feature data of body motion samples, Represents the characteristic data of heart rate samples, Represents breath sample characteristic data;

[0059] Performing mean processing on the feature sample matrix to obtain a processed matrix, and generating an association matrix based on the processed matrix, where i represents the i-th group and j represents the j-th sample feature;

[0060] Based on the correlation matrix, the grey correlation degree r of each sample feature is calculated. j ,in Among them A tj represents the data in the tth row and jth column of the association matrix, j represents the jth sample feature, and G represents the total number of groups;

[0061] Grey relational degree r based on the characteristics of each sample j , respectively calculate the first weight coefficient, the second weight parameter and the third weight parameter, wherein the first weight coefficient Among them, r1 represents the grey correlation degree corresponding to the body movement sample feature, r2 represents the grey correlation degree corresponding to the heart rate sample feature, and r3 represents the grey correlation degree corresponding to the breathing sample feature.

[0062] In this embodiment, multiple groups of different sample feature data can be understood as multiple different sampling points. Multiple groups of such data are collected for subsequent evaluation of the contribution of each feature data to the calculation of concentration. In this way, the contribution of concentration is not a preset value, but is calculated based on multiple different sampling point data, thereby making the first weight coefficient, the second weight parameter and the third weight parameter more accurate, so that the concentration value calculated by the concentration calculation formula is more accurate.

[0063] In the above embodiment, the current concentration value is calculated by combining the body movement characteristic data, breathing characteristic data and heart rate characteristic data in the current time period and the contribution degree corresponding to each characteristic, which can make the calculation of the concentration value more accurate, and the first weight coefficient, the second weight parameter and the third weight parameter are calculated based on multiple different sampling point data, thereby making the first weight coefficient, the second weight parameter and the third weight parameter more accurate, so that the concentration value calculated by the concentration calculation formula is more accurate.

[0064] In some embodiments, the calculating the current concentration level of the human body in the current time period based on the concentration detection data and using a second concentration calculation method includes:

[0065] forming input data of a pre-trained concentration detection model based on the body motion data, the breathing data and the heart rate data, wherein the pre-trained concentration detection model comprises a first convolutional layer, a second convolutional layer and a third convolutional layer, wherein the first convolutional layer is used to obtain a body motion feature vector from the body motion data, the second convolutional layer is used to obtain a breathing feature vector from the breathing data, and the third convolutional layer is used to obtain a heart rate feature vector from the heart rate data;

[0066] Inputting body motion input data formed by the body motion data into the first convolution layer to obtain the body motion feature vector, inputting breathing input data formed by the breathing data into the second convolution layer to obtain the breathing feature vector, and inputting heart rate input data formed by the heart rate data into the third convolution layer to obtain the heart rate feature vector;

[0067] splicing the body movement feature vector, the breathing feature vector and the heart rate feature vector to obtain a concentration splicing feature vector;

[0068] The concentration concatenated feature vector is input into a concentration classification network layer in a pre-trained concentration detection model, and the current concentration level is output through the concentration classification network layer.

[0069] In this embodiment, the pre-trained concentration detection model is obtained by training with a concentration sample data set. During the training process, the initial concentration detection model can learn the features in the sample based on the concentration sample data set. After the training is completed, the concentration detection model can accurately obtain the current concentration level based on the concentration detection data.

[0070] The pre-trained concentration detection model includes three different convolutional layers, each of which can be composed of one or more convolution kernels. These three different convolutional layers are used to extract body motion feature vectors, breathing feature vectors, and heart rate feature vectors from the concentration detection data, and then concatenate these three feature vectors. Finally, the concatenated feature vectors are input into the concentration classification network layer to output the current concentration level.

[0071] It is understandable that the concentration detection model can also include normalization processing, pooling layer, fully connected layer, etc. The normalization operation avoids a sudden increase or decrease in the value during the processing process, and the pooling layer is used to downsample the output of the convolution layer to reduce the data dimension, reduce the model complexity and the amount of calculation. The fully connected layer can expand the output obtained after processing by the convolution layer, activation layer and pooling layer, and connect the expanded content to the output layer through a fully connected method to obtain the output data.

[0072] In the above embodiment, the concentration level indicated by the concentration detection data is identified based on a pre-trained concentration detection model, and the concentration level is automatically identified by machine learning, rather than obtaining the concentration level based on a preset range value, thereby improving the accuracy of concentration recognition and reducing the misjudgment rate.

[0073] In some embodiments, optionally, the method further comprises:

[0074] Training the concentration detection model;

[0075] Wherein, the training of the concentration detection model comprises:

[0076] Acquire a concentration sample data set; wherein each concentration training sample in the concentration sample data set is associated with a concentration level label, and the concentration level label is determined according to body movement data, breathing data, and heart rate data in the concentration training sample;

[0077] Forming sample input data based on the concentration training sample, and inputting the sample input data into an initial concentration detection model including initial parameters;

[0078] Extracting body movement features, breathing features and heart rate features in the concentration training sample through the initial concentration detection model;

[0079] During the iterative training process, the recognition result of the concentration detection model in each iterative training is obtained according to the body movement characteristics, breathing characteristics and heart rate characteristics, and the loss value of each iteration is determined according to the matching degree between the recognition result and the concentration level label;

[0080] When the loss value of the current iteration reaches a preset iteration termination condition, the concentration detection model after the iteration termination is determined as the pre-trained concentration detection model.

[0081] In this embodiment, concentration training samples at various concentration levels are collected in advance. The richer the samples, the more accurate the concentration detection model obtained through training. During the training process, the concentration detection model gradually learns the features at each concentration level. After the iterative training is completed, the concentration level corresponding to the concentration detection data can be accurately identified. The iterative termination conditions include but are not limited to the number of iterations exceeding the preset number, the iterative loss value being less than the preset error value, etc.

[0082] When the sample input data is input into the concentration detection model for training, the concentration detection model is iteratively trained based on the concentration sample data set. In each iteration, the sample input data of the current iteration is input into the concentration detection model of the current iteration, and the concentration level of the current iteration is output. Based on the concentration level of the current iteration and the concentration level label corresponding to the sample input data of the current iteration, the loss value of the current iteration is calculated. If the current iteration does not meet the iteration termination condition, the sample input data is continuously obtained from the concentration sample data set for iterative training until the iteration termination condition is met.

[0083] In the above embodiment, the concentration detection model is trained and learned based on each training sample in the concentration sample data set and the concentration level label corresponding to each training sample. This enables the concentration detection model to use the concentration level label corresponding to each training sample as a training target, so that the concentration detection model can accurately identify the concentration level in the future, thereby improving the detection accuracy of the concentration level.

[0084] In some embodiments, the method further comprises:

[0085] receiving confirmation information of the terminal device regarding the concentration level in the current time period;

[0086] If the confirmation information contains a modified concentration level, the modified concentration level is used as the current concentration level; and / or if the confirmation information contains a modified concentration value, the modified concentration value is used as the current concentration value.

[0087] In one application scenario, the electronic device is a smart desk lamp, and the terminal device is a parent's mobile phone, etc. When the child is using the smart desk lamp, the concentration is monitored in real time. When it is detected that the current concentration value does not match the current concentration level, a confirmation message is sent to the terminal device to allow the parent to confirm the concentration status in the current time period to avoid recognition errors. Parents can modify the current concentration level and current concentration value in the user interface of the terminal device based on the current concentration value and the current concentration level and some returned feature data.

[0088] In the above embodiment, interaction between the electronic device and the terminal device is also provided, so that the user of the terminal device can reconfirm the concentration detection status in the current time period, thereby improving the accuracy of concentration detection and reducing misjudgment.

[0089] In some embodiments, the method further comprises:

[0090] receiving confirmation information of the terminal device regarding the concentration level in the current time period;

[0091] If the confirmation information contains a modified concentration level, the body movement data, the breathing data, the heart rate data and the modified concentration level are used to form a new concentration training sample, and the new concentration training sample is added to the concentration sample data set to continue training the concentration detection model.

[0092] In this embodiment, the user of the terminal device has modified the current concentration level, which means that the confidence of the concentration level recognized by the concentration detection model is not high enough and the concentration detection model needs to be trained again. Therefore, the concentration detection data and the modified concentration level can be formed into a concentration training sample and added to the concentration sample data set to continue training the concentration detection model.

[0093] In the above embodiment, during the monitoring process, sample data with low confidence is collected, added to the training data set, and the concentration detection model is continuously trained, thereby improving the accuracy of concentration detection and reducing misjudgments.

[0094] On the other hand, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the concentration detection method based on millimeter-wave radar described in any embodiment of the present application.

[0095] Among them, in the computer program product, an optional implementation form of the program module architecture of the computer program that implements each step of the target recognition method can be a concentration detection device based on millimeter wave radar.

[0096] See also Figure 6An embodiment of the present application provides a concentration detection device based on millimeter-wave radar, including: a control module 61, used to control the millimeter-wave radar sensor to send electromagnetic wave signals to a target space area, and receive an echo signal reflected by the target space area; a judgment module 62, used to judge whether there is a human body in the target space area according to the echo signal; an extraction module 63, used to extract concentration detection data of the human body based on the echo signal when it is determined that there is a human body in the target space area according to the echo signal; a calculation module 64, used to calculate the current concentration data of the human body in a current time period based on the concentration detection data.

[0097] Optionally, the computing module 64 is further configured to:

[0098] Based on the concentration detection data and using a first concentration calculation method, calculate a current concentration value of the human body in a current time period;

[0099] Based on the concentration detection data and using a second concentration calculation method, calculate the current concentration level of the human body in a current time period;

[0100] When the current concentration value matches the current concentration level, the current concentration value and the current concentration level are used as the current concentration data; when the current concentration value does not match the current concentration level, and the mismatch meets a preset condition, a prompt message is sent to a terminal device connected to the electronic device to request the terminal device to confirm the concentration within the current time period.

[0101] Optionally, the computing module 64 is further configured to:

[0102] Extracting body motion feature data, breathing feature data and heart rate feature data based on the concentration detection data, wherein the body motion feature data is used to indicate data of changes in the posture of a human body in the current time period; the breathing feature data is used to indicate data of fluctuations in breathing in the current time period; and the heart rate feature data is used to indicate heart rate data of a human body in the current time period;

[0103] Based on the body motion characteristic data, the breathing characteristic data and the heart rate characteristic data, the current concentration value is calculated using a concentration calculation formula, wherein the concentration calculation formula is as follows:

[0104]

[0105] Where V represents the current concentration value, α, β and γ are the first weight coefficient, the second weight parameter and the third weight parameter respectively, wherein the first weight coefficient represents the contribution of body motion feature data to the calculation of concentration, the second weight coefficient represents the contribution of heart rate feature data to the calculation of concentration, and the third weight coefficient represents the contribution of breathing feature data to the calculation of concentration, d i represents the body movement value at the i-th second in the current time period, n represents the total number of seconds in the current time period, h(i) represents the heart rate value at the i-th second, N represents the number of breaths in the current time period, x(i) represents the number of breaths at the i-th second, and μ represents the average number of breaths per minute.

[0106] Optionally, the computing module 64 is further configured to:

[0107] Calculating the first weight coefficient, the second weight parameter and the third weight parameter;

[0108] The calculating of the first weight coefficient, the second weight parameter and the third weight parameter comprises:

[0109] Acquire multiple sets of different sample feature data, and form a feature sample matrix with the multiple sets of sample feature data; each set of sample feature data includes body movement sample feature data, breathing sample feature data and heart rate sample feature data, where Represents the feature data of body motion samples, Represents the characteristic data of heart rate samples, Represents breath sample characteristic data;

[0110] Performing mean processing on the feature sample matrix to obtain a processed matrix, and generating an association matrix based on the processed matrix, where i represents the i-th group and j represents the j-th sample feature;

[0111] Based on the correlation matrix, the grey correlation degree r of each sample feature is calculated. j ,in Among them A tj represents the data in the tth row and jth column of the association matrix, j represents the jth sample feature, and G represents the total number of groups;

[0112] Grey relational degree r based on the characteristics of each sample j , respectively calculate the first weight coefficient, the second weight parameter and the third weight parameter, wherein the first weight coefficient Among them, r1 represents the grey correlation degree corresponding to the body movement sample feature, r2 represents the grey correlation degree corresponding to the heart rate sample feature, and r3 represents the grey correlation degree corresponding to the breathing sample feature.

[0113] Optionally, the computing module 64 is further configured to:

[0114] forming input data of a pre-trained concentration detection model based on the body motion data, the breathing data and the heart rate data, wherein the pre-trained concentration detection model comprises a first convolutional layer, a second convolutional layer and a third convolutional layer, wherein the first convolutional layer is used to obtain a body motion feature vector from the body motion data, the second convolutional layer is used to obtain a breathing feature vector from the breathing data, and the third convolutional layer is used to obtain a heart rate feature vector from the heart rate data;

[0115] Inputting body motion input data formed by the body motion data into the first convolution layer to obtain the body motion feature vector, inputting breathing input data formed by the breathing data into the second convolution layer to obtain the breathing feature vector, and inputting heart rate input data formed by the heart rate data into the third convolution layer to obtain the heart rate feature vector;

[0116] splicing the body movement feature vector, the breathing feature vector and the heart rate feature vector to obtain a concentration splicing feature vector;

[0117] The concentration concatenated feature vector is input into a concentration classification network layer in a pre-trained concentration detection model, and the current concentration level is output through the concentration classification network layer.

[0118] Optionally, the computing module 64 is further configured to:

[0119] Training the concentration detection model;

[0120] Wherein, the training of the concentration detection model comprises:

[0121] Acquire a concentration sample data set; wherein each concentration training sample in the concentration sample data set is associated with a concentration level label, and the concentration level label is determined according to body movement data, breathing data, and heart rate data in the concentration training sample;

[0122] Forming sample input data based on the concentration training sample, and inputting the sample input data into an initial concentration detection model including initial parameters;

[0123] Extracting body movement features, breathing features and heart rate features in the concentration training sample through the initial concentration detection model;

[0124] During the iterative training process, the recognition result of the concentration detection model in each iterative training is obtained according to the body movement characteristics, breathing characteristics and heart rate characteristics, and the loss value of each iteration is determined according to the matching degree between the recognition result and the concentration level label;

[0125] When the loss value of the current iteration reaches a preset iteration termination condition, the concentration detection model after the iteration termination is determined as the pre-trained concentration detection model.

[0126] Optionally, the computing module 64 is further configured to:

[0127] receiving confirmation information of the terminal device regarding the concentration level in the current time period;

[0128] If the confirmation information contains a modified concentration level, the modified concentration level is used as the current concentration level; and / or if the confirmation information contains a modified concentration value, the modified concentration value is used as the current concentration value.

[0129] Optionally, the computing module 64 is further configured to:

[0130] receiving confirmation information of the terminal device regarding the concentration level in the current time period;

[0131] If the confirmation information contains a modified concentration level, the body movement data, the breathing data, the heart rate data and the modified concentration level are used to form a new concentration training sample, and the new concentration training sample is added to the concentration sample data set to continue training the concentration detection model.

[0132] It can be understood by those skilled in the art that Figure 6 The structure of the concentration detection device based on millimeter-wave radar does not constitute a limitation on the concentration detection device based on millimeter-wave radar, and the various modules can be implemented in whole or in part by software, hardware, and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules. In other embodiments, the concentration detection device based on millimeter-wave radar may include more or fewer modules than shown in the figure.

[0133] See also Figure 7 In another aspect of the embodiment of the present application, an electronic device 10 is provided, including a processor 13 and a memory 14, wherein the memory 14 stores a computer program, and when the computer program is executed by the processor, the processor 13 executes the steps of the concentration detection method based on millimeter wave radar provided in any of the above embodiments of the present application. The electronic device 10 is a device with infrared imaging function, for example, it may include a computing device (for example, a desktop computer, a laptop computer, a tablet computer, a handheld computer, a smart speaker, a server, etc.), a terminal device (for example, a mobile phone, etc.), a wearable device (for example, a pair of smart glasses or a smart watch), a smart desk lamp, or a similar device.

[0134] The processor 13 is the control center, which uses various interfaces and lines to connect various parts of the entire computer device, and executes various functions of the computer device and processes data by running or executing software programs and / or modules stored in the memory 14, and calling data stored in the memory 14. Optionally, the processor 13 may include one or more processing cores; preferably, the processor 13 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user pages and application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 13.

[0135] The memory 14 can be used to store software programs and modules. The processor 13 executes various functional applications and data processing by running the software programs and modules stored in the memory 14. The memory 14 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the computer device, etc. In addition, the memory 14 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 14 can also include a memory processor to provide the processor 13 with access to the memory 14.

[0136] On the other hand, an embodiment of the present application further provides a storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of the concentration detection method based on millimeter wave radar provided in any of the above embodiments of the present application.

[0137] Those skilled in the art can understand that all or part of the processes in the methods provided in the above embodiments can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0138] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. The protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A concentration detection method based on millimeter wave radar, characterized in that: Applied in an electronic device, wherein the electronic device is provided with a millimeter wave radar sensor, including: Controlling the millimeter wave radar sensor to send an electromagnetic wave signal to a target space region, and receiving an echo signal reflected by the target space region; Determining whether there is a human body in the target space area according to the echo signal; When it is determined according to the echo signal that a human body exists in the target space area, extracting concentration detection data of the human body based on the echo signal; Based on the concentration detection data, current concentration data of the human body in a current time period is calculated.

2. The concentration detection method based on millimeter wave radar as claimed in claim 1, characterized in that: The calculating, based on the concentration detection data, current concentration data of the human body in a current time period comprises: Based on the concentration detection data and using a first concentration calculation method, calculate a current concentration value of the human body in a current time period; Based on the concentration detection data and using a second concentration calculation method, calculate the current concentration level of the human body in a current time period; When the current concentration value matches the current concentration level, the current concentration value and the current concentration level are used as the current concentration data; when the current concentration value does not match the current concentration level, and the mismatch meets a preset condition, a prompt message is sent to a terminal device connected to the electronic device to request the terminal device to confirm the concentration within the current time period.

3. The concentration detection method based on millimeter wave radar as claimed in claim 2, characterized in that: The calculating the current concentration value of the human body based on the concentration detection data and using a first concentration calculation method includes: Extracting body motion feature data, breathing feature data and heart rate feature data based on the concentration detection data, wherein the body motion feature data is used to indicate data of changes in the posture of a human body in the current time period; the breathing feature data is used to indicate data of fluctuations in breathing in the current time period; and the heart rate feature data is used to indicate heart rate data of a human body in the current time period; Based on the body motion characteristic data, the breathing characteristic data and the heart rate characteristic data, the current concentration value is calculated using a concentration calculation formula, wherein the concentration calculation formula is as follows: Where V represents the current concentration value, α, β and γ are the first weight coefficient, the second weight parameter and the third weight parameter respectively, wherein the first weight coefficient represents the contribution of body motion feature data to the calculation of concentration, the second weight coefficient represents the contribution of heart rate feature data to the calculation of concentration, and the third weight coefficient represents the contribution of breathing feature data to the calculation of concentration, d i represents the body movement value at the i-th second in the current time period, n represents the total number of seconds in the current time period, h(i) represents the heart rate value at the i-th second, N represents the number of breaths in the current time period, x(i) represents the number of breaths at the i-th second, and μ represents the average number of breaths per minute.

4. The concentration detection method based on millimeter wave radar as claimed in claim 3, characterized in that: The calculating the current concentration value of the human body based on the concentration detection data and using the first concentration calculation method also includes: Calculating the first weight coefficient, the second weight parameter and the third weight parameter; The calculating of the first weight coefficient, the second weight parameter and the third weight parameter comprises: Acquire multiple sets of different sample feature data, and form a feature sample matrix with the multiple sets of sample feature data; each set of sample feature data includes body movement sample feature data, breathing sample feature data and heart rate sample feature data, where Represents the feature data of body motion samples, Represents the characteristic data of heart rate samples, Represents breath sample characteristic data; Performing mean processing on the feature sample matrix to obtain a processed matrix, and generating an association matrix based on the processed matrix, where i represents the i-th group and j represents the j-th sample feature; Based on the correlation matrix, the grey correlation degree r of each sample feature is calculated. j ,in Among them A tj represents the data in the tth row and jth column of the association matrix, j represents the jth sample feature, and G represents the total number of groups; Grey relational degree r based on the characteristics of each sample j , respectively calculate the first weight coefficient, the second weight parameter and the third weight parameter, wherein the first weight coefficient Among them, r1 represents the grey correlation degree corresponding to the body movement sample feature, r2 represents the grey correlation degree corresponding to the heart rate sample feature, and r3 represents the grey correlation degree corresponding to the breathing sample feature.

5. The concentration detection method based on millimeter wave radar as claimed in claim 2, characterized in that: The concentration detection data includes body movement data, breathing data and heart rate data. The calculation of the current concentration level of the human body in the current time period based on the concentration detection data and using the second concentration calculation method includes: forming input data of a pre-trained concentration detection model based on the body motion data, the breathing data and the heart rate data, wherein the pre-trained concentration detection model comprises a first convolutional layer, a second convolutional layer and a third convolutional layer, wherein the first convolutional layer is used to obtain a body motion feature vector from the body motion data, the second convolutional layer is used to obtain a breathing feature vector from the breathing data, and the third convolutional layer is used to obtain a heart rate feature vector from the heart rate data; Inputting body motion input data formed by the body motion data into the first convolution layer to obtain the body motion feature vector, inputting breathing input data formed by the breathing data into the second convolution layer to obtain the breathing feature vector, and inputting heart rate input data formed by the heart rate data into the third convolution layer to obtain the heart rate feature vector; splicing the body movement feature vector, the breathing feature vector and the heart rate feature vector to obtain a concentration splicing feature vector; The concentration concatenated feature vector is input into a concentration classification network layer in a pre-trained concentration detection model, and the current concentration level is output through the concentration classification network layer.

6. The concentration detection method based on millimeter wave radar as claimed in claim 5, characterized in that: The method further comprises: Training the concentration detection model; Wherein, the training of the concentration detection model comprises: Acquire a concentration sample data set; wherein each concentration training sample in the concentration sample data set is associated with a concentration level label, and the concentration level label is determined according to body movement data, breathing data, and heart rate data in the concentration training sample; Forming sample input data based on the concentration training sample, and inputting the sample input data into an initial concentration detection model including initial parameters; Extracting body movement features, breathing features and heart rate features in the concentration training sample through the initial concentration detection model; During the iterative training process, the recognition result of the concentration detection model in each iterative training is obtained according to the body movement characteristics, breathing characteristics and heart rate characteristics, and the loss value of each iteration is determined according to the matching degree between the recognition result and the concentration level label; When the loss value of the current iteration reaches a preset iteration termination condition, the concentration detection model after the iteration termination is determined as the pre-trained concentration detection model.

7. The concentration detection method based on millimeter wave radar as claimed in claim 2, characterized in that: The method further comprises: receiving confirmation information of the terminal device regarding the concentration level in the current time period; If the confirmation information contains a modified concentration level, the modified concentration level is used as the current concentration level; and / or if the confirmation information contains a modified concentration value, the modified concentration value is used as the current concentration value.

8. The method for detecting concentration based on millimeter wave radar as claimed in claim 6, characterized in that: The method further comprises: receiving confirmation information of the terminal device regarding the concentration level in the current time period; If the confirmation information contains a modified concentration level, the body movement data, the breathing data, the heart rate data and the modified concentration level are used to form a new concentration training sample, and the new concentration training sample is added to the concentration sample data set to continue training the concentration detection model.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the concentration detection method based on millimeter-wave radar as described in any one of claims 1 to 8 is implemented.

10. An electronic device, characterized in that: It comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the concentration detection method based on millimeter-wave radar as described in any one of claims 1 to 8.

Images