Millimeter-Wave Radar-Based Attention Detection Method, Program Product, and Electronic Device

Abstract

The present invention discloses a method, program product and electronic device for detecting concentration based on a millimeter-wave radar. The method includes: controlling the millimeter-wave radar sensor to send an electromagnetic wave signal to a target space area and receiving an echo signal reflected by the target space area; judging whether there is a human body in the target space area according to the echo signal; when it is determined that there is a human body in the target space area according to the echo signal, extracting concentration detection data of the human body based on the echo signal; and calculating current concentration data of the human body within a current time period based on the concentration detection data.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular, to a method for detecting concentration based on a millimeter-wave radar, a computer program product, and an electronic device. Background Art

[0002] In an educational environment, the concentration of students directly affects the learning effect. By detecting the concentration, teachers can understand the learning status of students and take corresponding measures to improve the teaching quality. Therefore, real-time monitoring of the concentration of users during the learning process can remind users in real time when they are not focused to help users improve learning efficiency.

[0003] In the prior art, the concentration of users can be detected through electroencephalogram signals, but users need to wear devices to cooperate with the detection of concentration. This requires an additional cost for wearing the devices, and also requires users to actively cooperate with wearing the devices. However, some users may not cooperate with wearing, so real-time monitoring cannot be achieved. There are also methods that collect images of users through cameras and analyze the facial expressions and body movements of users through image data to determine the concentration of users. However, due to the large number of interfering background factors in the images, the recognition results are prone to self-contradiction or unstable fluctuations between front and back frames, affecting the accuracy of the final determination of the concentration. Summary of the Invention

[0004] In order to solve the existing technical problems, the present invention provides a method for detecting concentration based on a millimeter-wave radar, a computer program product, and an electronic device, which can improve the accuracy of concentration detection.

[0005] In a first aspect, a method for detecting concentration based on a millimeter-wave radar is provided, including: controlling the millimeter-wave radar sensor to send an electromagnetic wave signal to a target space area and receiving an echo signal reflected by the target space area; judging whether there is a human body in the target space area according to the echo signal; when it is determined that there is a human body in the target space area according to the echo signal, extracting concentration detection data of the human body based on the echo signal; and calculating the current concentration data of the human body within the current time period based on the concentration detection data.

[0006] In a second aspect, a computer program product is provided, including a computer program, which when executed by a processor, implements the method for detecting concentration based on a millimeter-wave radar as described in any embodiment of the present application.

[0007] In a third aspect, an electronic device is provided, including a memory and a processor, where the memory stores a computer program, and when the computer program is executed by the processor, the processor is caused to execute the method for detecting concentration based on a millimeter-wave radar as described in any embodiment of the present application.

[0008] In the embodiments of the present application, an electromagnetic wave signal is transmitted into a target space area by using a millimeter-wave radar sensor, and the reflected echo signal is received. First, it is determined whether there is a person based on the echo signal to detect whether there is a person in the target space area. If there is a person, the attention detection data is continuously extracted based on the echo signal, and the current attention data of the human body is monitored in real time based on the attention detection data. In this way, there is no need for the user to wear a dedicated device, which can save costs, and the attention detection data can be collected unconsciously. The attention detection data collected by the millimeter-wave radar sensor can be various, and the current attention can be obtained comprehensively based on multiple data, which can improve the accuracy of attention detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 FIG. is an application environment diagram of an attention detection method based on a millimeter-wave radar in an embodiment;

[0010] Figure 2 FIG. is a schematic diagram of an electronic device as a smart table lamp in an embodiment;

[0011] Figure 3 FIG. is an application environment diagram of an attention detection method based on a millimeter-wave radar in another embodiment;

[0012] Figure 4 FIG. is a flowchart of an attention detection method based on a millimeter-wave radar in an embodiment;

[0013] Figure 5 FIG. is a flowchart of an attention detection method based on a millimeter-wave radar in another embodiment;

[0014] Figure 6 FIG. is a schematic diagram of an attention detection device based on a millimeter-wave radar in an embodiment;

[0015] Figure 7 FIG. is a schematic diagram of an electronic device in an embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0016] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings of the specification and specific embodiments.

[0017] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the protection scope of the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0018] In the following description, the expression "some embodiments" describes a subset of all possible embodiments. However, it should be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0019] Referring to Figure 1 , it is an application environment diagram of a concentration detection method based on a millimeter-wave radar in an embodiment. The concentration detection method based on the millimeter-wave radar is applied to the electronic device 10. The electronic device 10 includes a millimeter-wave radar sensor 12 and a processor 13. The millimeter-wave radar sensor 12 is used to send electromagnetic wave signals to the target space area. Since there are multiple targets such as a human body and other objects in the target space area, these multiple targets will reflect the electromagnetic wave signals, and the millimeter-wave radar sensor 12 can receive the echo signals reflected by the multiple targets. The processor 13 detects the concentration of the human body in front of the electronic device according to the echo signals detected by the millimeter-wave radar sensor 12.

[0020] The millimeter-wave radar sensor 12 is a radar system that uses electromagnetic waves in the millimeter-wave band to detect targets. The wavelength range of millimeter waves is between 1 mm and 10 mm, which is between microwaves and terahertz waves. The millimeter-wave radar sensor obtains the information of the target by transmitting millimeter-wave signals and receiving the signals reflected by the target. The information of the target includes but is not limited to the body movement data of the target, the breathing data and heart rate data in the vital signs of the target.

[0021] Wherein the processor 13 can be one or more. When there are multiple processors 13, the multiple processors can be integrated on one chip or independently set on each chip. The electronic device 10 is a device equipped with a millimeter-wave radar sensor, and can include, for example, a computing device (such as a desktop computer, a laptop computer, a tablet computer, a handheld computer, a smart speaker, a server, etc.), a terminal device (such as a mobile phone, etc.), a wearable device (such as a pair of smart glasses or a smart watch), various infrared imaging devices, a smart table lamp or similar devices.

[0022] As Figure 2 shown, Figure 2Schematic diagram of an electronic device in an embodiment; the electronic device 10 is a smart table lamp, and a millimeter-wave radar sensor 12 is provided in the smart table lamp. The smart table lamp can be placed in the front area of the user. On the premise of providing the lighting function for the user, it can also detect the user's concentration, so that the user's concentration can be monitored unconsciously without adding wearable devices. When the smart table lamp is in the on state, that is, when providing the lighting function, the millimeter-wave radar sensor 12 sends an electromagnetic wave signal to the target space area corresponding to its installation position, and based on the echo signal reflected by the human body, the body movement data, breathing data and heart rate data of the human body can be obtained, and based on the obtained data, the user's concentration can be automatically detected.

[0023] As Figure 3 shown, it is an application environment diagram of a concentration detection method based on millimeter-wave radar in another embodiment. The electronic device 10 can also be communicatively connected to the terminal device 20, and the electronic device 10 can send the detected concentration situation of the user to the terminal device 20. For example, when a child is doing homework, the smart table lamp is turned on and used. The smart table lamp automatically detects the concentration of the child during the homework process and forms a report based on the detected concentration data and sends it to the parent's terminal device 20. In this way, the parent can understand the child's learning situation in real time without staring at the child all the time.

[0024] Please refer to Figure 4 , which is a flowchart of a concentration detection method based on millimeter-wave radar provided by an embodiment of the present application. The concentration detection method based on millimeter-wave radar is applied to an electronic device, and the concentration detection method based on millimeter-wave radar includes the following steps:

[0025] S11. Control the millimeter-wave radar sensor to send an electromagnetic wave signal to the target space area and receive the echo signal reflected by the target space area.

[0026] In this embodiment, the target space area indicates the area that the signal of the millimeter-wave radar sensor can transmit to. Due to the installation position of the millimeter-wave radar sensor in the electronic device, the target space area is related to the installation position and the parameters of the millimeter-wave radar sensor itself. When the electronic device is in the on state, control the millimeter-wave radar sensor to send an electromagnetic wave signal.

[0027] S12. Judge whether there is a human body in the target space area according to the echo signal.

[0028] In this embodiment, the echo signal is first preprocessed to obtain a preprocessed signal. The preprocessing operations include, but are not limited to, static clutter cancellation, mixer-based processing operations, analog-to-digital converter-based sampling operations, fast-time Fourier transform-based transformation operations, and so on. Among them, performing static clutter cancellation on the echo signal can reduce the influence of clutter signals generated by large static objects in the room on the target echo signal. The mixer-based processing operation mixes the transmitted signal and the echo signal to obtain a signal with a new frequency, which is an intermediate-frequency signal. The frequencies and phases of the echo signal and the transmitted signal are different. Therefore, data such as distance, speed, and angle can be extracted and analyzed from the superimposed intermediate-frequency signal. The analog-to-digital converter-based sampling operation is used to sample the processed echo signal, and the sampled signal is subjected to Fourier transform processing to obtain a discrete echo signal x(m,n) containing range dimension and slow-time dimension information, where m represents the slow-time dimension and is the m-th pulse echo, and n represents the range dimension and is the n-th range cell.

[0029] In some embodiments, it is then determined whether there is a human body in the target space region based on the preprocessed signal. Since the radar continuously transmits the transmitted signal, if a preset number of point cloud data are continuously obtained from the preprocessed signal within a preset time period, it indicates that there is a person in the detection scene. In other embodiments, the preprocessed signal can also be input into a human body classifier to identify whether there is a human body in the preprocessed signal through the human body classifier. If the human body classifier outputs that there is a person, it indicates that there is a human body; otherwise, there is no human body. This application does not make any limitations on the method for determining whether there is a human body in the target space region based on the echo signal.

[0030] S13. When it is determined that there is a human body in the target space region based on the echo signal, the attention detection data of the human body is extracted based on the echo signal.

[0031] In this embodiment, the attention detection data is data required for detecting attention, including but not limited to body movement data, breathing data, and heart rate data. Among them, the body movement data is used to represent information on the movement and posture changes of the human body, such as the human body posture change data when using a smart table lamp. The breathing data is used to indicate the breathing frequency of the human body within a certain time period, such as the breathing frequency of the human body when using a smart table lamp. The heart rate data is used to represent the heart rate data of the human body within a certain time period, such as the heart rate data of the human body when using a smart table lamp.

[0032] In some application scenarios, after the user turns on the electronic device, they may not appear in the target space area. For example, the user turns on the smart table lamp but does not sit in front of it. To reduce false detections or resource waste, after detecting a human body in the target space area, the concentration detection data of the human body is extracted. When no human body is detected in the target space area, return to continue executing S12.

[0033] In some embodiments, phase data is extracted based on the echo signal, and a phase unwrapping operation is performed on the extracted phase data. By subtracting consecutive phase values and performing a phase difference operation on the unwrapped phase, the heartbeat signal can be enhanced and any phase drift can be eliminated. According to the different heart rate and breathing frequencies, the phase values are filtered using a band-pass filter for differentiation, so that respiratory data and heart rate data can be obtained. Specifically, a Fourier transform is performed on the phase signal, and according to the peak magnitude and its harmonic characteristics, the corresponding respiratory frequency within N frame times is obtained. The respiratory frequency is recorded for a period of time, and the current respiratory frequency is judged according to different confidence indicators, and the relationship between the respiratory frequency and time is output. After filtering the phase, the purpose here is to reduce the influence on heart rate measurement caused by the relative position movement of the human body. Because the heart rate measurement is based on the distance difference caused by the tiny movement of the heart during systole and diastole, which causes a phase change. According to the micro-Doppler principle, when the human body makes a large swing, its accuracy will be affected. Here, by segmenting the samples and setting a threshold to judge whether it meets the heart rate change range, the vital sign signal is finally obtained.

[0034] In some embodiments, the Doppler shift in the echo signal can also be extracted based on the echo signal using methods such as Fourier transform, so as to obtain the body movement data of the human body.

[0035] S14. Based on the concentration detection data, calculate the current concentration data of the human body within the current time period.

[0036] In this embodiment, the current time period can be a time interval. For example, the time within the previous 1 minute of the current time constitutes a current time period, and the current time period is related to the preset time interval. In this way, the concentration data of the human body is calculated every preset time interval. The shorter the time interval, the more calculation times.

[0037] In the above embodiments, an electromagnetic wave signal is transmitted into the target space area by using a millimeter-wave radar sensor, and the reflected echo signal is received. First, it is determined whether there is a person based on the echo signal to detect whether there is a person in the target space area. If there is a person, the attention detection data is continuously extracted based on the echo signal, and the current attention data of the human body is monitored in real time based on the attention detection data. In this way, there is no need for the user to wear a dedicated device, which can save costs, and the attention detection data can be collected unconsciously. The attention detection data collected by the millimeter-wave radar sensor can be various, and the current attention can be obtained by synthesizing multiple data, which can improve the accuracy of attention detection.

[0038] In some embodiments, as Figure 5 shown, Figure 5 FIG. is a flowchart of an attention detection method based on a millimeter-wave radar in another embodiment. The attention detection method based on a millimeter-wave radar is applied to an electronic device, and the attention detection method based on a millimeter-wave radar includes the following steps:

[0039] S11. Control the millimeter-wave radar sensor to send an electromagnetic wave signal to the target space area and receive the echo signal reflected by the target space area.

[0040] S12. Determine whether there is a human body in the target space area according to the echo signal.

[0041] S13. When it is determined that there is a human body in the target space area according to the echo signal, extract the attention detection data of the human body based on the echo signal.

[0042] S14. Based on the attention detection data, and using the first attention calculation method, calculate the current attention value of the human body within the current time period.

[0043] S15. Based on the attention detection data, and using the second attention calculation method, calculate the current attention level of the human body within the current time period.

[0044] S16. Determine whether the current attention value matches the current attention level.

[0045] When the current attention value does not match the current attention level and the non-matching situation meets the preset conditions, execute S17; when the current attention value matches the current attention level, execute S18.

[0046] S17. Send a prompt message to the terminal device associated with the electronic device to request the terminal device to confirm the attention within the current time period.

[0047] S18. Use the current concentration value and the current concentration level as the current concentration data.

[0048] In this embodiment, S11 - S13 are the same as the steps of the above - described embodiment and will not be elaborated here. The first concentration calculation method is used to calculate the current concentration value within the current time period, and measures the user's current concentration situation through specific scores. The second concentration calculation method is used to calculate the current concentration level within the current time period, and measures the user's current concentration degree through level grades. For example, the concentration value can be set from 0 to 100 points, and the higher the score, the higher the concentration. The concentration level can be divided into high, medium, and low levels. Of course, more level grades can be set as needed. Different level grades can be preset according to different score values of the concentration value. For example, 0 to 60 is the low level, 60 - 85 is the medium level, and 85 to 100 is the high level. It is possible to determine whether the current concentration level matches the current concentration value based on the calculated current concentration value and the preset level grades. If the current concentration level is the same as the configured level corresponding to the current concentration value, it means that the current concentration level matches the current concentration value. If the current concentration level is different from the configured level corresponding to the current concentration value, it means that the current concentration level does not match the current concentration value. At this time, a prompt message can be sent to the terminal device to request the terminal device to confirm the concentration within the current time period, so as to reduce the probability of misjudgment.

[0049] In the above - mentioned embodiment, the current concentration value and the current concentration level are calculated by two different methods respectively, and the calculated current concentration value and the current concentration level are matched to determine whether the concentrations calculated by the two different methods are consistent, thereby reducing misjudgment. When the calculated current concentration value does not match the current concentration level, a message is sent to the terminal device to allow the user of the terminal device to confirm whether the concentration is accurate, thereby improving the accuracy of concentration detection.

[0050] In some embodiments, calculating the current concentration value of the human body based on the concentration feature data and using the first concentration calculation method includes:

[0051] Extract body movement feature data, respiratory feature data, and heart rate feature data based on the concentration detection data, where the body movement feature data is data indicating the posture change of the human body within the current time period; the respiratory feature data is data indicating the respiratory fluctuation within the current time period, and the heart rate feature data is data indicating the heart rate of the human body within the current time period;

[0052] Based on the body movement feature data, respiratory feature data, and heart rate feature data, calculate the current concentration value using the concentration calculation formula, where the concentration calculation formula is as follows:

[0053]

[0054] Among them, V represents the current concentration value, α, β, and γ are the first weight coefficient, the second weight parameter, and the third weight parameter respectively. The first weight coefficient represents the contribution degree of body movement characteristic data to the calculation of concentration, the second weight coefficient represents the contribution degree of heart rate characteristic data to the calculation of concentration, and the third weight coefficient represents the contribution degree of respiratory characteristic data to the calculation of concentration. d i represents the body movement value at the i-th second within the current time period, n represents the total number of seconds within the current time period, h(i) represents the heart rate value at the i-th second, N represents the number of breaths within the current time period, x(i) represents the number of breaths at the i-th second, and μ represents the average number of breaths per minute.

[0055] Optionally, calculating the current concentration value of the human body based on the concentration detection data and using the first concentration calculation method further includes:

[0056] Calculating the first weight coefficient, the second weight parameter, and the third weight parameter;

[0057] Among them, calculating the first weight coefficient, the second weight parameter, and the third weight parameter includes:

[0058] Obtaining multiple groups of different sample characteristic data and forming a characteristic sample matrix with the multiple groups of sample characteristic data; each group of sample characteristic data includes body movement sample characteristic data, respiratory sample characteristic data, and heart rate sample characteristic data, where represents the body movement sample characteristic data, represents the heart rate sample characteristic data, represents the respiratory sample characteristic data;

[0059] Performing an averaging process on the characteristic sample matrix to obtain a processed matrix, and generating a correlation matrix based on the processed matrix, where i represents the i-th group and j represents the j-th sample characteristic;

[0060] Based on the correlation matrix, calculating the grey correlation degree r j , where where A tj represents the data in the t-th row and j-th column of the correlation matrix, j represents the j-th sample characteristic, and G represents the total number of groups;

[0061] Based on the grey correlation degree r j of each sample characteristic, respectively calculating the first weight coefficient, the second weight parameter, and the third weight parameter, where the first weight coefficient where r 1represents the grey correlation degree corresponding to the body movement sample features, r 2 represents the grey correlation degree corresponding to the heart rate sample features, r 3 represents the grey correlation degree corresponding to the respiratory sample features.

[0062] In this embodiment, multiple groups of different sample feature data can be understood as multiple different sampling points. Collecting multiple groups of such data is used to evaluate the contribution degree of each feature data to the calculation of the concentration degree subsequently. In this way, the contribution degree of the concentration degree is not a preset value, but is calculated based on data from multiple different sampling points, so that the first weight coefficient, the second weight parameter, and the third weight parameter are more accurate, and thus the concentration degree value calculated through the concentration degree calculation formula is more accurate.

[0063] In the above embodiment, by combining the body movement feature data, the respiratory feature data, and the heart rate feature data within the current time period and the contribution degree corresponding to each feature, calculating the current concentration degree value can make the calculation of the concentration degree value more accurate. Moreover, the first weight coefficient, the second weight parameter, and the third weight parameter are calculated based on data from multiple different sampling points, so that the first weight coefficient, the second weight parameter, and the third weight parameter are more accurate, and thus the concentration degree value calculated through the concentration degree calculation formula is more accurate.

[0064] In some embodiments, calculating the current concentration degree level of the human body within the current time period based on the concentration degree detection data and using the second concentration degree calculation method includes:

[0065] Forming input data for a pre-trained concentration degree detection model based on the body movement data, the respiratory data, and the heart rate data. The pre-trained concentration degree detection model includes a first convolutional layer, a second convolutional layer, and a third convolutional layer. The first convolutional layer is used to obtain a body movement feature vector from the body movement data, the second convolutional layer is used to obtain a respiratory feature vector from the respiratory data, and the third convolutional layer is used to obtain a heart rate feature vector from the heart rate data;

[0066] Inputting the body movement input data formed by the body movement data into the first convolutional layer to obtain the body movement feature vector, inputting the respiratory input data formed by the respiratory data into the second convolutional layer to obtain the respiratory feature vector, and inputting the heart rate input data formed by the heart rate data into the third convolutional layer to obtain the heart rate feature vector;

[0067] Concatenating the body movement feature vector, the respiratory feature vector, and the heart rate feature vector to obtain a concentration degree concatenated feature vector;

[0068] Input the concentration splicing feature vector into the concentration classification network layer of the pre-trained concentration detection model, and output the current concentration level through the concentration classification network layer.

[0069] In this embodiment, the pre-trained concentration detection model is obtained by training with a concentration sample data set. During the training process, the initial concentration detection model can learn the features in the samples based on the concentration sample data set. After the training is completed, the concentration detection model can accurately obtain the current concentration level based on the concentration detection data.

[0070] The pre-trained concentration detection model includes three different convolutional layers, and each convolutional layer can be composed of one or more convolutional kernels. These three different convolutional layers are respectively used to extract the body movement feature vector, the breathing feature vector, and the heart rate feature vector from the concentration detection data, and then splice these three feature vectors. Finally, the spliced feature vector is input into the concentration classification network layer to output the current concentration level.

[0071] It can be understood that the concentration detection model can also include normalization processing, pooling layers, fully connected layers, etc. Among them, the normalization operation avoids the sudden increase or decrease of numerical values during the processing. The pooling layer is used to perform downsampling on the output of the convolutional layer to reduce the data dimension, reduce the model complexity and the amount of calculation. The fully connected layer can expand the output obtained after being processed by the convolutional layer, the activation layer, and the pooling layer, and connect the expanded content to the output layer in a fully connected manner to obtain the output data.

[0072] In the above embodiment, based on the pre-trained concentration detection model, the concentration level indicated by the concentration detection data is recognized, and the concentration level is automatically recognized by machine learning, rather than obtaining the concentration level according to the preset range value, so as to improve the accuracy of concentration recognition and reduce the misjudgment rate.

[0073] In some embodiments, optionally, the method further includes:

[0074] Train the concentration detection model;

[0075] Wherein, training the concentration detection model includes:

[0076] Obtain a concentration sample data set; wherein, each concentration training sample in the concentration sample data set is associated with a concentration level label, and the concentration level label is determined according to the body movement data, breathing data, and heart rate data in the concentration training sample;

[0077] Form sample input data based on the concentration training sample, and input the sample input data into the initial concentration detection model with initial parameters;

[0078] Extract the body movement features, respiratory features, and heart rate features in the attention training samples through the initial attention detection model;

[0079] In the iterative training process, obtain the recognition results of the attention detection model in each iterative training according to the body movement features, respiratory features, and heart rate features, and determine the loss value of each iteration according to the matching degree between the recognition results and the attention level labels;

[0080] When the loss value of the current iteration reaches the preset iteration termination condition, determine the attention detection model after the iteration termination as the pre-trained attention detection model.

[0081] In this embodiment, attention training samples under various different attention levels are collected in advance. The richer the samples are, the more accurate the trained attention detection model will be. During the training process, the attention detection model gradually learns the features under each attention level. After the iterative training is completed, it can accurately identify the attention level corresponding to the attention detection data. The iteration termination conditions include but are not limited to the number of iterations exceeding the preset number, the loss value of the iteration being less than the preset error value, and so on.

[0082] When inputting the sample input data into the attention detection model for training, iteratively train the attention detection model based on the attention sample data set. In each iteration, input the sample input data of the current iteration into the attention detection model of the current iteration, output the attention level of the current iteration, calculate the loss value of the current iteration based on the attention level of the current iteration and the attention level label corresponding to the sample input data of the current iteration. If the current iteration does not meet the iteration termination condition, continue to obtain the sample input data from the attention sample data set and perform iterative training until the iteration termination condition is met.

[0083] In the above embodiment, training and learning the attention detection model based on each training sample in the attention sample data set and the attention level label corresponding to each training sample can enable the attention detection model to use the attention level label corresponding to each training sample as the training target, which is convenient for the attention detection model to accurately identify the attention level subsequently, thereby improving the detection accuracy of the attention level.

[0084] In some embodiments, the method further includes:

[0085] Receive the confirmation information of the attention of the terminal device during the current time period;

[0086] If there is a modified concentration level in the confirmation information, use the modified concentration level as the current concentration level; and / or if there is a modified concentration value in the confirmation information, use the modified concentration value as the current concentration value.

[0087] In an application scenario, the electronic device is a smart table lamp, and the terminal device is the parent's mobile phone, etc. When the child is using the smart table lamp, the concentration is monitored in real time. When it is detected that the current concentration value does not match the current concentration level, a confirmation information is sent to the terminal device to allow the parent to confirm the concentration situation during the current time period to avoid misidentification. The parent can modify the current concentration level and the current concentration value on the user interface of the terminal device according to the current concentration value, the current concentration level, and some returned characteristic data.

[0088] In the above embodiment, an interaction between the electronic device and the terminal device is also provided to allow the user of the terminal device to confirm the concentration detection situation during the current time period again, thereby improving the accuracy of concentration detection and reducing misjudgment.

[0089] In some embodiments, the method further includes:

[0090] Receiving a confirmation information of the concentration during the current time period from the terminal device;

[0091] If there is a modified concentration level in the confirmation information, form a new concentration training sample with the body movement data, the breathing data, the heart rate data, and the modified concentration level, and add the new concentration training sample to the concentration sample data set to continue training the concentration detection model.

[0092] In this embodiment, when the user of the terminal device modifies the current concentration level, it means that the confidence level of the concentration level recognized by the concentration detection model is not high enough, and the concentration detection model needs to be trained again. Therefore, the concentration detection data and the modified concentration level can be formed into a concentration training sample and added to the concentration sample data set to continue training the concentration detection model.

[0093] In the above embodiment, during the monitoring process, sample data with low confidence is collected, and the sample data with low confidence is added to the training data set to continue training the concentration detection model, thereby improving the accuracy of concentration detection and reducing misjudgment.

[0094] On the other hand, the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the concentration detection method based on millimeter-wave radar according to any embodiment of the present application.

[0095] Among them, in the computer program product, an optional implementation form of the program module architecture of the computer program for implementing the steps of the target recognition method may be a concentration detection device based on a millimeter-wave radar.

[0096] Please refer to Figure 6 , an embodiment of the present application provides a concentration detection device based on a millimeter-wave radar, including: a control module 61, configured to control the millimeter-wave radar sensor to send an electromagnetic wave signal to a target space area and receive an echo signal reflected by the target space area; a judgment module 62, configured to judge whether there is a human body in the target space area according to the echo signal; an extraction module 63, configured to extract concentration detection data of the human body based on the echo signal when it is determined that there is a human body in the target space area according to the echo signal; a calculation module 64, configured to calculate the current concentration data of the human body within the current time period based on the concentration detection data.

[0097] Optionally, the calculation module 64 is further configured to:

[0098] Calculate the current concentration value of the human body within the current time period based on the concentration detection data and using a first concentration calculation method;

[0099] Calculate the current concentration level of the human body within the current time period based on the concentration detection data and using a second concentration calculation method;

[0100] When the current concentration value matches the current concentration level, use the current concentration value and the current concentration level as the current concentration data; when the current concentration value does not match the current concentration level and the non-matching situation meets a preset condition, send a prompt message to the terminal device associated with the electronic device to request the terminal device to confirm the concentration within the current time period.

[0101] Optionally, the calculation module 64 is further configured to:

[0102] Extract body movement feature data, breathing feature data, and heart rate feature data based on the concentration detection data, where the body movement feature data is data used to indicate the posture change of the human body within the current time period; the breathing feature data is data used to indicate the breathing fluctuation data within the current time period, and the heart rate feature data is data used to indicate the heart rate data of the human body within the current time period;

[0103] Calculate the current concentration value based on the body movement feature data, breathing feature data, and heart rate feature data and using a concentration calculation formula, where the concentration calculation formula is as follows:

[0104]

[0105] Where V represents the current concentration value, α, β, and γ are the first weight coefficient, the second weight parameter, and the third weight parameter respectively. The first weight coefficient represents the contribution degree of body movement feature data to the calculation of concentration, the second weight coefficient represents the contribution degree of heart rate feature data to the calculation of concentration, and the third weight coefficient represents the contribution degree of respiratory feature data to the calculation of concentration. d i represents the body movement value at the i-th second within the current time period, n represents the total number of seconds within the current time period, h(i) represents the heart rate value at the i-th second, N represents the number of breaths within the current time period, x(i) represents the number of breaths at the i-th second, and μ represents the average number of breaths per minute.

[0106] Optionally, the calculation module 64 is further configured to:

[0107] Calculate the first weight coefficient, the second weight parameter, and the third weight parameter;

[0108] Where the calculation of the first weight coefficient, the second weight parameter, and the third weight parameter includes:

[0109] Obtain multiple groups of different sample feature data and form a feature sample matrix with the multiple groups of sample feature data; each group of sample feature data includes body movement sample feature data, respiratory sample feature data, and heart rate sample feature data, where represents the body movement sample feature data, represents the heart rate sample feature data, represents the respiratory sample feature data;

[0110] Perform mean processing on the feature sample matrix to obtain a processed matrix, and generate a correlation matrix based on the processed matrix, where i represents the i-th group and j represents the j-th sample feature;

[0111] Based on the correlation matrix, calculate the grey correlation degree r j of each sample feature, where where A tj represents the data in the t-th row and j-th column of the correlation matrix, j represents the j-th sample feature, and G represents the total number of groups;

[0112] Based on the grey correlation degree r j of each sample feature, calculate the first weight coefficient, the second weight parameter, and the third weight parameter respectively, where the first weight coefficient where r 1 represents the grey correlation degree corresponding to the body movement sample feature, r 2 represents the grey correlation degree corresponding to the heart rate sample feature, r 3Represents the grey correlation degree corresponding to the respiratory sample characteristics.

[0113] Optionally, the calculation module 64 is further configured to:

[0114] Based on the body movement data, the respiratory data, and the heart rate data, form the input data of a pre-trained concentration detection model, where the pre-trained concentration detection model includes a first convolutional layer, a second convolutional layer, and a third convolutional layer, and where the first convolutional layer is used to obtain a body movement feature vector from the body movement data, the second convolutional layer is used to obtain a respiratory feature vector from the respiratory data, and the third convolutional layer is used to obtain a heart rate feature vector from the heart rate data;

[0115] Input the body movement input data formed by the body movement data into the first convolutional layer to obtain the body movement feature vector, input the respiratory input data formed by the respiratory data into the second convolutional layer to obtain the respiratory feature vector, and input the heart rate input data formed by the heart rate data into the third convolutional layer to obtain the heart rate feature vector;

[0116] Concatenate the body movement feature vector, the respiratory feature vector, and the heart rate feature vector to obtain a concentration concatenated feature vector;

[0117] Input the concentration concatenated feature vector into the concentration classification network layer in the pre-trained concentration detection model, and output the current concentration level through the concentration classification network layer.

[0118] Optionally, the calculation module 64 is further configured to:

[0119] Train the concentration detection model;

[0120] Wherein the training of the concentration detection model includes:

[0121] Obtain a concentration sample data set; wherein, each concentration training sample in the concentration sample data set is associated with a concentration level label, and the concentration level label is determined according to the body movement data, the respiratory data, and the heart rate data in the concentration training sample;

[0122] Based on the concentration training sample, form sample input data, and input the sample input data into an initial concentration detection model containing initial parameters;

[0123] Extract the body movement features, respiratory features, and heart rate features in the concentration training sample through the initial concentration detection model;

[0124] During the iterative training process, the recognition result of the concentration detection model in each iterative training is obtained according to the body movement feature, the breathing feature, and the heart rate feature, and the loss value of each iteration is determined according to the matching degree between the recognition result and the concentration level label;

[0125] When the loss value of the current iteration reaches the preset iteration termination condition, the concentration detection model after the iteration termination is determined as the pre-trained concentration detection model.

[0126] Optionally, the calculation module 64 is further configured to:

[0127] Receive the confirmation information of the concentration of the terminal device in the current time period;

[0128] If there is a modified concentration level in the confirmation information, use the modified concentration level as the current concentration level; and / or if there is a modified concentration value in the confirmation information, use the modified concentration value as the current concentration value.

[0129] Optionally, the calculation module 64 is further configured to:

[0130] Receive the confirmation information of the concentration of the terminal device in the current time period;

[0131] If there is a modified concentration level in the confirmation information, form a new concentration training sample with the body movement data, the breathing data, the heart rate data, and the modified concentration level, and add the new concentration training sample to the concentration sample dataset to continue training the concentration detection model.

[0132] Those skilled in the art can understand that Figure 6 the structure of the concentration detection device based on millimeter wave radar does not limit the concentration detection device based on millimeter wave radar, and the various modules can be implemented in whole or in part by software, hardware, and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned modules. In other embodiments, the concentration detection device based on millimeter wave radar may include more or fewer modules than shown in the figure.

[0133] Please refer to Figure 7, On the other hand, an embodiment of the present application further provides an electronic device 10, including a processor 13 and a memory 14. The memory 14 stores a computer program. When the computer program is executed by the processor, the processor 13 is caused to execute the steps of the method for detecting concentration based on a millimeter-wave radar provided in any of the above embodiments of the present application. The electronic device 10 is a device with infrared imaging function, and may include, for example, a computing device (such as a desktop computer, a laptop computer, a tablet computer, a handheld computer, a smart speaker, a server, etc.), a terminal device (such as a mobile phone, etc.), a wearable device (such as a pair of smart glasses or a smart watch), a smart table lamp, or a similar device.

[0134] The processor 13 is a control center that connects various parts of the entire computer device through various interfaces and lines. By running or executing software programs and / or modules stored in the memory 14, and by calling data stored in the memory 14, it executes various functions of the computer device and processes data. Optionally, the processor 13 may include one or more processing cores; preferably, the processor 13 may integrate an application processor and a modulation / demodulation processor. Among them, the application processor mainly processes the operating system, user interfaces, and application programs, etc., and the modulation / demodulation processor mainly processes wireless communication. It can be understood that the above modulation / demodulation processor may not be integrated into the processor 13.

[0135] The memory 14 can be used to store software programs and modules. The processor 13 executes various functional applications and data processing by running the software programs and modules stored in the memory 14. The memory 14 mainly includes a program storage area and a data storage area. Among them, the program storage area can store the operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the computer device. In addition, the memory 14 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. Correspondingly, the memory 14 may also include a memory controller to provide the processor 13 with access to the memory 14.

[0136] On the other hand, an embodiment of the present application further provides a storage medium storing a computer program. When the computer program is executed by a processor, the processor is caused to execute the steps of the method for detecting concentration based on a millimeter-wave radar provided in any of the above embodiments of the present application.

[0137] Those of ordinary skill in the art can understand that all or part of the processes in the methods provided in the above embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0138] As mentioned above, the above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered by the protection scope of the present invention. The protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A concentration detection method based on millimeter wave radar, characterized in that: Applied in an electronic device, wherein the electronic device is provided with a millimeter wave radar sensor, including: Controlling the millimeter wave radar sensor to send an electromagnetic wave signal to a target space region, and receiving an echo signal reflected by the target space region; Determining whether there is a human body in the target space area according to the echo signal; When it is determined according to the echo signal that a human body exists in the target space area, extracting concentration detection data of the human body based on the echo signal, the concentration detection data including body movement data, breathing data and heart rate data; Based on the concentration detection data, and using a first concentration calculation method, the current concentration value of the human body in the current time period is calculated, wherein the current concentration value of the human body in the current time period is calculated based on the concentration detection data and using the first concentration calculation method, including: extracting body motion feature data, breathing feature data and heart rate feature data based on the concentration detection data, wherein the body motion feature data is used to indicate data of posture changes of the human body in the current time period; the breathing feature data is used to indicate data of breathing fluctuations in the current time period, and the heart rate feature data is used to indicate data of the human heart rate in the current time period, and calculating the weighted sum of the body motion feature data and a first weight coefficient, the breathing feature data and a second weight parameter, and the heart rate feature data and a third weight parameter; Based on the concentration detection data, and using a second concentration calculation method, calculate a current concentration level of the human body in a current time period, wherein the current concentration level is input data of a pre-trained concentration detection model formed based on the body movement data, the breathing data, and the heart rate data, and is a result output by the concentration detection model; When the current concentration value matches the current concentration level, the current concentration value and the current concentration level are used as current concentration data; when the current concentration value does not match the current concentration level, and the mismatch satisfies a preset condition, a prompt message is sent to a terminal device connected to the electronic device to request the terminal device to confirm the concentration within the current time period; the weighted sum of the body motion feature data and the first weight coefficient, the breathing feature data and the second weight parameter, and the heart rate feature data and the third weight parameter is calculated, including: The current concentration value is calculated using a concentration calculation formula, wherein the concentration calculation formula is as follows: Where V represents the current concentration value, α, β and γ are the first weight coefficient, the second weight parameter and the third weight parameter respectively, wherein the first weight coefficient represents the contribution of body motion feature data to the calculation of concentration, the second weight coefficient represents the contribution of heart rate feature data to the calculation of concentration, and the third weight coefficient represents the contribution of breathing feature data to the calculation of concentration, d i represents the body movement value of the i-th second in the current time period, n represents the total number of seconds in the current time period, h(i) represents the heart rate value of the i-th second, N represents the number of breaths in the current time period, x(i) represents the number of breaths in the i-th minute in the current time period, and μ represents the average number of breaths per minute.

2. The concentration detection method based on millimeter wave radar as claimed in claim 1, characterized in that: The calculating the current concentration value of the human body based on the concentration detection data and using the first concentration calculation method further includes: Calculating the first weight coefficient, the second weight parameter and the third weight parameter; The calculating of the first weight coefficient, the second weight parameter and the third weight parameter comprises: Acquire multiple sets of different sample feature data, and form a feature sample matrix with the multiple sets of sample feature data; each set of sample feature data includes body movement sample feature data, breathing sample feature data and heart rate sample feature data, where Represents the feature data of body motion samples, Represents the characteristic data of heart rate samples, Represents breath sample characteristic data; Performing mean processing on the feature sample matrix to obtain a processed matrix, and generating a correlation matrix based on the processed matrix; Based on the correlation matrix, the grey correlation degree r of each sample feature is calculated. j ,in Among them A tj represents the data in the tth row and jth column of the association matrix, j represents the jth sample feature, and G represents the total number of groups; Grey relational degree r based on the characteristics of each sample j , respectively calculate the first weight coefficient, the second weight parameter and the third weight parameter, wherein the first weight coefficient Among them, r1 represents the grey correlation degree corresponding to the body movement sample feature, r2 represents the grey correlation degree corresponding to the heart rate sample feature, and r3 represents the grey correlation degree corresponding to the breathing sample feature.

3. The concentration detection method based on millimeter wave radar as claimed in claim 1, characterized in that: The calculating, based on the concentration detection data and using a second concentration calculation method, the current concentration level of the human body in the current time period comprises: forming input data of a pre-trained concentration detection model based on the body motion data, the breathing data and the heart rate data, wherein the pre-trained concentration detection model comprises a first convolutional layer, a second convolutional layer and a third convolutional layer, wherein the first convolutional layer is used to obtain a body motion feature vector from the body motion data, the second convolutional layer is used to obtain a breathing feature vector from the breathing data, and the third convolutional layer is used to obtain a heart rate feature vector from the heart rate data; Inputting body motion input data formed by the body motion data into the first convolution layer to obtain the body motion feature vector, inputting breathing input data formed by the breathing data into the second convolution layer to obtain the breathing feature vector, and inputting heart rate input data formed by the heart rate data into the third convolution layer to obtain the heart rate feature vector; splicing the body movement feature vector, the breathing feature vector and the heart rate feature vector to obtain a concentration splicing feature vector; The concentration concatenated feature vector is input into a concentration classification network layer in a pre-trained concentration detection model, and the current concentration level is output through the concentration classification network layer.

4. The concentration detection method based on millimeter wave radar as claimed in claim 3, characterized in that: The method further comprises: Training the concentration detection model; The training of the concentration detection model comprises: Acquire a concentration sample data set; wherein each concentration training sample in the concentration sample data set is associated with a concentration level label, and the concentration level label is determined according to body movement data, breathing data, and heart rate data in the concentration training sample; Forming sample input data based on the concentration training sample, and inputting the sample input data into an initial concentration detection model including initial parameters; Extracting body movement features, breathing features and heart rate features in the concentration training sample through the initial concentration detection model; During the iterative training process, the recognition result of the concentration detection model in each iterative training is obtained according to the body movement characteristics, breathing characteristics and heart rate characteristics, and the loss value of each iteration is determined according to the matching degree between the recognition result and the concentration level label; When the loss value of the current iteration reaches a preset iteration termination condition, the concentration detection model after the iteration termination is determined as the pre-trained concentration detection model.

5. The concentration detection method based on millimeter wave radar as claimed in claim 1, characterized in that: The method further comprises: receiving confirmation information of the terminal device on the concentration in the current time period; If the confirmation information contains a modified concentration level, the modified concentration level is used as the current concentration level; and / or if the confirmation information contains a modified concentration value, the modified concentration value is used as the current concentration value.

6. The concentration detection method based on millimeter wave radar as claimed in claim 4, characterized in that: The method further comprises: receiving confirmation information of the terminal device on the concentration in the current time period; If the confirmation information contains a modified concentration level, the body movement data, the breathing data, the heart rate data and the modified concentration level are used to form a new concentration training sample, and the new concentration training sample is added to the concentration sample data set to continue training the concentration detection model.

7. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the concentration detection method based on millimeter-wave radar as described in any one of claims 1 to 6 is implemented.

8. An electronic device, characterized in that: It includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the concentration detection method based on millimeter-wave radar as described in any one of claims 1 to 6.

Images