Learning efficiency detection method and electronic device based on millimeter wave radar

Abstract

The present invention discloses a learning efficiency detection method and electronic device based on millimeter wave radar, the method comprising: obtaining target echo data with a target duration detected by a millimeter wave radar sensor to a target space area; determining input data to be detected based on the target echo data; inputting the input data to be detected into a pre-trained learning efficiency detection model, the learning efficiency detection model outputs a first feature data indicating that the user is in an away state and a second feature data indicating different concentration levels through a learning feature extraction network, and performs learning efficiency detection based on the first feature data and the second feature data through a learning efficiency calculation network, and outputs a learning efficiency score corresponding to the input data to be detected; wherein the first feature data includes the away duration, and the second feature data includes the duration corresponding to different concentration levels.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a learning efficiency detection method and electronic equipment based on millimeter wave radar. Background Art

[0002] In an educational environment, students’ learning efficiency directly affects learning outcomes. By testing learning efficiency, teachers can understand students’ learning status and take corresponding measures to improve teaching quality. Therefore, by monitoring students’ learning efficiency in real time during the user’s learning process, users can be reminded in real time when their learning efficiency is not high to help them improve their learning efficiency.

[0003] In the prior art, the user's learning status is generally identified by detecting the user's sitting posture or facial expression through images. However, due to the large number of interfering background factors in the image, the recognition results are prone to self-contradiction or unstable fluctuations in the previous and next frames, affecting the accuracy of the final determination of the learning efficiency. In the prior art, learning efficiency is also detected through empirical formulas. Detection based on empirical formulas is subject to more interference from human factors and cannot correctly reflect the user's learning status, so parents cannot understand the user's learning efficiency. Summary of the invention

[0004] In order to solve the existing technical problems, the present invention provides a learning efficiency detection method and electronic equipment based on millimeter wave radar, which can improve the accuracy of learning efficiency detection.

[0005] In a first aspect, a learning efficiency detection method based on millimeter-wave radar is provided, comprising: acquiring target echo data with a target duration detected by a millimeter-wave radar sensor to a target space area; determining input data to be detected based on the target echo data; inputting the input data to be detected into a pre-trained learning efficiency detection model, the learning efficiency detection model outputs a first feature data indicating that the user is away and a second feature data indicating different concentration levels through a learning feature extraction network, and performs a learning efficiency detection based on the first feature data and the second feature data through a learning efficiency calculation network, and outputs a learning efficiency score corresponding to the input data to be detected; wherein the first feature data includes the absence duration, and the second feature data includes the duration corresponding to different concentration levels.

[0006] In a second aspect, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the learning efficiency detection method based on millimeter wave radar as described in any embodiment of the present application.

[0007] The embodiment of the present application obtains target echo data, forms input data to be detected based on the target echo data, extracts first feature data representing the leaving state and second feature data representing different concentration levels in the input data to be detected through a pre-trained learning efficiency detection model, and can also output a learning efficiency score corresponding to the input data to be detected through the pre-trained learning efficiency detection model; compared with the empirical formula in the prior art, the learning efficiency detection model trained by the training method can learn the features in the sample data set, so that it can accurately detect the learning features in the input data, and can directly output the learning efficiency score based on the detected learning features, so that the pre-trained learning efficiency detection model can not only output the concentration level of the learning state within the target time, but also can directly output the learning efficiency score based on the trained learning efficiency detection model, which is convenient for intuitively understanding the students' learning situation, and also improves the accuracy of learning efficiency detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 This is a diagram of an application environment of a learning efficiency detection method based on millimeter wave radar in an embodiment;

[0009] Figure 2 is a schematic diagram of an electronic device in an embodiment being a smart desk lamp;

[0010] Figure 3 is an application environment diagram of a learning efficiency detection method based on millimeter wave radar in another embodiment;

[0011] Figure 4 is a flow chart of a method for detecting learning efficiency based on millimeter wave radar in one embodiment;

[0012] Figure 5 It is a flowchart of training a learning efficiency detection model in a learning efficiency detection method based on millimeter wave radar in one embodiment;

[0013] Figure 6 Schematic diagram of the proportions of different concentration levels in one embodiment.

[0014] Figure 7 It is a schematic diagram of displaying recorded data in groups in one embodiment.

[0015] Figure 8 is a schematic diagram of a learning efficiency detection device based on millimeter wave radar in one embodiment;

[0016] Fig. 9 FIG. 1 is a schematic diagram of an electronic device in an embodiment. DETAILED DESCRIPTION

[0017] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments of the specification.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the scope of protection of the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0019] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it should be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0020] See also Figure 1 , is an application environment diagram of a learning efficiency detection method based on millimeter wave radar in an embodiment. The learning efficiency detection method based on millimeter wave radar is applied to an electronic device 10, and the electronic device 10 includes a millimeter wave radar sensor 12 and a processor 13. The millimeter wave radar sensor 12 is used to send electromagnetic wave signals to the target space area. Since there are multiple targets such as human bodies and other objects in the target space area, these multiple targets will reflect electromagnetic wave signals, and the millimeter wave radar sensor 12 can receive echo signals reflected by multiple targets. The processor 13 detects the learning efficiency of the user in front of the electronic device based on the echo signals detected by the millimeter wave radar sensor 12.

[0021] The millimeter wave radar sensor 12 is a radar system that uses electromagnetic waves in the millimeter wave band to detect targets. The wavelength of the millimeter wave ranges between 1 mm and 10 mm, which is between microwaves and terahertz waves. The millimeter wave radar sensor acquires information about the target by emitting millimeter wave signals and receiving signals reflected by the target. The target information includes but is not limited to the target's body movement data, the target's breathing data and heart rate data in its vital signs.

[0022] The processor 13 may be one or more processors. When there are multiple processors 13, the multiple processors may be integrated on one chip or independently arranged on each chip. The electronic device 10 is a device equipped with a millimeter wave radar sensor, for example, it may include a computing device (for example, a desktop computer, a laptop computer, a tablet computer, a handheld computer, a smart speaker, a server, etc.), a terminal device (for example, a mobile phone, etc.), a wearable device (for example, a pair of smart glasses or a smart watch), various infrared imaging devices, a smart desk lamp or similar devices.

[0023] like Figure 2 As shown, Figure 2Schematic diagram of an electronic device in an embodiment; the electronic device 10 is a smart desk lamp, in which a millimeter wave radar sensor 12 is provided. The smart desk lamp can be placed in the area in front of the user, and on the premise of providing the user with lighting function, the user's learning efficiency can also be detected, so that the user's learning efficiency can be unconsciously monitored without adding wearable devices. When the smart desk lamp is in the on state, that is, when providing lighting function, the millimeter wave radar sensor 12 sends an electromagnetic wave signal to the target space area corresponding to its installation position, and according to the echo signal reflected by the human body, the body movement data, breathing data and heart rate data of the human body can be obtained, and based on the acquired data, the user's learning efficiency is automatically detected.

[0024] like Figure 3 As shown, it is an application environment diagram of a learning efficiency detection method based on millimeter wave radar in another embodiment. The electronic device 10 can also be connected to the terminal device 20 for communication, and the electronic device 10 can send the detected user's concentration to the terminal device 20. For example, when a child is doing homework, he turns on the smart desk lamp, and the smart desk lamp automatically detects the child's concentration during the homework process, and sends the report to the parent's terminal device 20 in the form of a report based on the detected concentration data, so that the parent can understand the child's learning situation in real time without having to keep an eye on the child all the time.

[0025] See also Figure 4 , is a flow chart of a learning efficiency detection method based on millimeter wave radar provided in an embodiment of the present application. The learning efficiency detection method based on millimeter wave radar is applied to an electronic device, and the learning efficiency detection method based on millimeter wave radar includes the following steps:

[0026] S11. Acquire target echo data with a target duration detected by a millimeter wave radar sensor in a target space area.

[0027] In this embodiment, the millimeter wave radar sensor is controlled to send an electromagnetic wave signal to the target space area, and receive the echo signal reflected by the target space area. The target space area indicates the area to which the signal of the millimeter wave radar sensor can be transmitted. Due to the installation position of the millimeter wave radar sensor in the electronic device, the target space area is related to the installation position and the parameters of the millimeter wave radar sensor itself. When the electronic device is in the turned-on state, the millimeter wave radar sensor is controlled to send an electromagnetic wave signal.

[0028] The target duration may be a preset time length. After the electronic device is turned on, each preset target duration executes the learning efficiency detection method based on millimeter wave radar provided in the embodiment of the present application to calculate the learning efficiency once. For example, the target duration may be 15 minutes, half an hour, 1 hour, and so on. The target echo data is the echo data detected within the target duration. By analyzing the target echo data, the user's learning efficiency can be detected.

[0029] S12. Determine input data to be detected based on the target echo data.

[0030] In this embodiment, since the target echo data is data detected by the millimeter wave radar, the target echo data has noise and background data, and the noise and background data need to be processed to reduce the interference of subsequent detection. It is also necessary to perform some conversion operations on the processed echo data to obtain discrete echo signals, and use the obtained discrete echo signals as input data to be detected.

[0031] S13. Input the input data to be detected into a pre-trained learning efficiency detection model. The learning efficiency detection model outputs, through a learning feature extraction network, first feature data indicating that the user is away and second feature data indicating different concentration levels. A learning efficiency detection is performed based on the first feature data and the second feature data through a learning efficiency calculation network, and a learning efficiency score corresponding to the input data to be detected is output.

[0032] In the present embodiment, the learning efficiency detection model includes a learning feature extraction network and a learning efficiency calculation network, the learning feature extraction network is connected to the learning efficiency calculation network, and the output data of the learning feature extraction network is used as the input data of the learning efficiency calculation network. Wherein the learning efficiency detection model is obtained based on the training sample set training, and during the training process, the learning efficiency detection model can learn the features in various training samples, so after the training is completed, the learning feature extraction network can directly extract the first feature data of the user in the away state and the second feature data when the user is in the learning state from the input data to be detected, the first feature data can represent the departure time of the departure state, and the second feature data can represent the duration corresponding to different degrees of concentration in the learning state. Wherein the degree of concentration includes low, medium, high, etc. The first feature data and the second feature data are used as the input of the learning efficiency calculation network, and the corresponding learning efficiency score is output. For example, the input data to be detected is obtained based on the target echo data of 15 minutes, then after extraction by the learning feature extraction network, the departure time of 3 minutes, the duration corresponding to the medium concentration is 5 minutes, the duration corresponding to the low concentration is 1 minute, and the duration corresponding to the high concentration is 6 minutes. Then, the time of leaving for 3 minutes, the time corresponding to medium concentration for 5 minutes, the time corresponding to low concentration for 1 minute, and the time corresponding to high concentration for 6 minutes are used as the input data of the learning efficiency calculation network, and the corresponding learning efficiency score is directly output.

[0033] Therefore, the learning efficiency detection model can not only output the concentration level of the learning state within the target time, but also directly output the learning efficiency score based on the trained learning efficiency detection model, which is convenient for intuitively understanding the students' learning situation.

[0034] In the above embodiment, target echo data is obtained, input data to be detected is formed based on the target echo data, and first feature data representing the leaving state and second feature data representing different concentration levels in the input data to be detected are extracted through a pre-trained learning efficiency detection model. In addition, the pre-trained learning efficiency detection model can output the learning efficiency score corresponding to the input data to be detected. Compared with the empirical formula in the prior art, the learning efficiency detection model trained by the training method can learn the features in the sample data set, so that it can accurately detect the learning features in the input data, and can directly output the learning efficiency score based on the detected learning features. In this way, the pre-trained learning efficiency detection model can not only output the concentration level of the learning state within the target time, but also directly output the learning efficiency score based on the trained learning efficiency detection model, which is convenient for intuitively understanding the students' learning situation, and also improves the accuracy of learning efficiency detection.

[0035] In some embodiments, determining the input data to be detected based on the target echo data includes:

[0036] The target echo data is preprocessed to obtain the input data to be detected, wherein the preprocessing includes at least one of the following: static clutter elimination, mixer-based processing operation, analog-to-digital converter-based sampling operation, and fast-time Fourier transform-based transformation operation.

[0037] First, the target echo data is preprocessed to obtain the input data to be detected, wherein the preprocessing operation includes but is not limited to static clutter elimination, mixer-based processing operations, analog-to-digital converter-based sampling operations, fast-time Fourier transform-based transformation operations, etc. Among them, static clutter elimination is performed on the input data to be detected, and static clutter elimination processing can be performed to reduce the influence of clutter signals generated by large static objects indoors on the target echo signal. The processing operation based on the mixer mixes the transmission signal with the echo signal to obtain a signal with a new frequency, which is an intermediate frequency signal. The frequency and phase of the echo signal and the transmission signal are different, so the distance, speed, angle and other data can be extracted and analyzed from the superimposed intermediate frequency signal. The sampling operation based on the analog-to-digital converter is used to sample the processed echo signal, and the sampled signal is Fourier transformed to obtain a discrete echo signal x(m,n) containing distance dimension and slow time dimension information, wherein m represents the slow time dimension, which is the mth pulse echo, and n represents the distance dimension, which is the nth distance unit.

[0038] In the above embodiment, by preprocessing the target echo data, clutter interference signals can be eliminated, and sampling operations and transformation operations are performed at the same time to obtain discrete echo signals including multiple information, which facilitates the subsequent extraction of more feature information and improves the accuracy of learning efficiency detection.

[0039] In some embodiments, Figure 5 As shown, Figure 5 The flowchart of training a learning efficiency detection model in a learning efficiency detection method based on millimeter wave radar in an embodiment, wherein the method further includes:

[0040] S51. Obtain a training sample set.

[0041] In this embodiment, each training sample in the training sample set includes leaving sample data in the leaving state, learning sample data in the learning state, a learning efficiency score label corresponding to the training sample, and a concentration level label corresponding to the learning sample data. In an optional implementation, multiple electronic devices can be connected to a server, and the server can collect data from multiple electronic devices, and then collect training sample sets based on the data of multiple electronic devices. In this way, the training sample set can include learning status data of multiple student users, diversifying the training sample set, so that in the subsequent training process, more abundant feature data can be provided. The concentration level labels include low, medium, high, etc. More degree levels can also be set as needed.

[0042] S52: constructing an initial learning efficiency detection model, wherein the initial learning efficiency detection model includes a learning feature extraction network and a learning efficiency calculation network.

[0043] In this embodiment, the learning feature extraction network can be a deep neural network, and the learning feature extraction network includes an input layer, a hidden layer, a fully connected layer and an output layer. The input layer is used to receive input data and input the input data into the network, for example, it can be a discrete echo signal, etc. The hidden layer can be multiple hidden layers, each layer consisting of multiple neurons. These layers learn high-level features of the data through nonlinear activation functions (such as ReLU). Fully connected layers: In the last few layers of the network, there are usually fully connected layers, which map the learned features to the final categories. The output layer is usually a softmax layer, which converts the output of the network into a probability distribution, indicating the probability that the input data belongs to each category. The learning efficiency calculation network can also be a deep neural network.

[0044] S53. Sample input data is obtained based on the training sample set, and the sample input data is input into a learning efficiency detection model in iterative training for iterative training. The learning feature extraction network in the iterative training outputs the concentration level corresponding to the learning sample data, and outputs the learning sample duration and the time to leave the sample corresponding to different concentration levels.

[0045] In an optional implementation, the concentration level label corresponding to the learning sample data is the concentration level label output by the trained concentration classification network model. During the training process, the learning sample data in the sample input data is input into the trained concentration classification network model, and the concentration level label corresponding to the learning sample data is output.

[0046] In this embodiment, during the training process, since the trained concentration classification network model is a model that has been trained in advance based on the concentration training data set, it can accurately output the degree of concentration. When training the learning efficiency detection model, the trained concentration classification network model is equivalent to a teacher model, and the sample input data is input into the trained concentration classification network model and the learning feature extraction network at the same time, so that the concentration degree label corresponding to the sample input data can be obtained through the trained concentration classification network model. In this way, during the training process, the learning feature extraction network is equivalent to a student model, and the concentration degree label output by the teacher model is used as the training target. The learning feature extraction network in the learning efficiency detection model is trained, which can accelerate the convergence of the learning efficiency detection model and reduce the training time.

[0047] In an optional implementation, obtaining a trained concentration classification network model;

[0048] The method of obtaining the trained concentration classification network model includes:

[0049] Acquire a concentration training data set, wherein each training data in the concentration training data set includes heart rate sample data and a concentration degree label corresponding to the heart rate sample data;

[0050] Construct an initial concentration classification network model;

[0051] The concentration classification network model is trained based on the concentration training data set to obtain a trained concentration classification network model.

[0052] In this embodiment, the concentration training data set can be obtained by collecting heart rate data of multiple users. In order to enrich the concentration training data set, heart rate sample data at different concentration levels can be collected, so that during the training process, the concentration classification network model can learn the characteristics of heart rate samples at different concentration levels. The input data is obtained from the concentration training data set, and the input data is input into the concentration classification network model, and the loss value in each iteration process is calculated until the concentration classification network model converges to obtain the trained concentration classification network model.

[0053] S54, the learning efficiency calculation network in the iterative training calculates the current learning efficiency score in the current iteration based on the learning sample duration and the leaving sample duration corresponding to different concentration levels.

[0054] S55. During the iteration process, the current loss value in the current iteration is calculated based on the loss function, wherein the current loss value includes the concentration degree loss value between the concentration degree corresponding to the learning sample data output by the learning feature extraction network in the current iteration and the concentration degree label corresponding to the learning sample data, and the score loss value between the current learning efficiency score and the learning efficiency score label.

[0055] In some optional implementations, calculating the current loss value in the current iteration based on the loss function during the iteration includes:

[0056] Based on the first loss function, calculating the concentration degree loss value;

[0057] Based on the second loss function, the score loss value is calculated; wherein the loss function is:

[0058]

[0059] in, is the loss function, is the first loss function, is the second loss function, , are weight coefficients respectively. The first loss function and the second loss function can be the same or different. For example, the first loss function is the mean square error loss function, and the second loss function is the cross entropy loss function, etc.

[0060] S56. Determine whether the current iteration satisfies the iteration termination condition based on the current loss value; if the current iteration satisfies the iteration termination condition, use the learning efficiency detection model after stopping the iteration as the pre-trained learning efficiency detection model; if the current iteration does not meet the iteration termination condition, continue to process the training sample set to obtain sample input data, and iteratively train the learning efficiency detection model.

[0061] In the above embodiment, the learning efficiency detection model is trained and learned based on each training sample in the training sample set, and the learning efficiency score label corresponding to each training sample and the concentration level label corresponding to the learning sample data. The learning efficiency detection model can use the concentration level label corresponding to each training sample as the training target, thereby outputting the concentration level corresponding to the learning sample data, so that the learning efficiency detection model can accurately calculate the learning efficiency score in the future. During the training process, the learning efficiency detection model can use the learning efficiency score label corresponding to each training sample as the training target, thereby outputting an accurate learning efficiency score, thereby improving the accuracy of learning efficiency detection.

[0062] In some embodiments, the method further comprises at least one of the following:

[0063] Sending the learning efficiency score corresponding to the target duration to a terminal device communicating with the electronic device;

[0064] Displaying the status identifiers of each time period within the target duration in a time series order, wherein the status identifiers include the exit status identifier and the identifiers of different concentration levels in the learning state;

[0065] The proportion of different concentration levels in the learning state within the target duration and the duration corresponding to the different concentration levels are displayed.

[0066] In this embodiment, since the learning feature extraction network can extract the first feature data and the second feature data within the target duration, and the input data to be detected is data in a time series, each feature data corresponds to its own time period. Therefore, the state identifiers within each extracted time period can be displayed in time series order, so that the user can intuitively observe the user's learning status within the target duration. For example, if the target duration is 15 minutes, the time sequence shows that the state from the beginning to the 3rd minute is the leaving state, the state from the 3rd minute to the 8th minute is the medium concentration state, the state from the 8th minute to the 9th minute is the low concentration state, and the state from the 9th minute to the 15th minute is the high concentration state. The time series status display allows parents to understand the progress of the learning state. Figure 6 The figure is a schematic diagram of the proportion of different concentration levels in one embodiment, which shows the proportion of different concentration states and the duration corresponding to different concentration states, so that users can more intuitively understand the situation of each learning state, for example Figure 6 In the app, the data of deep concentration, moderate concentration and shallow concentration are displayed in a ring chart, and the duration of various concentration states is displayed, so that parents can intuitively understand the learning situation in a certain period of time.

[0067] In the above embodiment, the status identifiers of each time period are displayed in time series, and / or the proportion of different concentration levels in the learning state within the target time period and the duration corresponding to different concentration levels are displayed, which can help parents to intuitively understand the students' learning situation in a certain time period.

[0068] In some embodiments, the method further comprises:

[0069] Based on the second feature data, calculate the concentration score; obtain the historical target time period sent by the terminal device, obtain multiple recorded data in the historical target time period, and display the multiple recorded data;

[0070] The display of multiple recorded data includes at least one of the following:

[0071] Displaying the changing trend of each data in the recorded data on a time axis, wherein the recorded data includes a learning efficiency score and a concentration degree score;

[0072] The recorded data for each time and the corresponding time period are displayed in groups.

[0073] In this embodiment, the concentration score can also be calculated based on the second characteristic data representing the learning state within the target time length, for example, different scores and weights are assigned to different concentration levels, and the concentration score can be calculated. A time selection control is displayed on the user interface of the terminal device, and parents can use the time selection control to select the historical target time period to be viewed and send it to the electronic device. The electronic device obtains multiple recorded data within the historical target time period from the memory and displays it. In an optional implementation, for any type of data, the changing trend of the data can be displayed on a timeline. For example, taking the learning efficiency score as an example, the learning efficiency score of the historical target time period is displayed, so that parents can intuitively observe whether the child’s learning efficiency in the historical target time period is improving or regressing. In an optional implementation, such as Figure 7 As shown, Figure 7 This is a schematic diagram of displaying recorded data in groups in an embodiment, where each recorded data can be recorded in groups.

[0074] In the above embodiment, the changing trend of each data in the recorded data is displayed on a time axis, and / or each recorded data and the corresponding time period are displayed in groups, so that parents can intuitively understand the learning situation of students in a certain time period.

[0075] On the other hand, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the learning efficiency detection method based on millimeter wave radar described in any embodiment of the present application.

[0076] Among them, in the computer program product, an optional implementation form of the program module architecture of the computer program that implements each step of the learning efficiency detection method based on millimeter wave radar can be a learning efficiency detection device based on millimeter wave radar.

[0077] See also Figure 8An embodiment of the present application provides a learning efficiency detection device based on millimeter wave radar, including: an acquisition module 81, used to acquire target echo data with a target duration detected by a millimeter wave radar sensor to a target space area; a determination module 82, used to determine input data to be detected based on the target echo data; a detection module 83, used to input the input data to be detected into a pre-trained learning efficiency detection model, the learning efficiency detection model outputs a first feature data indicating that the user is in an away state and a second feature data indicating different concentration levels through a learning feature extraction network, and performs learning efficiency detection based on the first feature data and the second feature data through a learning efficiency calculation network, and outputs a learning efficiency score corresponding to the input data to be detected; wherein the first feature data includes the away duration, and the second feature data includes the duration corresponding to different concentration levels.

[0078] Optionally, the determination module 82 is further configured to:

[0079] The target echo data is preprocessed to obtain the input data to be detected, wherein the preprocessing includes at least one of the following: static clutter elimination, mixer-based processing operation, analog-to-digital converter-based sampling operation, and fast-time Fourier transform-based transformation operation.

[0080] Optionally, the detection module 83 is further used for:

[0081] Acquire a training sample set, wherein each training sample in the training sample set includes leaving sample data in a leaving state, learning sample data in a learning state, a learning efficiency score label corresponding to the training sample, and a concentration degree label corresponding to the learning sample data;

[0082] Constructing an initial learning efficiency detection model, wherein the initial learning efficiency detection model includes a learning feature extraction network and a learning efficiency calculation network;

[0083] Based on the training sample set, sample input data is obtained, the sample input data is input into the learning efficiency detection model in iterative training for iterative training, and the learning feature extraction network in iterative training outputs the degree of concentration corresponding to the learning sample data, and outputs the learning sample duration and the time to leave the sample corresponding to different degrees of concentration;

[0084] The learning efficiency calculation network in the iterative training calculates the current learning efficiency score in the current iteration based on the learning sample duration and the leaving sample duration corresponding to different concentration levels;

[0085] Calculating a current loss value in the current iteration based on the loss function during the iteration, the current loss value including a concentration loss value between a concentration degree corresponding to the learning sample data output by the learning feature extraction network in the current iteration and a concentration degree label corresponding to the learning sample data, and a score loss value between the current learning efficiency score and the learning efficiency score label;

[0086] Based on the current loss value, it is judged whether the current iteration meets the iteration termination condition; if the current iteration meets the iteration termination condition, the learning efficiency detection model after stopping the iteration is used as the pre-trained learning efficiency detection model; if the current iteration does not meet the iteration termination condition, continue to process the training sample set to obtain sample input data, and iteratively train the learning efficiency detection model.

[0087] Optionally, the concentration level label corresponding to the learning sample data is a concentration level label output by a trained concentration classification network model.

[0088] Optionally, the detection module 83 is further used for:

[0089] Obtain the trained concentration classification network model;

[0090] The method of obtaining the trained concentration classification network model includes:

[0091] Acquire a concentration training data set, wherein each training data in the concentration training data set includes heart rate sample data and a concentration degree label corresponding to the heart rate sample data;

[0092] Construct an initial concentration classification network model;

[0093] The concentration classification network model is trained based on the concentration training data set to obtain a trained concentration classification network model.

[0094] Optionally, the detection module 83 is further used for:

[0095] During the iteration process, the learning sample data in the sample input data is input into the trained concentration classification network model, and the concentration degree label corresponding to the learning sample data is output.

[0096] Optionally, the detection module 83 is further used for:

[0097] Based on the first loss function, calculating the concentration degree loss value;

[0098] Based on the second loss function, the score loss value is calculated; wherein the loss function is:

[0099]

[0100] in, is the loss function, is the first loss function, is the second loss function, , are weight coefficients respectively.

[0101] Optionally, the detection module 83 is further used for:

[0102] Sending the learning efficiency score corresponding to the target duration to a terminal device communicating with the electronic device;

[0103] Displaying the status identifiers of each time period within the target duration in a time series order, wherein the status identifiers include the exit status identifier and the identifiers of different concentration levels in the learning state;

[0104] The proportion of different concentration levels in the learning state within the target duration and the duration corresponding to the different concentration levels are displayed.

[0105] Optionally, the detection module 83 is further used for:

[0106] Based on the second feature data, calculate the concentration score; obtain the historical target time period sent by the terminal device, obtain multiple recorded data in the historical target time period, and display the multiple recorded data;

[0107] The display of multiple recorded data includes at least one of the following:

[0108] Displaying the changing trend of each data in the recorded data on a time axis, wherein the recorded data includes a learning efficiency score and a concentration degree score;

[0109] The recorded data for each time and the corresponding time period are displayed in groups.

[0110] It can be understood by those skilled in the art that Figure 8 The structure of the learning efficiency detection device based on millimeter-wave radar does not constitute a limitation on the learning efficiency detection device based on millimeter-wave radar, and the various modules can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules. In other embodiments, the learning efficiency detection device based on millimeter-wave radar may include more or fewer modules than shown in the figure.

[0111] See also Fig. 9In another aspect of the embodiment of the present application, an electronic device 10 is provided, including a processor 13 and a memory 14, wherein the memory 14 stores a computer program, and when the computer program is executed by the processor, the processor 13 executes the steps of the learning efficiency detection method based on millimeter wave radar provided in any of the above embodiments of the present application. The electronic device 10 is a device with infrared imaging function, for example, it may include a computing device (for example, a desktop computer, a laptop computer, a tablet computer, a handheld computer, a smart speaker, a server, etc.), a terminal device (for example, a mobile phone, etc.), a wearable device (for example, a pair of smart glasses or a smart watch), a smart desk lamp, or a similar device.

[0112] The processor 13 is the control center, which uses various interfaces and lines to connect various parts of the entire computer device, and executes various functions of the computer device and processes data by running or executing software programs and / or modules stored in the memory 14, and calling data stored in the memory 14. Optionally, the processor 13 may include one or more processing cores; preferably, the processor 13 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user pages and application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 13.

[0113] The memory 14 can be used to store software programs and modules. The processor 13 executes various functional applications and data processing by running the software programs and modules stored in the memory 14. The memory 14 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the computer device, etc. In addition, the memory 14 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 14 can also include a memory processor to provide the processor 13 with access to the memory 14.

[0114] On the other hand, an embodiment of the present application further provides a storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of the learning efficiency detection method based on millimeter wave radar provided in any of the above embodiments of the present application.

[0115] Those skilled in the art can understand that all or part of the processes in the methods provided in the above embodiments can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0116] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. The protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A learning efficiency detection method based on millimeter wave radar, characterized in that: Applied in an electronic device, wherein the electronic device is provided with a millimeter wave radar sensor, including: Acquire target echo data with a target duration detected by a millimeter wave radar sensor toward a target space area; Determine input data to be detected based on the target echo data, wherein the input data to be detected includes body movement data, breathing data and heart rate data; The input data to be detected is input into a pre-trained learning efficiency detection model, the learning efficiency detection model outputs a first feature data indicating that the user is in an away state and a second feature data indicating different concentration levels through a learning feature extraction network, and performs learning efficiency detection based on the first feature data and the second feature data through a learning efficiency calculation network, and outputs a learning efficiency score corresponding to the input data to be detected; wherein the first feature data includes the away time, and the second feature data includes the time corresponding to different concentration levels; the learning efficiency detection model is obtained based on training of a training sample set and a loss function, wherein each training sample in the training sample set includes away sample data in an away state, learning sample data in a learning state, a learning efficiency score label corresponding to the training sample, and a concentration level label corresponding to the learning sample data; the loss function includes a first loss function and a second loss function, wherein the first loss function is used to calculate a concentration level loss value between a concentration level corresponding to the learning sample data output by the learning feature extraction network in a training iteration and a concentration level label corresponding to the learning sample data, and the second loss function is used to calculate a score loss value between a current learning efficiency score output by the learning efficiency calculation network in a training iteration and a learning efficiency score label.

2. The method for detecting learning efficiency based on millimeter wave radar according to claim 1, characterized in that: The step of determining the input data to be detected based on the target echo data comprises: The target echo data is preprocessed to obtain the input data to be detected, wherein the preprocessing includes at least one of the following: static clutter elimination, mixer-based processing operation, analog-to-digital converter-based sampling operation, and fast-time Fourier transform-based transformation operation.

3. The method for detecting learning efficiency based on millimeter wave radar according to claim 1, characterized in that: The method further comprises: Obtain a training sample set; Constructing an initial learning efficiency detection model, wherein the initial learning efficiency detection model includes a learning feature extraction network and a learning efficiency calculation network; Based on the training sample set, sample input data is obtained, the sample input data is input into the learning efficiency detection model in iterative training for iterative training, and the learning feature extraction network in iterative training outputs the degree of concentration corresponding to the learning sample data, and outputs the learning sample duration and the time to leave the sample corresponding to different degrees of concentration; The learning efficiency calculation network in the iterative training calculates the current learning efficiency score in the current iteration based on the learning sample duration and the leaving sample duration corresponding to different concentration levels; Calculating a current loss value in the current iteration based on the loss function during the iteration, the current loss value including a concentration loss value between a concentration degree corresponding to the learning sample data output by the learning feature extraction network in the current iteration and a concentration degree label corresponding to the learning sample data, and a score loss value between the current learning efficiency score and the learning efficiency score label; Based on the current loss value, it is judged whether the current iteration meets the iteration termination condition; if the current iteration meets the iteration termination condition, the learning efficiency detection model after stopping the iteration is used as the pre-trained learning efficiency detection model; if the current iteration does not meet the iteration termination condition, continue to process the training sample set to obtain sample input data, and iteratively train the learning efficiency detection model.

4. The method for detecting learning efficiency based on millimeter wave radar as claimed in claim 3, characterized in that: The concentration degree label corresponding to the learning sample data is the concentration degree label output by the trained concentration classification network model.

5. The method for detecting learning efficiency based on millimeter wave radar as claimed in claim 4, characterized in that: The method further comprises: Obtain the trained concentration classification network model; The method of obtaining the trained concentration classification network model includes: Acquire a concentration training data set, wherein each training data in the concentration training data set includes heart rate sample data and a concentration degree label corresponding to the heart rate sample data; Construct an initial concentration classification network model; The concentration classification network model is trained based on the concentration training data set to obtain a trained concentration classification network model.

6. The method for detecting learning efficiency based on millimeter wave radar as claimed in claim 4, characterized in that: The processing based on the training sample set to obtain sample input data, and inputting the sample input data into the learning efficiency detection model in iterative training for iterative training, further includes: During the iteration process, the learning sample data in the sample input data is input into the trained concentration classification network model, and the concentration degree label corresponding to the learning sample data is output.

7. The method for detecting learning efficiency based on millimeter wave radar as claimed in claim 4, characterized in that: Calculating the current loss value in the current iteration based on the loss function during the iteration process includes: Based on the first loss function, calculating the concentration degree loss value; Based on the second loss function, the score loss value is calculated; wherein the loss function is: in, is the loss function, is the first loss function, is the second loss function, , are weight coefficients respectively.

8. The method for detecting learning efficiency based on millimeter wave radar as claimed in claim 1, characterized in that: The method further comprises at least one of the following: Sending the learning efficiency score corresponding to the target duration to a terminal device communicating with the electronic device; Displaying the status identifiers of each time period within the target duration in a time series order, wherein the status identifiers include the exit status identifier and the identifiers of different concentration levels in the learning state; The proportion of different concentration levels in the learning state within the target duration and the duration corresponding to the different concentration levels are displayed.

9. The method for detecting learning efficiency based on millimeter wave radar as claimed in claim 1, characterized in that: The method further comprises: Based on the second feature data, calculate the concentration score; obtain the historical target time period sent by the terminal device, obtain multiple recorded data in the historical target time period, and display the multiple recorded data; The display of multiple recorded data includes at least one of the following: Displaying the changing trend of each data in the recorded data on a time axis, wherein the recorded data includes a learning efficiency score and a concentration degree score; The recorded data for each time and the corresponding time period are displayed in groups.

10. An electronic device, characterized in that: It includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the learning efficiency detection method based on millimeter wave radar as described in any one of claims 1 to 9.

Images