Online study room monitoring method and system based on millimeter wave radar

Abstract

The present invention discloses an online study room monitoring method and system based on millimeter wave radar, the method comprising: obtaining a study room keyword obtained based on a user interface, and sending the study room keyword to a server communicating with the electronic device, so that the server matches the study room keyword with an online virtual study room in a database to obtain study room matching data; obtaining the study room matching data, and based on the study room matching data, determining a target study room, and sending the data to the server; after the user enters the target study room, obtaining the user's learning status data through the millimeter wave radar sensor, and sending the data to the server so that the server generates the real-time online user's study data in the target study room according to the learning status data of each user; obtaining the study data in the target study room, and displaying the study data in the target study room.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an online study room monitoring method and system based on millimeter wave radar. Background Art

[0002] In an educational environment, students’ learning efficiency directly affects learning outcomes. By testing learning efficiency, teachers can understand students’ learning status and take corresponding measures to improve teaching quality. Therefore, by monitoring students’ learning efficiency in real time during the user’s learning process, users can be reminded in real time when their learning efficiency is not high to help them improve their learning efficiency.

[0003] In the prior art, when students study at home, it is impossible to understand the status of students at home. When students are not in a good state of study, it is impossible to train their attention to improve their learning efficiency, and they can only rely on parental supervision. Moreover, when students study at home, they cannot form a group learning atmosphere. Summary of the invention

[0004] In order to solve the existing technical problems, the present invention provides an online study room monitoring method and system based on millimeter wave radar, which can realize a virtual online study room environment and improve the learning efficiency of users.

[0005] In a first aspect, a method for monitoring an online study room based on millimeter-wave radar is provided, which is applied to an electronic device in which a millimeter-wave radar sensor is provided, comprising: obtaining a study room keyword obtained based on a user interface, and sending the study room keyword to a server communicating with the electronic device, so that the server matches the study room keyword with an online virtual study room in a database to obtain study room matching data; obtaining the study room matching data, and determining a target study room based on the study room matching data, and sending the data to the server; after a user enters the target study room, obtaining the user's learning status data through the millimeter-wave radar sensor, and sending the data to the server so that the server generates the study data of the target study room according to the learning status data of each user; obtaining the study data in the target study room, and displaying the study data in the target study room.

[0006] In a second aspect, an online study room monitoring system based on millimeter wave radar is provided, the system comprising the electronic device described in any embodiment of the present application, and a server communicating with the electronic device.

[0007] The embodiment of the present application obtains the study room keyword of the virtual study room that the user wants to enter, and then sends it to the server. The server can match based on the study room keyword to obtain study room matching data, and then determine the target study room with the study room matching data. After the user enters the target study room, the user's learning status data is obtained through the millimeter wave radar sensor, so that the server can obtain the learning status data of each user, thereby obtaining the study data, and sending it to the electronic device to display the study data. In this way, the user can also achieve a virtual online study environment in a home environment, providing a good learning atmosphere, and can also monitor the user's learning status data in real time, so as to improve the user's learning efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 This is a diagram of the application environment of an online study room monitoring method based on millimeter wave radar in one embodiment;

[0009] Figure 2 is a schematic diagram of an electronic device in an embodiment being a smart desk lamp;

[0010] Figure 3 This is a diagram of an application environment of an online study room monitoring method based on millimeter wave radar in another embodiment;

[0011] Figure 4 Flow chart of an online study room monitoring method based on millimeter wave radar in one embodiment;

[0012] Figure 5 is a schematic diagram of a virtual study room in one embodiment;

[0013] Figure 6 This is a schematic diagram of learning status data when a user is in a self-study state in one embodiment;

[0014] Figure 7 Schematic diagram of an online study room monitoring device based on millimeter wave radar in one embodiment;

[0015] Figure 8 FIG. 1 is a schematic diagram of an electronic device in an embodiment. DETAILED DESCRIPTION

[0016] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0017] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the scope of protection of the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0018] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it should be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0019] See also Figure 1 , is an application environment diagram of an online study room monitoring method based on millimeter wave radar in one embodiment. The online study room monitoring method based on millimeter wave radar is applied to an electronic device 10, and the electronic device 10 includes a millimeter wave radar sensor 12 and a processor 13. The millimeter wave radar sensor 12 is used to send electromagnetic wave signals to the target space area. Since there are multiple targets such as human bodies and other objects in the target space area, these multiple targets will reflect electromagnetic wave signals, and the millimeter wave radar sensor 12 can receive the echo signals reflected by the multiple targets. The processor 13 detects the learning status of the user in front of the electronic device based on the echo signals detected by the millimeter wave radar sensor 12. In some embodiments, the electronic device 10 may also include a display device, which provides a user interface to the user through the display device. The display device can be set in the electronic device 10, or it can exist independently of the electronic device 10.

[0020] The millimeter wave radar sensor 12 is a radar system that uses electromagnetic waves in the millimeter wave frequency band to detect targets. The wavelength of the millimeter wave ranges between 1 mm and 10 mm, which is between microwaves and terahertz waves. The millimeter wave radar sensor acquires information about the target by emitting millimeter wave signals and receiving signals reflected by the target. The target information includes but is not limited to the target's body movement data, sitting posture data, and breathing data and heart rate data in the target's vital signs.

[0021] The processor 13 may be one or more processors. When there are multiple processors 13, the multiple processors may be integrated on one chip or independently arranged on each chip. The electronic device 10 is a device equipped with a millimeter wave radar sensor, for example, it may include a computing device (for example, a desktop computer, a laptop computer, a tablet computer, a handheld computer, a smart speaker, a server, etc.), a terminal device (for example, a mobile phone, etc.), a wearable device (for example, a pair of smart glasses or a smart watch), various infrared imaging devices, a smart desk lamp or similar devices.

[0022] like Figure 2 As shown, Figure 2Schematic diagram of an electronic device in an embodiment; the electronic device 10 is a smart desk lamp, in which a millimeter-wave radar sensor 12 is provided. The smart desk lamp can be placed in the area in front of the user, and can detect the user's learning status while providing the user with lighting function, so that the user's learning status can be unconsciously monitored without adding wearable devices. When the smart desk lamp is in the on state, that is, when providing lighting function, the millimeter-wave radar sensor 12 sends an electromagnetic wave signal to the target space area corresponding to its installation position, and according to the echo signal reflected by the human body, the body movement data, sitting posture data, breathing data and heart rate data of the human body can be obtained, and the user's learning status can be automatically detected based on the acquired data.

[0023] like Figure 3 As shown, it is an application environment diagram of an online study room monitoring method based on millimeter wave radar in another embodiment. The electronic device 10 can also be connected to the server 20 for communication. The electronic device 10 sends the acquired study room keywords to the server 20. The server 20 matches the online virtual study room in the database according to the study room keywords to obtain the study room matching data. The electronic device 10 determines the target study room based on the study room matching data, and after the user enters the target study room, the user's learning status data is obtained through the millimeter wave radar sensor 12, and sent to the server 20. The server 20 generates the real-time online user's study data in the target study room according to the learning status data of each user in the target study room, and sends it to the electronic device of each user. The electronic device 10 obtains the study data in the target study room and displays it. In some embodiments, the electronic device 10 can also be connected to the terminal device. For example, when a child is doing homework, the smart desk lamp is turned on for use. The smart desk lamp automatically detects the child's learning status during the homework process, and forms a report based on the detected learning status data and sends it to the terminal device of the parent, so that the parent does not need to always stare at the child, and can understand the child's learning situation in real time.

[0024] See also Figure 4 , is a flow chart of an online study room monitoring method based on millimeter wave radar provided in an embodiment of the present application. The online study room monitoring method based on millimeter wave radar is applied to an electronic device, and the online study room monitoring method based on millimeter wave radar includes the following steps:

[0025] S11, obtaining a study room keyword based on the user interface, and sending the study room keyword to a server communicating with the electronic device, so that the server matches the study room keyword with an online virtual study room in a database to obtain study room matching data.

[0026] In this embodiment, the server is connected to multiple electronic devices. For example, if the electronic device is a smart desk lamp based on millimeter wave radar, the server is connected to multiple smart desk lamps. The server can provide multiple virtual study room environments. When the user wants to enter a specific study room, such as Class 6, Grade 3 of School A, or wants to enter a study room of a certain type, the study room keyword can be input through the keyword input control, voice or gesture provided by the user interface. It can be understood that the keyword input control includes but is not limited to input boxes, text boxes, check boxes, drop-down menus, etc. The study room pre-stored in the database is a virtual study room environment. Figure 5 FIG. 1 is a schematic diagram of a virtual study room in an embodiment. Figure 5 It can be seen that the learning environment of the virtual study room is similar to that of the classroom. Figure 5 The one shown in the figure is a study room, and there is currently one online user.

[0027] The study room keyword indicates the keyword of the study room that the user wants to enter, and the study room keyword can be one or more. The electronic device sends the study room keyword to the server, and the server matches the study room keyword with the label data of the virtual study room in the database to obtain the study room matching data that matches the study room keyword. The label data is used to identify the study room, and the label data can indicate the name, purpose, etc. of the study room.

[0028] The study room matching data represents one or more virtual study rooms that match the study room keyword.

[0029] S12. Obtain study room matching data, determine the target study room based on the study room matching data, and send it to the server.

[0030] In this embodiment, the study room indicated in the study room matching data may be one or more study rooms. When the study room matching data indicates multiple study rooms, the study room matching data may be displayed on the user interface, and the user may determine the target study room. The target study room represents the study room that the user wants to enter. The electronic device may send an identifier corresponding to the target study room to the server, so that the server may determine the target study room based on the received identifier. For any study room, an identifier may correspond, and the identifier may uniquely correspond to a study room, and the identifier may include but is not limited to a name, a number, and the like.

[0031] S13. After the user enters the target study room, the user's learning status data is obtained through the millimeter wave radar sensor and sent to the server so that the server generates the self-study data in the target study room according to the learning status data of each user.

[0032] In this embodiment, after the user enters the target study room, at least the following data of the user can be obtained through the millimeter wave radar sensor: heart rate data, breathing data, learning concentration, learning time under different concentration levels, learning efficiency value, learning posture state, etc. In an optional implementation, after the user triggers the entry control of the target study room, it means that the user enters the target study room.

[0033] The millimeter-wave radar sensor is controlled to send electromagnetic wave signals to the target space area and receive the echo signals reflected by the target space area. The target space area indicates the area where the millimeter-wave radar sensor's signal can be transmitted. Due to the installation position of the millimeter-wave radar sensor in the electronic device, the target space area is related to the installation position and the parameters of the millimeter-wave radar sensor itself. When the electronic device is turned on, the millimeter-wave radar sensor is controlled to send electromagnetic wave signals.

[0034] The echo signal is preprocessed to obtain preprocessed echo data, wherein the preprocessing includes at least one of the following: static clutter elimination, mixer-based processing operations, analog-to-digital converter-based sampling operations, and fast-time Fourier transform-based transformation operations. The echo signal is preprocessed to obtain preprocessed echo data, wherein the preprocessing operations include but are not limited to static clutter elimination, mixer-based processing operations, analog-to-digital converter-based sampling operations, and fast-time Fourier transform-based transformation operations. The echo signal performs static clutter elimination, which can reduce the influence of clutter signals generated by large static objects in the room on the target echo signal. The transmission signal and the echo signal are mixed based on the processing operation of the mixer to obtain a signal with a new frequency, which is an intermediate frequency signal. The frequency and phase of the echo signal and the transmission signal are different. Therefore, data such as distance, speed and angle can be extracted and analyzed from the superimposed intermediate frequency signal. The sampling operation based on the analog-to-digital converter is used to sample the processed echo signal, and the sampled signal is Fourier transformed to obtain a discrete echo signal x(m,n) containing distance dimension and slow time dimension information, where m represents the slow time dimension, which is the mth pulse echo, and n represents the distance dimension, which is the nth distance unit. Then extract heart rate data, breathing data, learning posture data, etc. from the discrete echo signal. Based on the learning heart rate data and / or breathing data, the user's learning concentration, learning time under different concentration levels, and learning efficiency value can be determined. The learning posture state can be determined according to the learning posture data.

[0035] In this way, for the target study room, the server can obtain the learning status data of each user, and then generate the self-study data of the target study room based on the learning status data, wherein the self-study data is used to indicate the self-study situation in the study room, wherein the self-study data includes at least one of the following: concentration level, concentration value, learning efficiency value, learning posture status, study time, learning efficiency value ranking, concentration ranking, concentration value ranking, and the proportion of deep concentration.

[0036] S14. Obtain the self-study data in the target self-study room, and display the self-study data in the target self-study room.

[0037] In this embodiment, after the electronic device obtains the self-study data from the server, the self-study data can be displayed on the user interface, for example, Figure 5 The study room shown is in Figure 5 Indicate the length of time each person spends studying on their own. Figure 6 As shown, Figure 6 Schematic diagram of learning status data when a user is in self-study state in one embodiment; the user can also switch to the personal learning status data display interface, which can display the current learning status data, such as Figure 6 As shown, it can display whether the user's study posture is correct, activity status, the name of the current study room, whether there are vacancies in the study room, heart rate data, breathing data, study time, etc.

[0038] In the above embodiment, the study room keyword of the virtual study room that the user wants to enter is obtained and then sent to the server. The server can match based on the study room keyword to obtain the study room matching data, and then determine the target study room with the study room matching data. After the user enters the target study room, the user's learning status data is obtained through the millimeter wave radar sensor, so that the server can obtain the learning status data of each user, thereby obtaining the study data, and sending it to the electronic device to display the study data. In this way, the user can also achieve a virtual online study environment in the home environment, providing a good learning atmosphere, and can also monitor the user's learning status data in real time, so as to improve the user's learning efficiency.

[0039] In some embodiments, the acquiring of the study room keywords based on the user interface includes at least one of the following:

[0040] Acquire a school selection operation and a class selection operation based on a user interface, and use a school identifier corresponding to the school selection operation and a class identifier corresponding to the class selection operation as the study room keyword;

[0041] Based on the user interface, multiple keywords are obtained. When the multiple keywords indicate that there is no corresponding study room identification, historical study room data used by the user is obtained, and label data corresponding to the historical study room data and the multiple keywords are used as the study room keywords.

[0042] In this embodiment, the study room keyword can indicate a specific study room. For example, a school selection control and a class selection control are displayed on the user interface. A study selection operation is obtained through the school selection control to obtain the school ID. A class selection operation is obtained through the class selection control to obtain the class ID. The combination of the school ID and the class ID can determine a specific study room, such as Class 6, Grade 3, School A. In this way, all students in this class can enter the study room of Class 6, Grade 3, School A through the selection method provided on the user interface, which is convenient for students to centrally manage and monitor their learning situation when they study at home.

[0043] In this embodiment, when the user is not sure about the specific study room, multiple keywords can be entered to express their needs for the study room. When multiple keywords are obtained, the historical study room data used by the user can also be obtained, and the label data and multiple keywords corresponding to the historical study room data are used as the study room keywords. Combining the historical usage records and the multiple keywords entered, the study room that the user wants to enter can be matched more accurately. The historical study room data includes the logo corresponding to the historical study room, the length of time spent studying in the study room, and so on.

[0044] In the above embodiment, the school selection operation and the class selection operation can be obtained in the user interface to determine a target study room, or multiple keywords can be entered and combined with historical study room data to determine the study room keywords, so as to more accurately recommend suitable study rooms.

[0045] In some embodiments, determining the target study room based on the study room matching data includes at least one of the following:

[0046] Displaying usage data of each candidate study room in the study room matching data on the user interface, and acquiring a target study room based on each candidate study room in the user interface, wherein the usage data includes at least one of the following: purpose of the study room, establishment time of the study room, overall learning efficiency data of historical users in the study room, and overall learning efficiency data of the current user in the study room, wherein the overall learning efficiency data of the historical users is determined based on the historical usage records of the study room, and the overall learning efficiency data of the current user is determined based on the usage records of the current user in the study room;

[0047] A sorting item for sorting the study rooms is obtained based on the user interface, each candidate study room in the study room matching data is sorted, and a target study room is obtained based on the sorted result.

[0048] In the present embodiment, the overall learning efficiency data reflects the overall learning efficiency level of each user in the study room, for example, the heart rate data and breathing data of each user can be obtained respectively, the learning efficiency value of each user can be calculated, and then the overall learning efficiency data of the learning efficiency value of each user can be averaged. The usage data of each candidate study room is displayed on the user interface, which is convenient for the user to intuitively understand each study room so as to select the required study room. Candidate study rooms can also be sorted based on sorting items, wherein the sorting items can be any one of the usage data. In an optional implementation, for the heart rate data and breathing data of each user, the heart rate data and breathing data of each user can be input into a pre-trained learning efficiency detection model, and the learning efficiency value of each user is output by the pre-trained learning efficiency detection model. Wherein the learning efficiency detection model is obtained based on training data set training, and the training data set includes the heart rate data and breathing data of each sample user, and the learning efficiency value corresponding to each sample user.

[0049] In the above embodiment, the usage data of each candidate study room in the study room matching data can be displayed on the user interface, so that the user can quickly select the required study room from the candidate study rooms. The candidate study rooms can also be sorted based on a certain sorting item, which makes it easier for the user to intuitively see the ranking of the candidate study rooms and thus determine the required target study room.

[0050] In some embodiments, after the user enters the target study room, obtaining the user's learning status data through the millimeter wave radar sensor includes:

[0051] Acquire the user's heart rate data and breathing data through the millimeter wave radar sensor, and determine the user's concentration level and concentration value based on the heart rate data and breathing data;

[0052] The millimeter wave radar sensor is used to obtain the user's absence time and the study time at different concentration levels, and the user's learning efficiency value is calculated based on the user's concentration level and the study time at different concentration levels;

[0053] Acquiring posture data of a user in a learning state through the millimeter wave radar sensor, and determining a learning posture state of the user based on the posture data;

[0054] At least one of the following data is used as the learning status data, and the at least one data includes: concentration level, concentration value, learning efficiency value, and learning posture status.

[0055] In this embodiment, phase data is extracted based on the echo signal, and a phase unwrapping operation is performed on the extracted phase data. By subtracting continuous phase values ​​and performing a phase difference operation on the unwrapped phase, the heartbeat signal can be enhanced and any phase drift can be eliminated. According to the difference in heartbeat and breathing frequency, the phase value is filtered by a bandpass filter to distinguish, so that breathing data and heart rate data can be obtained. The user's sitting posture can be obtained through the distance information represented in the discrete echo signal, and it can be judged whether the learning posture state is correct. When the sitting posture is within the preset sitting posture range, it indicates that the learning posture state is correct. When the sitting posture is not within the preset sitting posture range, it indicates that the learning posture state is wrong.

[0056] In an optional embodiment, the calculating of the user's learning efficiency value based on the user's concentration level and the learning time at different concentration levels includes:

[0057] Obtain the time spent away from the computer when no heart rate data or breathing data is detected, and the time spent studying at different concentration levels;

[0058] Get the weights corresponding to the pre-configured time away from home and the weights corresponding to the study time at different concentration levels;

[0059] The learning efficiency value is calculated based on the weight corresponding to the absence time and the weight corresponding to the study time at different concentration levels, the absence time and the study time at different concentration levels.

[0060] Specifically, the heart rate data and breathing data obtained within a period of time can be input into a pre-trained concentration detection model, and the user's concentration and concentration value are output, wherein the concentration detection model is obtained based on the training data set training, and each sample in the training data set includes a heart rate sample data set breathing sample data, and a label corresponding to each sample, and the label includes a concentration label and a concentration value label. In an optional embodiment, when the heart rate data and breathing data are not detected within the preset time, it means that the user is in the absence time. Therefore, during the user's self-study process, the degree of concentration in each time period during the self-study process can be identified, and the weight corresponding to the departure time and the weight corresponding to the learning time under different concentration levels can be pre-configured, and then based on the weight corresponding to the departure time and the weight corresponding to the learning time under different concentration levels, the departure time and the learning time under different concentration levels, the learning efficiency value is calculated. Wherein the learning efficiency value = departure time × the weight corresponding to the departure time × the first preset coefficient + the weight corresponding to the learning time under different concentration levels × the learning time under different concentration levels × the second preset coefficient. The first preset coefficient represents the relationship between the absence time and the learning efficiency value, and the second preset coefficient represents the relationship between the learning time and the learning efficiency value under different concentration levels.

[0061] In the above embodiment, the millimeter wave radar sensor can be used to monitor the user's various learning status data, thereby monitoring the user's self-study status in many aspects, thereby facilitating improving the user's learning efficiency.

[0062] In some embodiments, displaying the self-study data in the target self-study room includes at least one of the following:

[0063] Displaying a virtual environment of the target study room, and displaying the positions of each current user in the target study room in the virtual environment, wherein the virtual environment is used to simulate the space of a learning environment;

[0064] Displaying the self-study data of each current user in the target study room, wherein the self-study data includes at least one of the following: concentration level, concentration value, learning efficiency value, learning posture state, learning time, learning efficiency value ranking, concentration ranking, concentration value ranking, and proportion of deep concentration;

[0065] Acquire a target current user selected by a user based on a user interface, and display the self-study data of the target current user;

[0066] The current number of users and the number of vacant seats in the target study room are displayed.

[0067] In the present embodiment, the degree of concentration includes deep concentration, medium concentration and low concentration. By displaying the self-study data of each current user in the target self-study room, it is convenient for each current user to understand his or her self-study status, and through various ranking data, it is convenient for the user to understand the comparison data with other self-study personnel to encourage the user to study actively. Through the user interface, the user can also click on the location area of ​​a certain target current user to understand the self-study data of the target current user.

[0068] In the above embodiment, the self-study data of the target study room is displayed in real time on the user interface, so that each user of the target study room can intuitively understand their own learning status and understand the comparative data with other self-study personnel, thereby motivating users to study actively and improve learning efficiency.

[0069] In some embodiments, the method further comprises:

[0070] Acquire a focus mode configured based on the user interface and focus configuration data corresponding to the focus mode, wherein the focus configuration data includes at least one of the following: a focus time period for entering focused learning, a focus duration for entering focused learning, a rest duration, and a target focus level, wherein the target focus level indicates a focus level that the user expects to achieve;

[0071] After obtaining the selected operation of the focus mode, detecting the heart rate data and breathing data of the user in the target time period corresponding to the focus configuration data through the millimeter wave radar sensor, and determining the current concentration level of the user based on the heart rate data and breathing data of the user in the target time period;

[0072] Based on the current concentration level and the concentration configuration data, outputting prompt information to prompt the user;

[0073] Wherein, based on the current concentration level and the concentration configuration data, outputting prompt information to prompt the user includes at least one of the following:

[0074] When the current concentration level is lower than the target concentration level, outputting a first prompt message to prompt the user to concentrate;

[0075] When the current concentration time reaches the concentration time, a second prompt message is output to prompt the user to pay attention to learning;

[0076] When the current rest time reaches the rest time, a third prompt message is output to prompt the user to enter the focus mode.

[0077] In this embodiment, the electronic device provides a configuration of a focus mode, which can help users improve their concentration and learning efficiency. After the user chooses to enter the focus mode, the user's concentration can be monitored in real time during the focus time period corresponding to the focus mode. For example, the user configures a focus time of 10 minutes and a rest time of 1 minute. After entering the focus mode, the timer automatically counts for 10 minutes. During these 10 minutes, the user needs to maintain a state of focused learning. After 10 minutes, the user is reminded to enter a rest state. After one minute of rest, the user is reminded to enter the focus mode again, and the cycle continues until the focus mode ends. During these 10 minutes, if it is detected that the current concentration level is lower than the target concentration level configured by the user, the user is reminded to concentrate. For example, the target concentration level configured by the user is a moderate concentration level. If a low concentration level is detected, the user is reminded. Therefore, after the user enters the focus mode, the electronic device can help the user improve his concentration, and when the concentration level is lower than the target concentration level, the user is promptly reminded to correct the current self-study state.

[0078] In the above embodiment, by configuring the focus mode, after the user enters the focus mode, the user's current concentration level is monitored in real time. When the current concentration level is lower than the target concentration level, the user is promptly reminded to correct the current self-study state. This cycle is repeated until the focus mode is ended and the user is reminded in time, which can help the user improve his learning efficiency.

[0079] In some embodiments, the method further comprises: acquiring a selected target current user based on each current user in the virtual environment, and displaying a dialogue control with the target current user on the virtual environment;

[0080] Acquire interaction information based on the dialog control, and send the interaction information to the server so that the server sends the interaction information to a terminal device of the target current user;

[0081] The reply content of the target current user sent by the server is received and displayed through the dialog control.

[0082] In this embodiment, the user can select the target current user to communicate with based on the user interface, for example Figure 5 As shown, by clicking on a user's avatar, a dialogue control with the target current user can be displayed on the interface of the virtual study room. The user can input interaction information through the dialogue control, and then the server sends the interaction information to the terminal device of the target current user, and sends the reply content of the target current user to the user's electronic device, and the electronic device displays the reply content in the dialogue control.

[0083] In the above embodiment, based on the virtual study room, real-time communication with the target current user can be achieved, thereby promoting exchanges and communication and improving learning efficiency.

[0084] In some embodiments, the user interface can also provide various types of learning resource identifiers. After the user clicks on the learning material identifier, the electronic device obtains the target learning material identifier and sends it to the server. The server obtains the corresponding resources based on the target learning material identifier and sends the corresponding resources to the user. In this way, the user can also obtain the required learning materials in real time through the electronic device, thereby improving learning efficiency.

[0085] In some embodiments, a millimeter wave radar-based online study room monitoring system includes the electronic device described in any embodiment of the present application, and a server communicating with the electronic device.

[0086] In some embodiments, the system further comprises:

[0087] The server obtains the study room keyword, determines a target study room tag matching the study room keyword, determines each study room to be selected corresponding to the target study room tag, and obtains tag data of each study room to be selected;

[0088] The study room keywords are converted into keyword vectors and the label data of each of the study rooms to be selected are converted into label vectors, and the similarities between the keyword vectors and the label vectors are calculated to obtain the similarity values ​​corresponding to the label vectors;

[0089] The similarity values ​​corresponding to the label vectors are sorted from high to low according to the similarity, and the study rooms to be selected corresponding to the similarity values ​​ranked in the front preset position are used as the study room matching data.

[0090] In this embodiment, a plurality of study room labels are stored in the database of the server, and the keywords are matched with the plurality of study room labels, and the study room labels that are the same or similar to the study room keywords are determined as the target study room labels. Since each study room label corresponds to a study room, and a study room can have multiple study room labels, the study room corresponding to the target study room label is determined as the study room to be selected, and then the similarity is calculated between all the label data of the study room to be selected and the study room keywords, and the study room to be selected with high similarity is determined as the candidate study room in the study room matching data.

[0091] In the above embodiment, the server first determines the target study room label that matches the study room keyword based on the study room keyword, then determines the study room to be selected based on the target study room label, and then obtains the candidate study room based on the similarity between the label data of the study room to be selected and the study room keyword, thereby improving the accuracy of the study room recommendation.

[0092] In some embodiments, the system further comprises at least one of the following:

[0093] For any self-study room, the server obtains the learning status data of each current user in the any self-study room, determines the self-study data of each current user and the overall self-study data of the any self-study room based on the learning status data of each current user, the overall self-study data indicates the self-study situation of the any self-study room, and the overall self-study data includes at least one of the following: average concentration level, average concentration value, average learning efficiency value, average learning time, ranking of learning efficiency values ​​of each current user, ranking of concentration of each current user, ranking of concentration value of each current user, proportion of overall deep concentration, and proportion of overall correct learning posture state;

[0094] For any user, the server obtains the historical self-study record of any user in the historical target time period, and outputs the learning analysis report of any user according to the historical self-study record, wherein the historical self-study record includes self-study data recorded multiple times, wherein the self-study data includes at least one of the following: concentration level, concentration value, learning efficiency value, learning posture state, learning time, learning efficiency value ranking, concentration ranking, concentration value ranking, and proportion of deep concentration, wherein the learning analysis report includes at least one of the following: the self-study data of any user in the historical target time period, the weaknesses that any user needs to strengthen, the strengths of any user, and optimization suggestions corresponding to the weaknesses;

[0095] Wherein, outputting the learning analysis report of any user according to the historical self-study record comprises at least one of the following:

[0096] Displaying the self-study data of any user in the historical target time period in a graphical manner;

[0097] Automatically identify weak points based on the historical self-study records, and automatically output optimization suggestions corresponding to the weak points, wherein when the weak points indicate concentration, a suggestion for using a concentration mode is output in the optimization suggestion; when the weak points indicate an incorrect learning posture, an illustration of a correct learning posture is output in the optimization suggestion.

[0098] In this embodiment, for any study room, the average concentration level is the average of the concentration levels of each current user, the average concentration value is calculated by averaging the concentration values ​​of each current user, the average learning efficiency value is the average of the learning efficiency values ​​of each current user, the average learning time is the average of the learning time of each current user, and the overall deep concentration ratio represents the ratio of deep concentration among all current users. For example, all current users are monitored 40 times in total, of which 10 times are deep concentration levels, then the deep concentration ratio is 10 / 40, and the overall correct learning posture status ratio represents the ratio of the number of times the correct learning posture status is monitored among all current users to the total number of monitoring times. Therefore, the overall self-study data can reflect the self-study situation of a study room. If a user wants to enter a high-quality study room, he can filter it through the overall self-study data.

[0099] For any user, the server can obtain the user's historical records, analyze the user's self-study situation by analyzing the historical records, and output a learning analysis report and suggestions. The user's average self-study data can be calculated through the self-study data recorded multiple times. The average self-study data includes but is not limited to: average concentration level, average concentration value, average learning efficiency value, correct learning posture state ratio, average learning time, and deep concentration ratio. The average self-study data represents the average self-study situation of the user during the historical target time period. In the average self-study data, items below the preset average limit are determined as weak items, and the optimization suggestions corresponding to the weak items are automatically matched from the database, so as to help users improve their learning efficiency in a targeted manner and understand their own self-study situation as a whole. In the average self-study data, items higher than or equal to the preset average maximum limit are determined as advantage items.

[0100] In the above embodiment, the server can automatically analyze the overall self-study situation of each study room, intuitively understand the status of each study room, and facilitate users to screen suitable study rooms; the server can also output a learning analysis report based on the self-study situation in each user's historical records, so that users can intuitively understand their own weaknesses and strengths, and can provide optimization suggestions through the learning analysis report, thereby helping users improve their learning efficiency.

[0101] On the other hand, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the online study room monitoring method based on millimeter wave radar described in any embodiment of the present application.

[0102] Among them, in the computer program product, an optional implementation form of the program module architecture of the computer program that implements each step of the target identification method can be an online study room monitoring device based on millimeter wave radar.

[0103] See also Figure 7 An embodiment of the present application provides an online study room monitoring device based on millimeter wave radar, including: an acquisition module 71, used to acquire a study room keyword acquired based on a user interface, and send the study room keyword to a server communicating with the electronic device, so that the server matches the study room keyword with an online virtual study room in a database to obtain study room matching data; a determination module 72, used to acquire the study room matching data, and based on the study room matching data, determine the target study room, and send it to the server; a monitoring module 73, used to acquire the user's learning status data through the millimeter wave radar sensor after the user enters the target study room, and send it to the server so that the server generates the study data in the target study room according to the learning status data of each user; a display module 74, used to acquire the study data in the target study room, and display the study data in the target study room.

[0104] Optionally, the acquisition module 71 is further used for:

[0105] Acquire a school selection operation and a class selection operation based on a user interface, and use a school identifier corresponding to the school selection operation and a class identifier corresponding to the class selection operation as the study room keyword;

[0106] Based on the user interface, multiple keywords are obtained. When the multiple keywords indicate that there is no corresponding study room identification, historical study room data used by the user is obtained, and label data corresponding to the historical study room data and the multiple keywords are used as the study room keywords.

[0107] Optionally, the determination module 72 is further configured to:

[0108] Displaying usage data of each candidate study room in the study room matching data on the user interface, and acquiring a target study room based on each candidate study room in the user interface, wherein the usage data includes at least one of the following: purpose of the study room, establishment time of the study room, overall learning efficiency data of historical users in the study room, and overall learning efficiency data of the current user in the study room, wherein the overall learning efficiency data of the historical users is determined based on the historical usage records of the study room, and the overall learning efficiency data of the current user is determined based on the usage records of the current user in the study room;

[0109] A sorting item for sorting the study rooms is obtained based on the user interface, each candidate study room in the study room matching data is sorted, and a target study room is obtained based on the sorted result.

[0110] Optionally, the monitoring module 73 is further used for:

[0111] Acquire the user's heart rate data and breathing data through the millimeter wave radar sensor, and determine the user's concentration level and concentration value based on the heart rate data and breathing data;

[0112] The millimeter wave radar sensor is used to obtain the user's absence time and the study time at different concentration levels, and the user's learning efficiency value is calculated based on the user's concentration level and the study time at different concentration levels;

[0113] Acquiring posture data of a user in a learning state through the millimeter wave radar sensor, and determining a learning posture state of the user based on the posture data;

[0114] At least one of the following data is used as the learning status data, and the at least one data includes: concentration level, concentration value, learning efficiency value, and learning posture status.

[0115] Optionally, the display module 74 is also used for:

[0116] Displaying a virtual environment of the target study room, and displaying the positions of each current user in the target study room in the virtual environment, wherein the virtual environment is used to simulate the space of a learning environment;

[0117] Displaying the self-study data of each current user in the target study room, wherein the self-study data includes at least one of the following: concentration level, concentration value, learning efficiency value, learning posture state, learning time, learning efficiency value ranking, concentration ranking, concentration value ranking, and proportion of deep concentration;

[0118] Acquire a target current user selected by a user based on a user interface, and display the self-study data of the target current user;

[0119] The current number of users and the number of vacant seats in the target study room are displayed.

[0120] Optionally, the monitoring module 73 is further used for:

[0121] Acquire a focus mode configured based on the user interface and focus configuration data corresponding to the focus mode, wherein the focus configuration data includes at least one of the following: a focus time period for entering focused learning, a focus duration for entering focused learning, a rest duration, and a target focus level, wherein the target focus level indicates a focus level that the user expects to achieve;

[0122] After obtaining the selected operation of the focus mode, detecting the heart rate data and breathing data of the user in the target time period corresponding to the focus configuration data through the millimeter wave radar sensor, and determining the current concentration level of the user based on the heart rate data and breathing data of the user in the target time period;

[0123] Based on the current concentration level and the concentration configuration data, outputting prompt information to prompt the user;

[0124] Wherein, based on the current concentration level and the concentration configuration data, outputting prompt information to prompt the user includes at least one of the following:

[0125] When the current concentration level is lower than the target concentration level, outputting a first prompt message to prompt the user to concentrate;

[0126] When the current concentration time reaches the concentration time, a second prompt message is output to prompt the user to pay attention to learning;

[0127] When the current rest time reaches the rest time, a third prompt message is output to prompt the user to enter the focus mode.

[0128] Optionally, the display module 74 is also used for:

[0129] Based on each current user in the virtual environment, obtaining a selected target current user, and displaying a dialogue control with the target current user on the virtual environment;

[0130] Acquire interaction information based on the dialog control, and send the interaction information to the server so that the server sends the interaction information to a terminal device of the target current user;

[0131] The reply content of the target current user sent by the server is received and displayed through the dialog control.

[0132] It can be understood by those skilled in the art that Figure 7The structure of the online study room monitoring device based on millimeter-wave radar does not constitute a limitation on the online study room monitoring device based on millimeter-wave radar, and each module can be implemented in whole or in part by software, hardware, and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules. In other embodiments, the online study room monitoring device based on millimeter-wave radar may include more or fewer modules than shown in the figure.

[0133] See also Figure 8 In another aspect of the embodiment of the present application, an electronic device 10 is provided, including a processor 13 and a memory 14, wherein the memory 14 stores a computer program, and when the computer program is executed by the processor, the processor 13 executes the steps of the online study room monitoring method based on millimeter wave radar provided in any of the above embodiments of the present application. The electronic device 10 is a device with infrared imaging function, for example, it may include a computing device (for example, a desktop computer, a laptop computer, a tablet computer, a handheld computer, a smart speaker, a server, etc.), a terminal device (for example, a mobile phone, etc.), a wearable device (for example, a pair of smart glasses or a smart watch), a smart desk lamp, or a similar device.

[0134] The processor 13 is the control center, which uses various interfaces and lines to connect various parts of the entire computer device, and executes various functions of the computer device and processes data by running or executing software programs and / or modules stored in the memory 14, and calling data stored in the memory 14. Optionally, the processor 13 may include one or more processing cores; preferably, the processor 13 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user pages and application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 13.

[0135] The memory 14 can be used to store software programs and modules. The processor 13 executes various functional applications and data processing by running the software programs and modules stored in the memory 14. The memory 14 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the computer device, etc. In addition, the memory 14 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 14 can also include a memory processor to provide the processor 13 with access to the memory 14.

[0136] On the other hand, an embodiment of the present application further provides a storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of the online study room monitoring method based on millimeter wave radar provided in any of the above embodiments of the present application.

[0137] Those skilled in the art can understand that all or part of the processes in the methods provided in the above embodiments can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0138] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. The protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for monitoring an online study room based on millimeter wave radar, characterized in that: Applied in an electronic device, wherein the electronic device is provided with a millimeter wave radar sensor, including: Acquire a study room keyword acquired based on the user interface, and send the study room keyword to a server in communication with the electronic device, so that the server matches the study room keyword with an online virtual study room in a database to obtain study room matching data; Acquire the study room matching data, and determine the target study room based on the study room matching data, and send it to the server; After the user enters the target study room, the user's learning status data is obtained through the millimeter-wave radar sensor, and sent to the server so that the server generates the self-study data of the target study room according to the learning status data of each user; wherein after the user enters the target study room, the user's learning status data is obtained through the millimeter-wave radar sensor, including: obtaining the user's heart rate data and breathing data through the millimeter-wave radar sensor, and determining the user's concentration level and concentration value based on the heart rate data and breathing data; obtaining the user's absence time and the study time at different concentration levels through the millimeter-wave radar sensor, and calculating the user's learning efficiency value based on the user's concentration level and the study time at different concentration levels; using the concentration level, concentration value, and learning efficiency value as the learning status data; The method of obtaining the user's heart rate data and breathing data through the millimeter wave radar sensor, and determining the user's concentration level and concentration value based on the heart rate data and breathing data includes: inputting the obtained heart rate data and breathing data within a period of time into a pre-trained concentration detection model, and outputting the user's concentration level and concentration value, wherein the concentration detection model is trained based on a training data set, each sample in the training data set includes heart rate sample data and breathing sample data, and a label corresponding to each sample, and the label includes a concentration level label and a concentration value label; The calculation of the learning efficiency value of the user based on the user's concentration level and the learning time at different concentration levels includes: obtaining the time when the heart rate data and breathing data are not detected and the learning time at different concentration levels; obtaining the weight corresponding to the pre-configured time when the learning time is away and the weight corresponding to the learning time at different concentration levels; calculating the learning efficiency value based on the weight corresponding to the time when the learning time is away and the weight corresponding to the learning time at different concentration levels, the time when the learning time is away and the learning time at different concentration levels; The self-study data in the target self-study room is acquired, and the self-study data in the target self-study room is displayed.

2. The online study room monitoring method based on millimeter wave radar as claimed in claim 1 is characterized in that: The acquiring of the study room keywords based on the user interface includes at least one of the following: Acquire a school selection operation and a class selection operation based on a user interface, and use a school identifier corresponding to the school selection operation and a class identifier corresponding to the class selection operation as the study room keyword; Based on the user interface, multiple keywords are obtained. When the multiple keywords indicate that there is no corresponding study room identification, historical study room data used by the user is obtained, and label data corresponding to the historical study room data and the multiple keywords are used as the study room keywords.

3. The online study room monitoring method based on millimeter wave radar as claimed in claim 1 is characterized in that: The determining of the target study room based on the study room matching data includes at least one of the following: Displaying usage data of each candidate study room in the study room matching data on the user interface, and acquiring a target study room based on each candidate study room in the user interface, wherein the usage data includes at least one of the following: purpose of the study room, establishment time of the study room, overall learning efficiency data of historical users in the study room, and overall learning efficiency data of the current user in the study room, wherein the overall learning efficiency data of the historical users is determined based on the historical usage records of the study room, and the overall learning efficiency data of the current user is determined based on the usage records of the current user in the study room; A sorting item for sorting the study rooms is obtained based on the user interface, each candidate study room in the study room matching data is sorted, and a target study room is obtained based on the sorted result.

4. The online study room monitoring method based on millimeter wave radar as claimed in claim 1 is characterized in that: After the user enters the target study room, obtaining the user's learning status data through the millimeter wave radar sensor includes: Acquiring posture data of a user in a learning state through the millimeter wave radar sensor, and determining a learning posture state of the user based on the posture data; The learning posture state is used as the learning state data.

5. The online study room monitoring method based on millimeter wave radar as claimed in claim 1 is characterized in that: The displaying of the self-study data in the target self-study room includes at least one of the following: Displaying a virtual environment of the target study room and displaying the positions of each current user in the target study room in the virtual environment, wherein the virtual environment is used to simulate the space of a learning environment; Displaying the self-study data of each current user in the target study room, wherein the self-study data includes at least one of the following: concentration level, concentration value, learning efficiency value, learning posture state, learning time, learning efficiency value ranking, concentration ranking, concentration value ranking, and proportion of deep concentration; Acquire a target current user selected by a user based on a user interface, and display the self-study data of the target current user; The current number of users and the number of vacant seats in the target study room are displayed.

6. The online study room monitoring method based on millimeter wave radar as claimed in claim 1, characterized in that: The method further comprises: Acquire a focus mode configured based on the user interface and focus configuration data corresponding to the focus mode, wherein the focus configuration data includes at least one of the following: a focus time period for entering focused learning, a focus duration for entering focused learning, a rest duration, and a target focus level, wherein the target focus level indicates a focus level that the user expects to achieve; After obtaining the selected operation of the focus mode, detecting the heart rate data and breathing data of the user in the target time period corresponding to the focus configuration data through the millimeter wave radar sensor, and determining the current concentration level of the user based on the heart rate data and breathing data of the user in the target time period; Based on the current concentration level and the concentration configuration data, outputting prompt information to prompt the user; Wherein, based on the current concentration level and the concentration configuration data, outputting prompt information to prompt the user includes at least one of the following: When the current concentration level is lower than the target concentration level, outputting a first prompt message to prompt the user to concentrate; When the current concentration time reaches the concentration time, a second prompt message is output to prompt the user to pay attention to study; When the current rest time reaches the rest time, a third prompt message is output to prompt the user to enter the focus mode.

7. The online study room monitoring method based on millimeter wave radar as claimed in claim 5 is characterized in that: The method further comprises: Based on each current user in the virtual environment, obtaining a selected target current user, and displaying a dialogue control with the target current user on the virtual environment; Acquire interaction information based on the dialog control, and send the interaction information to the server so that the server sends the interaction information to a terminal device of the target current user; The reply content of the target current user sent by the server is received and displayed through the dialog control.

8. An online study room monitoring system based on millimeter wave radar, characterized in that: The system comprises the electronic device according to any one of claims 1 to 7, and a server communicating with the electronic device.

9. The online study room monitoring system based on millimeter wave radar as claimed in claim 8, characterized in that: The system further comprises: The server obtains the study room keyword, determines a target study room tag matching the study room keyword, determines each study room to be selected corresponding to the target study room tag, and obtains tag data of each study room to be selected; The study room keywords are converted into keyword vectors and the label data of each of the study rooms to be selected are converted into label vectors, and the similarities between the keyword vectors and the label vectors are calculated to obtain the similarity values ​​corresponding to the label vectors; The similarity values ​​corresponding to the label vectors are sorted from high to low according to the similarity, and the study rooms to be selected corresponding to the similarity values ​​ranked in the front preset position are used as the study room matching data.

10. The online study room monitoring system based on millimeter wave radar as claimed in claim 8, characterized in that: The system further comprises at least one of the following: For any self-study room, the server obtains the learning status data of each current user in the any self-study room, determines the self-study data of each current user and the overall self-study data of the any self-study room based on the learning status data of each current user, the overall self-study data indicates the self-study situation of the any self-study room, and the overall self-study data includes at least one of the following: average concentration level, average concentration value, average learning efficiency value, average learning time, ranking of learning efficiency values ​​of each current user, ranking of concentration of each current user, ranking of concentration value of each current user, proportion of overall deep concentration, and proportion of overall correct learning posture state; For any user, the server obtains the historical self-study record of any user in the historical target time period, and outputs the learning analysis report of any user according to the historical self-study record, wherein the historical self-study record includes self-study data recorded multiple times, wherein the self-study data includes at least one of the following: concentration level, concentration value, learning efficiency value, learning posture state, learning time, learning efficiency value ranking, concentration ranking, concentration value ranking, and proportion of deep concentration, wherein the learning analysis report includes at least one of the following: the self-study data of any user in the historical target time period, the weaknesses that any user needs to strengthen, the strengths of any user, and optimization suggestions corresponding to the weaknesses; Wherein, outputting the learning analysis report of any user according to the historical self-study record comprises at least one of the following: Displaying the self-study data of any user in the historical target time period in a graphical manner; Automatically identify weak points based on the historical self-study records, and automatically output optimization suggestions corresponding to the weak points, wherein when the weak points indicate concentration, a suggestion for using a concentration mode is output in the optimization suggestion; when the weak points indicate an incorrect learning posture, an illustration of a correct learning posture is output in the optimization suggestion.

Images