Sitting posture prompting method, system and electronic equipment
Through bracelet recognition and binocular camera calculation, an identity binding table and sitting posture recognition system are built, which solves the accuracy and comprehensiveness of students' sitting posture correction in the classroom, and realizes comprehensive monitoring and early warning of students' sitting posture and vision health.
Patent Information
- Application Number
- CN202311171964.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-12
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2043-09-12
AI Technical Summary
The existing technology cannot fully monitor and correct students' sitting posture in the classroom, resulting in long-term incorrect sitting postures that can lead to vision loss and spinal development problems.
By constructing an identity binding table, using a bracelet to identify the student's sitting posture type and send prompt information, and combining a binocular camera to calculate the writing distance, it can accurately correct and monitor the student's sitting posture.
It improves the accuracy and comprehensiveness of students' sitting posture correction, reduces false triggers, provides comprehensive monitoring and early warning of students' sitting posture and vision health, forming a virtuous cycle.
Smart Images

Figure CN118230376B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of smart classrooms, and in particular to a sitting posture prompting method, system and electronic equipment. Background Art
[0002] At present, primary and secondary school students spend more and more time studying at desks, and many of them sit in incorrect postures. In the classroom, although the existing technology can mark the sitting posture of students, it is still unable to fully supervise and remind students in the classroom, and it is impossible to correct students' sitting posture in time. If the sitting posture is incorrect for a long time, it is easy to cause students' vision to decline or affect the spinal development of adolescents during their growth period. Summary of the invention
[0003] In order to overcome the defects and shortcomings of the prior art, the present application proposes a sitting posture prompting method, system and electronic device. The method constructs an identity binding table to form a one-to-one correspondence between bracelet users and bracelets, and each student becomes a bracelet user by wearing a bracelet. Then, by identifying the sitting posture type of the bracelet user, the specific bracelet is determined, and the prompts of the bracelet are used to improve the accuracy of correcting the student's sitting posture, and also assist teachers or parents to remind students to develop correct sitting posture habits in time.
[0004] The first purpose of the present application is to provide a sitting posture prompt method, the second purpose is to provide a sitting posture prompt system, and the third purpose is to provide an electronic device.
[0005] In order to achieve the above first purpose, this application adopts the following technical solutions:
[0006] A sitting posture prompting method, used in a smart classroom, comprises the following steps:
[0007] The first face detection step: identifying multiple target objects, identifying corresponding target IDs respectively to locate multiple face regions, each face region corresponding to a target ID;
[0008] User identity binding step: performing similarity comparison on the multiple face regions according to the wristband user face database to match the faces, adding the identity IDs of all successful face matches, the target ID and the wristband ID as the same row of information to the identity binding table, the identity ID is used to correspond to each student face image in the wristband user face database, and the identity ID also corresponds to the wristband ID to form a one-to-one correspondence between the wristband user and the wristband;
[0009] Sitting posture recognition step: Identify the sitting posture type of each bracelet user;
[0010] First prompt judgment step: perform the following processing on each target object, and when it is judged that the sitting posture type is not the correct sitting posture type, determine it as the first object to be prompted;
[0011] The first sitting posture prompting step: determining a plurality of first target wristbands according to the wristband IDs of all the first objects to be prompted, and sending the prompting information to the plurality of first target wristbands to implement the prompting.
[0012] Preferably, the method further comprises the following steps:
[0013] Second face detection step: when a wristband user is photographed entering or leaving the classroom, multiple target objects are identified, and corresponding target IDs are marked to locate multiple face areas, a queue to be updated is generated, and the target IDs identified at this time are added to it;
[0014] User identity update step: compare the similarity of the multiple face areas according to the bracelet user face database to match the face, add the identity ID and the corresponding target ID to the queue to be updated, when all faces are matched successfully, if there are repeated identity IDs in the identity binding table and the binding queue to be updated, then the target ID corresponding to the identity ID in the identity binding table shall be used as the basis for correction. In the case where the identity ID is the same but the corresponding target ID is different, the remaining non-duplicate identity ID, the corresponding target ID, and the corresponding bracelet ID are added as the same row of information to the identity binding table to complete the update process.
[0015] Preferably, the method further comprises the following steps:
[0016] Writing distance prediction step: Calculate the writing distance of each bracelet user;
[0017] The second prompt judgment step: determining the writing distance according to the target ID corresponding to each first object to be prompted, determining the distance threshold range based on the target ID, counting the number of offset values within a preset time, and determining the second object to be prompted when the number of offset values exceeds the preset number, wherein the offset value is obtained according to the difference between the writing distance and the distance threshold range;
[0018] The second sitting posture prompting step: determining a second target wristband according to the wristband ID of the second object to be prompted, and sending the prompting information to the second target wristband to implement the prompting.
[0019] Preferably, the first video data and the second video data are respectively left and right video data obtained by the corresponding binocular camera;
[0020] The calculation of the writing distance of each bracelet user includes:
[0021] Camera calibration step: photographing the calibration plate to determine calibration parameters, wherein the calibration parameters include internal parameters and external parameters;
[0022] Feature point matching step: extracting eye feature points and desktop feature points for each bracelet user respectively, and matching the eye feature points and desktop feature points extracted by each bracelet user to form a corresponding relationship between the feature points;
[0023] Parallax calculation step: obtaining the parallax of each bracelet user's eyes and desktop in the image according to the correspondence relationship of the feature points;
[0024] Depth information calculation step: calculating the depth information of the eyes and desktop of each bracelet user based on the parallax;
[0025] Coordinate conversion step: combining the depth information with the calibration parameters to convert the pixel coordinates of the binocular camera into physical space coordinates, converting the coordinate system so that different coordinate systems can be converted and compared with each other, and obtaining the three-dimensional coordinates of the eyes and desktop of each bracelet user according to the depth information and coordinate conversion;
[0026] Writing distance calculation step: determining the writing distance of each bracelet user according to the three-dimensional coordinates of the eyes and the desktop of each bracelet user.
[0027] Preferably, when executing the first prompt determination step, the step further includes:
[0028] The sub-step of handling misjudgment of speaking is to determine whether all the first objects to be prompted are in the state of speaking in class. If so, the first objects to be prompted are eliminated.
[0029] Preferably, the method further comprises:
[0030] The first statistical analysis step: collecting the sitting posture types and offset values of all target objects within a preset time interval, each moment value corresponds to a sitting posture type, a classroom speech state and an offset value, correcting the actual sitting posture type according to the classroom speech state, and then summarizing and analyzing the sitting posture and writing distance state to generate an early warning report for the target object, the early warning report includes the sitting posture change trend and the writing distance change trend, each identity ID corresponds to a student's early warning report, and the early warning report is used to provide the teacher with a display of the analysis results of the sitting posture changes and writing distance changes of the students in the class;
[0031] The second statistical analysis step: obtaining the vision values of all students, and matching the sitting posture warning level according to each student's vision value and the corresponding warning report analysis. The sitting posture warning level is used to identify the urgency of correcting the student's sitting posture. The vision values of all students are obtained by querying the vision health monitoring database based on the student's identity ID.
[0032] Preferably, the similarity comparison is specifically to compare the similarity of two face feature vectors by using a Euclidean distance calculation method or a cosine similarity calculation method, wherein the two face feature vectors are respectively a current face feature vector traversed in a face feature vector set and a current face feature vector traversed in a face feature vector standard set;
[0033] The facial feature vector set is formed by inputting each facial region into a feature extraction model to obtain a corresponding facial feature vector, wherein the feature extraction model is obtained by machine learning training using multiple sets of data, wherein each set of data in the multiple sets of data includes information of a facial feature image and a facial feature vector corresponding to the facial feature image;
[0034] The standard set of facial feature vectors is obtained by obtaining all student face images from the bracelet user face database, and inputting them into the feature extraction model to extract facial feature vectors.
[0035] In order to achieve the above second purpose, the present application adopts the following technical solutions:
[0036] A sitting posture prompting system for a smart classroom, the system comprising:
[0037] A first face detection module is used to identify multiple target objects, and identify corresponding target IDs to locate multiple face regions, each face region corresponding to a target ID;
[0038] A user identity binding module is used to compare the similarity of the multiple face areas according to the wristband user face database to match the face, and add the identity IDs of all successful face matches, the target ID and the wristband ID as the same row of information to the identity binding table, and the identity ID is used to correspond to each student face image in the wristband user face database, and the identity ID also corresponds to the wristband ID to form a one-to-one correspondence between the wristband user and the wristband;
[0039] Sitting posture recognition module, used to identify the sitting posture type of each bracelet user;
[0040] The first prompt judgment module is used to perform the following processing on each target object, and when it is judged that the sitting posture type is not the correct sitting posture type, it is determined as the first object to be prompted;
[0041] A first sitting posture prompting module, used for determining a plurality of first target wristbands according to the wristband IDs of all first objects to be prompted, and sending prompt information to the plurality of first target wristbands to implement the prompt;
[0042] The second face detection module is used to identify multiple target objects when a wristband user enters or leaves the classroom, and identify the corresponding target IDs to locate multiple face areas, generate a queue to be updated, and add the target IDs identified at this time into it;
[0043] The user identity update module is used to compare the similarity of the multiple face areas according to the bracelet user face database to match the faces, and add the identity ID and the corresponding target ID to the queue to be updated. When all faces are matched successfully, if there are repeated identity IDs in the identity binding table and the binding queue to be updated, the target ID corresponding to the identity ID in the identity binding table is used as the basis for correction. In the case where the identity IDs are the same but the corresponding target IDs are different, the remaining non-repeated identity IDs, the corresponding target IDs, and the corresponding bracelet IDs are added to the identity binding table as the same row of information to complete the update process.
[0044] In order to achieve the third objective above, this application adopts the following technical solutions:
[0045] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, any of the above-mentioned sitting posture prompting methods is implemented.
[0046] Compared with the prior art, this application has the following advantages and beneficial effects:
[0047] (1) Obtain left and right video data through binocular cameras and use them as first video data and second video data respectively. Take the first video data as the basis for image processing. Perform similarity comparison on multiple face regions according to the wristband user face database to match the face, and then construct an identity binding table so that the identity ID, target ID and wristband ID correspond to each other, thereby forming a one-to-one correspondence between the wristband user and the wristband. When it is determined that the sitting posture type is not the correct sitting posture type, determine the target wristband that needs to vibrate to prompt according to the wristband ID, thereby achieving accurate sitting posture correction effect in the smart classroom scenario. By judging the sitting posture of students in the classroom and prompting students with wrong sitting postures through designated wristbands, the sitting posture correction of classroom students is more comprehensive and accurate, so as to assist in strengthening the prevention and control of myopia among children and adolescents.
[0048] (2) When a wristband user is photographed entering or leaving the classroom, a new target ID is identified, causing the target ID to change. By removing duplicate identity IDs from the identity binding table and the binding queue to be updated, and using the target ID corresponding to the identity ID in the identity binding table as the basis for correction, the identity binding table is updated with the remaining non-duplicate identity IDs, the corresponding target IDs, and the corresponding wristband IDs, thereby avoiding confusion between identity IDs and target IDs caused by students entering or leaving the classroom during image recognition processing and improving the stability of the identity binding table.
[0049] (3) On the basis of sitting posture recognition, the number of offset values within the preset time is counted according to the writing distance, and then the sitting posture of students is corrected in a timely and accurate manner based on the sitting posture type and offset value, which improves the accuracy of the sitting posture recognition processing of the whole students in the smart classroom and effectively improves the accuracy of correcting the sitting posture of students.
[0050] (4) By judging the classroom speaking status before the bracelet prompts, the corresponding first-to-be-prompted object is eliminated, thereby avoiding the situation where students accidentally touch the bracelet prompt due to classroom speaking, and improving the accuracy of smart classrooms in recognizing students' sitting posture.
[0051] (5) Through the linkage with the vision health monitoring database, a comprehensive and accurate analysis is conducted to analyze the changing trends of the bracelet user's sitting posture and writing distance when writing at a desk, and form corresponding posture and writing distance warning reports. In this way, teachers or parents can remind and correct them based on the analysis results, thus forming a virtuous cycle of comprehensive intelligent monitoring, early warning, and timely correction. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] Figure 1 A schematic diagram of the steps of a sitting posture prompting method in an embodiment of the present application;
[0053] Figure 2 This is a schematic diagram of the specific steps of extracting facial feature vectors in the user identity binding step in the embodiment of the present application;
[0054] Figure 3 This is a schematic diagram of the specific steps of the writing distance prediction step in Example 3 of the present application;
[0055] Figure 4 This is a schematic diagram of the specific steps of the sub-step of handling misjudgment of speech during classroom speech in Example 4 of the present application;
[0056] Figure 5 This is a schematic diagram of the framework of a sitting posture prompting system in Example 5 of the present application;
[0057] Figure 6 This is a schematic diagram of the framework of the electronic device in Example 6 of the present application.
[0058] Figure numbers: 10, sitting posture prompt system; 100, first face detection module; 200, user identity binding module; 300, sitting posture recognition module; 400, first prompt judgment module; 500, first sitting posture prompt module; 600, second face detection module; 700, user identity update module; 800, writing distance prediction module; 900, second prompt judgment module; 1000, second sitting posture prompt module; 1100, first statistical analysis module; 1200, second statistical analysis module; 1300, an electronic device; 1301, memory; 1302, processor. DETAILED DESCRIPTION
[0059] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0060] Example 1
[0061] like Figure 1 As shown, this embodiment proposes a sitting posture prompting method for a smart classroom, which includes the following steps:
[0062] S100, first face detection step: using a face detection model to perform face detection processing on the first video data to identify multiple target objects, and respectively identify corresponding target IDs to locate multiple face areas, the face detection model is obtained by machine learning training using multiple sets of data, wherein each set of data in the multiple sets of data includes a face feature image and face identification information corresponding to the face feature image;
[0063] In this embodiment, the execution subject is a terminal or a server, and the machine learning model used by the face detection model specifically adopts models such as YOLO, RetinaFace, etc., which are not limited here, and technicians in this field can choose according to actual conditions.
[0064] In actual application, during execution, image data of a specified number of frames are extracted from the first video data for processing, without the need to process all image data in the first video data, thereby achieving a better detection effect while reducing a large amount of calculation.
[0065] In this embodiment, when locating multiple face regions, the face regions are also subjected to region cropping processing. By performing the cropping processing on multiple faces, each face region is the target region during the face detection processing, and the target ID, position and region size of the target region are marked. Specifically, the region cropping processing adopts any one of the edge detection method and the threshold segmentation method. The edge detection method is to perform edge detection on the image based on the Canny algorithm to obtain the connected region, and then implement the image segmentation and region division according to the connected region. The threshold segmentation method is to divide the pixels in the image into two parts according to the gray value size according to the preset segmentation threshold, thereby implementing the image segmentation and region extraction. In practical application, the cropping processing is performed to accurately locate the processing task of the face region. In this way, in the subsequent similarity comparison, only the specific region of the face region needs to be paid attention to, and the information of other regions is not analyzed. At this time, the region cropping can accurately locate the location of the target, avoid the interference of irrelevant information, and also improve the operation speed of the overall task, avoiding the situation that when the resolution of the image or video is relatively large, directly processing the entire image will occupy a lot of computing resources and time.
[0066] S200, user identity binding step: setting an identity binding table, extracting facial feature vectors from the facial area corresponding to each target object based on the wristband user face database, performing similarity calculation and comparison based on the facial feature vectors, and when the similarity exceeds the similarity threshold, it is regarded as a successful face match, and the identity ID is bound to the corresponding target ID, and the identity ID, target ID and wristband ID of all successful face matches are added to the identity binding table as the same row of information, wherein the wristband user face database contains all student face images, and each student face image corresponds to a unique identity ID, and the identity ID also corresponds to the wristband ID, so as to form a one-to-one correspondence between the wristband user and the wristband;
[0067] In this embodiment, the identity ID is used to identify different students. Specifically, the identity ID may be a unique identification number such as an identity card number or a student ID number.
[0068] Combination Figure 2 As shown, in S200, in the user identity binding step, facial feature vectors are extracted from the facial regions corresponding to each target object based on the bracelet user face database, specifically including:
[0069] S201, a first feature extraction step: inputting each face region into a feature extraction model to obtain a corresponding face feature vector, thereby forming a face feature vector set, wherein the feature extraction model is obtained by machine learning training using multiple sets of data, wherein each set of data in the multiple sets of data includes a face feature image and information of a face feature vector corresponding to the face feature image;
[0070] S202, second feature extraction step: obtain all student face images from the bracelet user face database, and input them into the feature extraction model to extract face feature vectors to obtain a standard set of face feature vectors, where each student face image corresponds to a unique identity ID.
[0071] In this embodiment, the similarity calculation specifically adopts the Euclidean distance calculation method or the cosine similarity calculation method to calculate the similarity of two face feature vectors, wherein the two face feature vectors are respectively the current face feature vector traversed in the face feature vector set and the current face feature vector traversed in the face feature vector standard set;
[0072] S300, sitting posture recognition step: inputting the first video data into a sitting posture recognition model to recognize the sitting posture type of each bracelet user;
[0073] In this embodiment, the sitting posture recognition model is obtained by machine learning training using multiple sets of data, wherein each set of the multiple sets of data includes a sitting posture image and sitting posture type identification information corresponding to the sitting posture image.
[0074] S400, first prompt determination step: performing the following processing on each target object, and when it is determined that the sitting posture type is not the correct sitting posture type, determining it as the first object to be prompted;
[0075] In this embodiment, the sitting posture type includes a correct sitting posture type and an incorrect sitting posture type. In the incorrect sitting posture type, there are multiple sitting posture subtypes such as bending the waist, supporting the chin with the left hand, leaning the arm on the table, lying on the table, etc. In actual application, different sitting posture types or sitting posture subtypes can be distinguished according to different values, which is not limited in this embodiment.
[0076] S500, first sitting posture prompting step: determining multiple first target wristbands according to the wristband IDs of all first objects to be prompted, and sending prompt information to the multiple first target wristbands to implement the prompt.
[0077] In this embodiment, different bracelets are distinguished by the identification of the bracelet ID to determine the bracelet user who needs to correct the sitting posture, so that the sitting posture correction method in the smart classroom scenario can more accurately prompt the target students.
[0078] Example 2
[0079] This embodiment 2 is an improvement on the basis of embodiment 1 to solve the problem that when a bracelet user enters and exits the classroom during target tracking, a new target ID will be regenerated each time the bracelet user re-enters the screen due to the movement of the bracelet user, resulting in inconsistent target IDs corresponding to students with the same identity ID, which in turn causes disorder in the identity binding table during maintenance and confusion in the binding relationship.
[0080] Combination Figure 1 As shown, a sitting posture prompting method proposed in this embodiment 2 also includes the following steps:
[0081] S600, second face detection step: when the first video data captures a bracelet user entering or leaving the classroom, a face detection model is used to perform face detection processing on the first video data to identify multiple target objects, and corresponding target IDs are identified to locate multiple face areas, a queue to be updated is generated, and the target IDs identified at this time are added to the queue;
[0082] S700, user identity update step: based on the bracelet user face database, the face area corresponding to each target object is compared for similarity. When the similarity reaches the similarity threshold, it is regarded as a successful face match, and the identity ID and the corresponding target ID are added to the queue to be updated. When all faces are matched successfully, if there are duplicate identity IDs in the identity binding table and the binding queue to be updated, the target ID corresponding to the identity ID in the identity binding table is used as the basis for correction. In the case where the identity IDs are the same but the corresponding target IDs are different, the remaining non-duplicate identity IDs, the corresponding target IDs, and the corresponding bracelet IDs are added to the identity binding table as the same row of information to complete the update process.
[0083] Example 3
[0084] Based on the above embodiment, this embodiment 3 adds a prediction process of the writing distance to increase the accuracy of the recognition of the sitting posture of students writing in class. Figure 1 As shown, the sitting posture prompting method also includes the following steps:
[0085] S800, writing distance prediction step: calculating the writing distance of each bracelet user according to the first video data and the second video data;
[0086] In this embodiment, the first video data and the second video data are respectively left and right video data obtained by the corresponding binocular camera.
[0087] See also Figure 3 As shown, in S800, the writing distance prediction step specifically includes:
[0088] S801, camera calibration step: photograph the calibration plate to determine the calibration parameters, which include internal parameters and external parameters. The internal parameters include the camera focal length and the principal point position, and the external parameters include the baseline distance and relative attitude angle between the binocular cameras. In actual application, since the two cameras in the binocular camera have different viewing angles, calibration is required to determine the geometric relationship between the two cameras.
[0089] S802, feature point matching step: extract eye feature points and desktop feature points for each bracelet user respectively, match the eye feature points and desktop feature points of the first video data and the second video data to form a corresponding relationship between the feature points; specifically, by extracting features such as SIFT and SURF to correspond the objects in the two images, and then complete the matching of feature points.
[0090] S803, disparity calculation step: obtaining the disparity of each bracelet user's eye and desktop in the image according to the corresponding relationship of the feature points;
[0091] In this embodiment, parallax refers to the difference in the positions of objects under the viewing angles of the left and right cameras, which results in a difference in the positions of the same object in the left and right images when being imaged.
[0092] S804, depth information calculation step: extracting the depth information of the eyes and the desktop of each bracelet user based on parallax calculation;
[0093] In this embodiment, the depth information is the distance of the object from the camera. The depth information of the object is calculated based on parallax by triangulation. The triangulation method is to construct a spatial triangle of two objects using the principle of similar triangles, thereby solving the actual distance of the objects.
[0094] S805, coordinate conversion step: combining the depth information and the calibration parameters to convert the pixel coordinates of the binocular camera into physical space coordinates, converting the coordinate system so that different coordinate systems can be converted and compared with each other, and obtaining the three-dimensional coordinates of the eyes and desktop of each bracelet user according to the depth information and coordinate conversion;
[0095] In this embodiment, the calibration parameters are used to provide the spatial relationship of the binocular cameras.
[0096] S806, writing distance calculation step: determining the writing distance of each bracelet user according to the three-dimensional coordinates of the eyes and the desktop of each bracelet user.
[0097] S900, second prompt determination step: determining the writing distance according to the target ID corresponding to each first object to be prompted, determining the distance threshold range based on the target ID, counting the number of offset values within a preset time, and determining the second object to be prompted when the number of offset values exceeds the preset number, wherein the offset value is obtained according to the difference between the writing distance and the distance threshold range;
[0098] In this embodiment, the distance threshold range is determined based on the target ID, specifically: the identity binding table is queried according to the target ID to obtain the identity ID of the bracelet user, and then the height of the bracelet user is obtained through the identity ID, and the corresponding upper and lower distance thresholds are matched according to the height, and a numerical range is formed by the upper and lower distance thresholds to obtain the distance threshold range.
[0099] In this embodiment, the upper distance threshold and the lower distance threshold are preset values corresponding to the height matching, and the height is data pre-stored in the bracelet user face database, which can be obtained by querying the bracelet user face database based on the identity ID.
[0100] In this embodiment, by counting the number of offset values within a preset time period, combining the preset time period, the offset value, and the number of offset values, a writing distance evaluation report for each first object to be prompted is formed. The writing distance evaluation report can be sent to a designated teacher's terminal device according to the class ID of the current classroom to assist the teacher in analyzing the sitting posture of the students in the class. In actual application, the preset time period can be set to 30s, 60s, 5min, etc., which is not limited here.
[0101] S1000, second sitting posture prompting step: determining a second target bracelet according to the bracelet ID of the second object to be prompted, and sending prompt information to the second target bracelet to implement the prompt.
[0102] In this embodiment, when the bracelet receives a prompt message, the bracelet vibrates to prompt through the vibration plate, thereby achieving accurate sitting posture prompts in the classroom scene while avoiding affecting surrounding students.
[0103] Example 4
[0104] In this embodiment 4, a sitting posture prompt method in the above embodiment is improved. In class, there is a scenario where students stand up to answer the teacher's questions. At this time, if the incorrect sitting posture type is judged, a falsely triggered sitting posture prompt will be generated. In this regard, it is necessary to increase the judgment of the classroom speech status to avoid the falsely triggered sitting posture prompt at this time.
[0105] See also Figure 4 As shown, in this embodiment, when executing S400, the first prompt determination step, the step further includes:
[0106] S401, speech misjudgment processing sub-step: determine whether all first objects to be prompted are in the classroom speaking state, and if so, remove the first objects to be prompted.
[0107] In this embodiment, it is determined whether all the first objects to be prompted are in the classroom speaking state, specifically, whether they are in the classroom speaking state is determined by the first classroom speaking recognition model, the first classroom speaking recognition model is obtained by machine learning training using multiple sets of data, wherein each set of data in the multiple sets of data includes a face image and identification information of the classroom speaking state corresponding to the face image. In actual application, by intercepting a specified number of frames of images from the first video data, and cropping out the face area of the first object to be prompted as the input of the first classroom speaking recognition model, the classroom speaking state is obtained, and the first object to be prompted is eliminated according to the classroom speaking state, thereby avoiding the false triggering of the sitting posture prompt during classroom speaking.
[0108] In this embodiment, it is determined whether all the first objects to be prompted are in the classroom speech state, and it is also possible to determine whether they are in the classroom speech state through the second classroom speech recognition model, which is obtained by machine learning training using multiple sets of data, wherein each set of data in the multiple sets of data includes student speech audio information and identification information of the classroom speech state corresponding to the student speech audio information, wherein the identification information of the classroom speech state corresponding to the student speech audio information is used to determine whether they are in the classroom speech state. In actual application, the speech audio information of the current bracelet user is obtained by acquiring the speech audio information, which is collected by the bracelet worn by the student through the audio collection unit and transmitted to the execution subject.
[0109] Combination Figure 1 As shown, the sitting posture prompting method in this embodiment also includes:
[0110] S1100, first statistical analysis step: collect the sitting posture types and offset values of all target objects within a preset time interval, each moment value corresponds to a sitting posture type, a classroom speech status and an offset value, correct the actual sitting posture type according to the classroom speech status, and then summarize and analyze the sitting posture and writing distance status to generate an early warning report for the target object. The early warning report includes the sitting posture change trend and the writing distance change trend. Each identity ID corresponds to a student's early warning report. The early warning report is used to provide teachers with analysis results of the sitting posture changes and writing distance changes of the students in the class, thereby improving the intelligence of the smart classroom's supervision of the sitting posture and writing distance of the students in the class.
[0111] S1200, second statistical analysis step: obtain the vision values of all students, traverse the vision values of all students and match the corresponding sitting posture warning threshold and offset value warning threshold according to the vision values, count multiple groups of incorrect sitting posture types, classroom speech status and offset values from the warning report, correct the sitting posture type corresponding to the corresponding time value according to the classroom speech status, count the number of all incorrect sitting posture types and the number of non-zero offset values in the preset time period, and then judge whether it is necessary to warn the students based on the sitting posture warning threshold and the offset value warning threshold. When the number of incorrect sitting posture types exceeds the sitting posture warning threshold, set the first value, and when the number of non-zero offset values exceeds the offset value warning threshold, set the second value, and finally assign weighted proportions to sum them respectively, match the sitting posture warning degree to the summation result, and improve the personalized effect of the smart classroom on the sitting posture warning of the class students, so as to facilitate the teacher to clearly understand the sitting posture warning degree of the class students, and correct the sitting posture in time to prevent the students' myopia from further deteriorating. Among them, the vision values of all students are obtained by the execution subject from the vision health monitoring database according to the identity ID query.
[0112] Example 5
[0113] like Figure 5 As shown, this embodiment 5 provides a sitting posture prompting system 10, which corresponds to the method steps of the above embodiment. The system is used in a smart classroom and includes:
[0114] A first face detection module 100 is used to perform face detection processing on the first video data using a face detection model to identify multiple target objects, and to identify corresponding target IDs to locate multiple faces, wherein the face detection model is obtained by machine learning training using multiple sets of data, wherein each set of data in the multiple sets of data includes a face feature image and face identification information corresponding to the face feature image;
[0115] The user identity binding module 200 is used to set an identity binding table, extract facial feature vectors from the facial area corresponding to each target object based on the wristband user face database, calculate and compare similarity based on the facial feature vectors, and when the similarity exceeds the similarity threshold, it is regarded as a successful face match, and the identity ID is bound to the corresponding target ID, and the identity ID, target ID and wristband ID of all successful face matches are added to the identity binding table as the same row of information, and the identity ID also corresponds to the wristband ID to form a one-to-one correspondence between the wristband user and the wristband;
[0116] A sitting posture recognition module 300, used for inputting the first video data into a sitting posture recognition model to recognize the sitting posture type of each bracelet user;
[0117] The first prompt judging module 400 is used to perform the following processing on each target object, and when it is judged that the sitting posture type is not the first sitting posture type, it is determined as the first object to be prompted;
[0118] The first sitting posture prompting module 500 is used to determine multiple first target wristbands according to the wristband IDs of all first objects to be prompted, and send prompt information to the multiple first target wristbands to implement the prompt;
[0119] The second face detection module 600 is used to perform face detection processing on the first video data using a face detection model to identify multiple target objects when the first video data captures a bracelet user entering or leaving the classroom, and to identify corresponding target IDs to locate multiple face areas, generate a queue to be updated, and add the target IDs identified at this time into the queue;
[0120] The user identity update module 700 is used to compare the similarity of the face area corresponding to each target object based on the wristband user face database. When the similarity reaches the similarity threshold, it is regarded as a successful face match, and the identity ID and the corresponding target ID are added to the queue to be updated. When all faces are successfully matched, if there are repeated identity IDs in the identity binding table and the queue to be updated, the target ID corresponding to the identity ID in the identity binding table is used as the basis for correction. If the identity ID is the same but the corresponding target ID is different, the remaining non-duplicate identity ID, the corresponding target ID, and the corresponding wristband ID are added to the identity binding table as the same row of information to complete the update process;
[0121] A writing distance prediction module 800 is used to calculate the writing distance of each bracelet user according to the first video data and the second video data;
[0122] The second prompt judgment module 900 is used to determine the writing distance according to the target ID corresponding to each first object to be prompted, determine the distance threshold range based on the target ID, count the number of offset values within a preset time, and when the number of offset values exceeds the preset number, determine it as the second object to be prompted, wherein the offset value is obtained according to the difference between the writing distance and the distance threshold range;
[0123] The second sitting posture prompting module 1000 is used to determine the second target bracelet according to the bracelet ID of the second object to be prompted, and send the prompt information to the second target bracelet to realize the prompt;
[0124] The first statistical analysis module 1100 is used to collect the sitting posture types and offset values of all target objects within a preset time interval, each moment value corresponds to a sitting posture type, a classroom speech state and an offset value, and the actual sitting posture type is corrected according to the classroom speech state, and then the sitting posture and writing distance state are summarized and analyzed to generate an early warning report of the target object, and the early warning report is used to provide the teacher with the analysis results of the sitting posture changes and writing distance changes of the students in the class;
[0125] The second statistical analysis module 1200 is used to obtain the vision values of all students, and to match the sitting posture warning level according to the vision value of each student and the corresponding warning report analysis. The sitting posture warning level is used to identify the urgency of correcting the student's sitting posture. The vision values of all students are obtained by querying the vision health monitoring database based on the student's identity ID.
[0126] Example 6
[0127] like Figure 6 As shown, this embodiment provides an electronic device 1300, including a memory 1301, a processor 1302, and a computer program stored in the memory 1301 and executable on the processor, and when the processor executes the computer program, any one of the sitting posture prompt methods in embodiments 1 to 4 of the present application is implemented.
[0128] Example 7
[0129] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, any one of the sitting posture prompt methods in Embodiments 1 to 4 of the present application is implemented.
[0130] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
[0131] In addition, the terms "first", "second", and "third" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance. Similarly, words such as "one", "an", or "the" do not indicate a quantitative limitation, but rather indicate the presence of at least one. Words such as "include" or "comprise" and the like mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Words such as "connected" or "connected" and the like are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.
[0132] The above embodiments are preferred implementation modes of the present application, but the implementation modes of the present application are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present application should be equivalent replacement methods and are included in the protection scope of the present application.
Claims
1. A sitting posture prompting method, characterized in that: For smart classrooms, the following steps are included: The first face detection step: identifying multiple target objects, identifying corresponding target IDs respectively to locate multiple face regions, each face region corresponding to a target ID; User identity binding step: performing similarity comparison on the multiple face regions according to the wristband user face database to match the faces, adding the identity IDs of all successful face matches, the target ID and the wristband ID as the same row of information to the identity binding table, the identity ID is used to correspond to each student face image in the wristband user face database, and the identity ID also corresponds to the wristband ID to form a one-to-one correspondence between the wristband user and the wristband; Sitting posture recognition step: Identify the sitting posture type of each bracelet user; First prompt judgment step: perform the following processing on each target object, and when it is judged that the sitting posture type is not the correct sitting posture type, determine it as the first object to be prompted; Determine whether all the first objects to be prompted are in a classroom speaking state, and if so, remove the first objects to be prompted; The first sitting posture prompting step: determining a plurality of first target wristbands according to the wristband IDs of all first objects to be prompted, and sending prompt information to the plurality of first target wristbands to implement the prompting; The first statistical analysis step: collecting the sitting posture types and offset values of all target objects within a preset time interval, each moment value corresponds to a sitting posture type, a classroom speech state and an offset value, correcting the actual sitting posture type according to the classroom speech state, and then summarizing and analyzing the sitting posture and writing distance state to generate an early warning report for the target object, and the early warning report is used to provide the teacher with the analysis results of the sitting posture changes and writing distance changes of the students in the class; The second statistical analysis step: obtaining the vision values of all students, and matching the sitting posture warning level according to each student's vision value and the corresponding warning report analysis. The sitting posture warning level is used to identify the urgency of correcting the student's sitting posture. The vision values of all students are obtained by querying the vision health monitoring database based on the student's identity ID.
2. A sitting posture prompting method according to claim 1, characterized in that: The following steps are also included: Second face detection step: when a wristband user is photographed entering or leaving the classroom, multiple target objects are identified, and corresponding target IDs are marked to locate multiple face areas, a queue to be updated is generated, and the target IDs identified at this time are added to it; User identity update step: perform similarity comparison on the multiple face areas according to the bracelet user face database to match the faces, add the identity ID and the corresponding target ID to the queue to be updated, and when all faces are matched successfully, if there are duplicate identity IDs in the identity binding table and the queue to be updated, the target ID corresponding to the identity ID in the identity binding table shall be used as the basis for correction. In the case where the identity IDs are the same but the corresponding target IDs are different, the remaining non-duplicate identity IDs, the corresponding target IDs, and the corresponding bracelet IDs are added to the identity binding table as the same row of information to complete the update process.
3. A sitting posture prompting method according to claim 1, characterized in that: The following steps are also included: Writing distance prediction step: Calculate the writing distance of each bracelet user; The second prompt judgment step: determining the writing distance according to the target ID corresponding to each first object to be prompted, determining the distance threshold range based on the target ID, counting the number of offset values within a preset time, and determining the second object to be prompted when the number of offset values exceeds the preset number, wherein the offset value is obtained according to the difference between the writing distance and the distance threshold range; The second sitting posture prompting step: determining a second target wristband according to the wristband ID of the second object to be prompted, and sending the prompting information to the second target wristband to implement the prompting.
4. A sitting posture prompting method according to claim 3, characterized in that: The calculation of the writing distance of each bracelet user includes: Camera calibration step: photographing the calibration plate to determine calibration parameters, wherein the calibration parameters include internal parameters and external parameters; Feature point matching step: extracting eye feature points and desktop feature points for each bracelet user respectively, and matching the eye feature points and desktop feature points extracted by each bracelet user to form a corresponding relationship between the feature points; Parallax calculation step: obtaining the parallax of each bracelet user's eyes and desktop in the image according to the correspondence relationship of the feature points; Depth information calculation step: calculating the depth information of the eyes and desktop of each bracelet user based on the parallax; Coordinate conversion step: combining the depth information with the calibration parameters to convert the pixel coordinates of the binocular camera into physical space coordinates, converting the coordinate system so that different coordinate systems can be converted and compared with each other, and obtaining the three-dimensional coordinates of the eyes and desktop of each bracelet user according to the depth information and coordinate conversion; Writing distance calculation step: determining the writing distance of each bracelet user according to the three-dimensional coordinates of the eyes and the desktop of each bracelet user.
5. A sitting posture prompting method according to claim 1, characterized in that: The similarity comparison is specifically to compare the similarity of two face feature vectors by using a Euclidean distance calculation method or a cosine similarity calculation method, wherein the two face feature vectors are respectively a current face feature vector traversed in a face feature vector set and a current face feature vector traversed in a face feature vector standard set; The facial feature vector set is formed by inputting each facial region into a feature extraction model to obtain a corresponding facial feature vector, wherein the feature extraction model is obtained by machine learning training using multiple sets of data, wherein each set of data in the multiple sets of data includes information of a facial feature image and a facial feature vector corresponding to the facial feature image; The standard set of facial feature vectors is obtained by obtaining all student face images from the bracelet user face database, and inputting them into the feature extraction model to extract facial feature vectors.
6. A sitting posture prompting system, used in the steps of a sitting posture prompting method as claimed in any one of claims 1 to 5, characterized in that: For use in a smart classroom, the system includes: A first face detection module is used to identify multiple target objects, and identify corresponding target IDs to locate multiple face regions, each face region corresponding to a target ID; A user identity binding module is used to compare the similarity of the multiple face areas according to the wristband user face database to match the face, and add the identity IDs of all successful face matches, the target ID and the wristband ID as the same row of information to the identity binding table, and the identity ID is used to correspond to each student face image in the wristband user face database, and the identity ID also corresponds to the wristband ID to form a one-to-one correspondence between the wristband user and the wristband; Sitting posture recognition module, used to identify the sitting posture type of each bracelet user; A first prompt judgment module is used to perform the following processing on each target object, and when it is judged that the sitting posture type is not the correct sitting posture type, it is determined as the first object to be prompted; A first sitting posture prompting module, used for determining a plurality of first target wristbands according to the wristband IDs of all first objects to be prompted, and sending prompt information to the plurality of first target wristbands to implement the prompt; The second face detection module is used to identify multiple target objects when a wristband user enters or leaves the classroom, and identify the corresponding target IDs to locate multiple face areas, generate a queue to be updated, and add the target IDs identified at this time into it; The user identity update module is used to compare the similarity of the multiple face areas according to the bracelet user face database to match the faces, and add the identity ID and the corresponding target ID to the queue to be updated. When all faces are matched successfully, if there are repeated identity IDs in the identity binding table and the queue to be updated, the target ID corresponding to the identity ID in the identity binding table is used as the basis for correction. In the case where the identity IDs are the same but the corresponding target IDs are different, the remaining non-duplicate identity IDs, the corresponding target IDs, and the corresponding bracelet IDs are added to the identity binding table as the same row of information to complete the update process.
7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 5 is implemented.
Citation Information
Patent Citations
Method and system for detecting classroom attention of student
CN104517102A
Binocular camera-based real time human sitting posture condition detection method
CN106022304A
Student work intelligent monitoring method and system
CN108460707A
Sitting posture monitoring method and device
CN108665687A
Sitting posture detection method and wearable device
CN108921125A
