Sitting posture scoring method and device and computer readable storage medium
By adding a small object detection module to the key point detection model, the problem of low detection accuracy of small object in the prior art is solved, and higher key point detection accuracy and sitting position scoring accuracy are achieved.
Patent Information
- Application Number
- CN202411933143.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-26
- Publication Date
- 2025-04-25
AI Technical Summary
The existing deep learning-based sitting position scoring method has low detection accuracy on targets far away from the camera, resulting in inaccurate target miss detection and key point identification.
A small object detection module is added to the key point detection model, and the detection accuracy of small objects can be improved through multiple upsampling layers and depth separation convolution layers.
The detection accuracy of small targets in the picture is improved, the missed detection of targets far away from the camera in the target scene is reduced, and the accuracy of key point detection and the accuracy of sitting position scores are improved.
Smart Images
Figure CN119888813A_ABST
Abstract
Description
Technical Field
[0001] The present application generally relates to the field of image processing technology. More specifically, the present application relates to a method, device and computer-readable storage medium for sitting posture scoring. Background Art
[0002] Nowadays, with the popularity of electronic devices and the increase in academic burden, adolescent health issues have received widespread attention, including the negative impact of bad sitting posture. Especially in elementary school, long-term incorrect sitting posture can cause problems such as decreased vision and scoliosis. However, in actual situations, it is difficult for teachers to ensure that the sitting posture of each student is taken into account. On the one hand, there are a large number of students in a class, and on the other hand, there is a lack of scientific methods to accurately evaluate students' sitting posture. Therefore, it is of great practical significance to provide a solution that can automatically evaluate students' sitting posture.
[0003] At present, the sitting posture scoring based on deep learning first needs to detect the student target from the monitoring screen, extract the facial features and key point coordinates, then use the facial features to match the identity, and use the key point coordinates to calculate the sitting posture score. However, the pixels of the target far away from the camera are too small, resulting in problems such as missed detection and inaccurate key point recognition.
[0004] In view of this, there is an urgent need to provide a solution for sitting posture scoring so as to improve the detection accuracy of small targets in the picture, thereby improving the accuracy of sitting posture scoring. Summary of the invention
[0005] In order to at least solve one or more technical problems mentioned above, the present application proposes a solution for sitting posture scoring in multiple aspects.
[0006] In a first aspect, the present application provides a method for sitting posture scoring, comprising: collecting a video dataset in a target scene; marking target detection boxes and key points in the video dataset as label data; training a key point detection model based on the video dataset and the label data, wherein the key point detection model includes a small target detection module for detecting small targets in the video dataset; obtaining video data to be detected and inputting the video data to be detected into a trained key point detection model for detection to obtain key point information; and calculating the sitting posture score of the target in the target scene based on the key point information.
[0007] In some embodiments, the target scenario includes at least classroom teaching.
[0008] In other embodiments, the key points include at least the target's nose, eyes, ears, shoulders, elbows, wrists, head and neck.
[0009] In some further embodiments, the key point detection model comprises a yolov8 model.
[0010] In some further embodiments, the small target detection module is disposed after the target feature layer of the key point detection model, and the small target detection module includes a plurality of upsampling layers and a plurality of depth-separable convolutional layers, and each of the upsampling layers adopts bilinear interpolation.
[0011] In some further embodiments, the key point information includes at least key point coordinates and a visibility score.
[0012] In some other embodiments, calculating the sitting posture score of the target in the target scene based on the key point information includes: judging whether the target lowers its head, turns its head, and turns its body based on the key point coordinates and the visibility score; scoring based on the degree of lowering its head, turning its head, and turning its body to obtain a target score; and subtracting the target score from the initial score to obtain a sitting posture score.
[0013] In some other embodiments, it also includes: based on the video data to be detected, using a face recognition model to extract facial features of the target; matching the facial features with the identity information of the target to obtain information matching results for each target; and associating the sitting posture score of each target with the corresponding information matching result.
[0014] In a second aspect, the present application provides a device for sitting posture scoring, comprising: a processor; and a memory on which computer instructions for sitting posture scoring are stored, and when the computer instructions are executed by the processor, one or more embodiments of the aforementioned first aspect are implemented.
[0015] In a third aspect, the present application provides a computer-readable storage medium, which includes computer program instructions for sitting posture scoring stored thereon, and when the computer program instructions are executed by one or more processors, one or more embodiments of the first aspect are implemented.
[0016] Through the solution for sitting posture scoring provided above, the embodiment of the present application reduces the missed detection of targets in the target scene that are far away from the camera, for example, by adding a small target detection module for detecting small targets in the video data set to the key point detection model. As a result, the detection accuracy of small targets in the picture is improved, the detection probability of key points is increased, the accuracy of key point detection by the key point detection model is improved, and more accurate key point information is obtained. Then, the sitting posture score of the target is calculated based on the key point information, thereby improving the accuracy of the sitting posture score. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present application will become easy to understand. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:
[0018] Figure 1 is an exemplary flowchart showing a method for sitting posture scoring according to an embodiment of the present application;
[0019] Figure 2 is an exemplary flowchart showing the overall process for sitting posture scoring according to an embodiment of the present application;
[0020] Figure 3 is another exemplary flow chart showing the overall process for sitting posture scoring according to an embodiment of the present application
[0021] Figure 4 is an exemplary structural block diagram showing a device 400 for sitting posture scoring according to an embodiment of the present application. DETAILED DESCRIPTION
[0022] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.
[0023] It should be understood that the terms "include" and "comprising" used in the specification and claims of the present application indicate the presence of described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.
[0024] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this application specification and claims, unless the context clearly indicates otherwise, the singular forms of "a", "an" and "the" are intended to include plural forms. It should also be further understood that the term "and / or" used in this application specification and claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.
[0025] As used in this specification and claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.
[0026] The specific implementation of the present application is described in detail below with reference to the accompanying drawings.
[0027] Figure 1 FIG. 1 is an exemplary flowchart of a method 100 for scoring a sitting posture according to an embodiment of the present application. Figure 1 As shown in, the method 100 includes: step S101: collecting a video data set in a target scene; step S102: marking target detection boxes and key points in the video data set as label data; step S103: training a key point detection model based on the video data set and the label data, wherein the key point detection model includes a small target detection module for detecting small targets in the video data set; step S104: obtaining the video data to be detected and inputting the video data to be detected into the trained key point detection model for detection to obtain key point information; and step S105: calculating the sitting posture score of the target in the target scene according to the key point information.
[0028] First, at step S101, a video data set in a target scene is collected. In some embodiments, the target scene may include at least classroom teaching. Preferably, the target scene is for primary school classroom teaching. In some embodiments, video data is collected by setting, for example, a camera in the target scene. After obtaining the video stream address of the monitoring point, a tool such as FFmpeg, OpenCV, etc. can be used to record the video, and the video frames can be intercepted at certain time intervals. In other embodiments, the image data in the target scene can also be directly collected.
[0029] Then, at step S102, the target detection frame and key points are marked in the video data set as label data. In some embodiments, the aforementioned key points may include but are not limited to the target's nose, eyes, ears, shoulders, elbows, wrists, head and neck, etc. More specifically, the aforementioned key points may include the target's nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, head and neck, etc. It can be understood that in the target scene of classroom teaching, the target is a student in the classroom. In some implementation scenarios, the target detection frame and key points in the video can be manually marked or automatically marked, and corrected by, for example, the coco-annatator annotation platform, and the marked data can be exported to json format as label data. In other embodiments, open source key point annotation data can also be used as a supplementary sample.
[0030] Further, at step S103, a key point detection model is trained based on the video data set and the label data. Among them, the key point detection model includes a small target detection module for detecting small targets in the video data set. The small target refers to a target with fewer pixels or a smaller target detection box. In some embodiments, the key point detection model can be, for example, a yolov8 model. In some embodiments, the small target detection module can be set after the target feature layer of the key point detection model, and the small target detection module includes multiple upsampling layers and multiple depth-separable convolutional layers, and each upsampling layer uses bilinear interpolation.
[0031] It can be understood that the yolov8 model includes a backbone network and a task head, wherein the backbone network is used to extract image features, and the task head varies according to different tasks. For example, in an embodiment of the present application, the task head corresponds to key point detection. The aforementioned backbone network comprises 0-9 layers, each of which is composed of Conv, C2f and / or SPPF. The task head comprises 10-21 layers, which can be composed of a classifier and / or a regressor. In the implementation scenario, after extracting features in the yolov8 model, P1-P5 feature layers can be formed. Preferably, the embodiment of the present application can increase the upsampling layer correspondingly after the P3, P4 and P5 feature layers, and fuse the low-level features during the upsampling process, and then extract the key point features through multiple depth-separable convolutional layers.
[0032] Through the above upsampling operation, the high-level feature map can be enlarged to the same or close spatial resolution as the low-level feature map, ensuring the preservation of details during information transmission. The upsampled feature map is fused with the corresponding low-level feature map by splicing to form a new feature representation containing rich semantic information and high-resolution details. As an example, assuming that the upsampled features are p3, p4 and p5 in sequence, the p3 feature is spliced with the feature in the P3 feature layer. Similarly, the p4 feature is spliced with the feature in the P4 feature layer, and the p5 feature is spliced with the feature in the P5 feature layer. This cross-layer connection mechanism promotes the communication between features at different levels, can enhance the overall expression ability of the key point detection model, and helps to alleviate the gradient vanishing problem that may occur during deep network training. Furthermore, multiple depth-separable convolutional layers are connected after the upsampling layer, first performing spatial convolution on each input channel separately, and then performing point-by-point convolution on the results of all channels. This design greatly reduces the number of parameters and computational complexity, while maintaining or even improving the feature expression ability. Therefore, the embodiment of the present application can improve the detection accuracy of small targets in the picture by adding a small target detection module to the key point detection model, thereby improving the accuracy of key point detection. Finally, the result after point-by-point convolution can be detected by the target detection task head to obtain key point information.
[0033] In some implementation scenarios, the video data set and the label data are divided into a training set and a test set. Based on the training set, the key point detection model is trained forward and backward to output key point information to obtain a trained key point detection model. The aforementioned test set can be used to test the performance of the trained key point detection model.
[0034] Based on the trained key point detection model, at step S104, the video data to be detected is obtained and input into the trained key point detection model for detection to obtain key point information. That is, based on the video data to be detected, the key point information is obtained by reasoning with the trained key point detection model. In some embodiments, the key point information may include but is not limited to key point coordinates (including horizontal and vertical coordinates) and visibility scores, etc., wherein the visibility score is, for example, a score for whether the key point is visible, and its range is between 0-1. In some implementation scenarios, the key information can be represented by a two-dimensional array, such as points
[13] [3]. Among them, 13 represents 13 key points (i.e.), and 3 represents the three values of the horizontal coordinate, vertical coordinate and visibility score of the key point. Specifically, the order of the 13 key points is 0: nose; 1: left eye; 2: right eye; 3: left ear; 4: right ear; 5: left shoulder; 6: right shoulder; 7: left elbow; 8: right elbow; 9: left wrist; 10: right wrist; 11: top of the head (or center of the head); 12: neck.
[0035] Further, at step S105, the sitting posture score of the target in the target scene is calculated according to the key point information. In some embodiments, it can be determined whether the target has lowered its head, turned its head, or turned its body according to the key point coordinates and the visibility score, and then the target score is obtained based on the degree of lowering its head, turning its head, and turning its body, and the target score is subtracted from the initial score to obtain the sitting posture score. In some implementation scenarios, the initial score is 100 points.
[0036] Specifically, for bowing the head, it can be judged whether the vertical coordinate of the nose is greater than the vertical coordinate of the left shoulder or the right shoulder. If the vertical coordinate of the nose is greater than the vertical coordinate of the left shoulder or the right shoulder, it is judged as bowing the head; otherwise, it is not bowing the head. Points are deducted according to the specific degree of bowing the head, which can be specifically expressed by the following formula:
[0037] score-=0.3×(point
[12] [1]-min(points[1][1]-oint[2][1]))(1)
[0038] For head turning, it can be judged based on the visibility scores of the key points nose, eyes, and ears. For example, if the visibility score of any of the key points nose, eyes, and ears is lower than a threshold (e.g., 0.2), it is judged that the head is turned. When the visibility scores of the key points nose, eyes, and ears are all greater than the threshold, it is judged that there is no head turn. Furthermore, different degrees of deduction are made according to the head turning angle.
[0039] As an example, if the abscissa of the right eye is less than or equal to the abscissa of the right ear, and the abscissa of the left eye is greater than or equal to the abscissa of the left ear, and the visibility scores of the nose, left eye, and right eye are lower than the threshold value of 0.2, then the head is considered to have completely turned to the side, that is, the head turning angle exceeds 135 degrees. In this case, the score will be deducted by 100 points because this is a very serious posture problem. If the abscissa of the right eye is smaller than the abscissa of the right ear, but does not meet the above-mentioned large head turning condition, the head is considered to be turned to the left, and the head turning angle is between 45 and 135 degrees. At this time, the score will be deducted by 20 points. If the abscissa of the left eye is greater than the abscissa of the left ear, and also does not meet the large head turning condition, the head is considered to be turned to the right, and the head turning angle is also between 45 and 135 degrees. At this time, the score will also be deducted by 20 points. In an implementation scenario, it can be expressed by the following formula:
[0040]
[0041] For body turning, the relative position between the left shoulder and the right shoulder can be checked to determine whether the body has turned more than 90 degrees. For example, if the horizontal coordinate of the right shoulder is greater than or equal to the horizontal coordinate of the left shoulder, the body is considered to have turned 90 degrees or more. In this case, the score will be deducted by 100 points because this is a very serious posture problem. If the body rotation angle is less than 90 degrees, further subdivision is required for more precise evaluation. In some implementation scenarios, the angle can be measured using the tangent value by calculating the slope of the line connecting the left shoulder and the right shoulder. If the angle is between 20-45 degrees, the score is deducted by 20 points, and if it is greater than 45 degrees, 50 points are deducted. In addition, if too many points are deducted and the final score is less than 0, it will be recorded as 0 points.
[0042] Combined with the above description, it can be seen that the embodiment of the present application improves the key point detection model and adds a small target detection module, thereby improving the detection accuracy of small targets and reducing the missed detection of student targets that are far away from the camera in the target scene, so as to improve the accuracy of key point detection and thus improve the accuracy of sitting posture scoring. Furthermore, the embodiment of the present application can further improve the accuracy of sitting posture scoring by combining key point information (such as coordinates and visibility scores) for score quantitative evaluation.
[0043] Figure 2 FIG. 1 is an exemplary flowchart showing the overall process for sitting posture scoring according to an embodiment of the present application. Figure 2 As shown in FIG. 1 , at step S201, a video data set in a target scene is collected. Based on the video data set, at step S202, the video data set is annotated, including a target detection frame and key points. For specific key points, please refer to the above Figure 1 Description made. After annotating the video data set, the annotated video data set is divided into a training set and a test set. At step S203, the training set is used to train the key point detection model, and then at step S204, the trained key point detection model is obtained. In some embodiments, the key point detection model may be, for example, a yolov8 model. By adding a small target detection module to the target feature layer of the yolov8 model, the detection accuracy of small targets can be improved. Preferably, a small target detection module can be placed after, for example, the P3, P4 and P5 feature layers, and the small target detection module may include multiple upsampling layers and multiple depth-separable convolutional layers, wherein each upsampling layer uses bilinear interpolation.
[0044] Based on the trained key point detection model, at step S205, the video data to be detected is obtained, and the video data to be detected is input into the trained key point detection model for detection, so as to obtain the key point information at step S206. In some embodiments, the key point information may include key point coordinates and visibility scores. Then, at step S207, it is determined whether there is head lowering, head turning, and body turning according to the key point information, and the degree of head lowering, head turning, and body turning is determined, and the corresponding target score is calculated. For more details about the above judgment, please refer to the above Figure 1 For the situations of head lowering, head turning and body turning and the corresponding target scores, at step S208, the target score is subtracted from the initial score to obtain the sitting posture score.
[0045] In some embodiments, the embodiments of the present application also involve extracting facial features of a target using a face recognition model based on the video data to be detected, matching the facial features with the identity information of the target, obtaining information matching results for each target, and associating the sitting posture score of each target with the corresponding information matching results. In other words, facial features are obtained through face recognition, the facial features are matched with the identity of the target, the target is matched with the face, and then the sitting posture score is associated with the corresponding target.
[0046] In some embodiments, the identity information of the aforementioned target includes but is not limited to the basic information such as the target's name, student number, grade and class. In some embodiments, the aforementioned face recognition model may be, for example, an open source SeetaFace2 model. The SeetaFace2 model may include a face detection module, a facial key point positioning module, and a face feature extraction and comparison module. In the implementation scenario, the student target frame can be obtained by cutting out the corresponding area from the original video frame, and then using, for example, the SeetaFace2 model for face detection and face recognition to obtain the face features corresponding to each target frame. Based on the face features, it is compared with the face features in the database to obtain the identity information of the target corresponding to each target frame, thereby obtaining the information matching results of each target. Specifically, the basic information and face features of all students in the current class are queried from the database, and the comparison module of the SeetaFace2 model is used to calculate the similarity of each target frame with all the face features of the current class one by one. For each target frame, the student with the highest similarity in face features is selected as the corresponding student of the current target frame.
[0047] Furthermore, the student information obtained through face recognition and the corresponding sitting posture score are stored in a database, and the average score is calculated for all records of each student on that day. For students with low sitting posture scores, teachers and parents pay special attention to them.
[0048] Figure 3 FIG. 2 is another exemplary flow chart showing the overall process for sitting posture scoring according to an embodiment of the present application. Figure 3 As shown in FIG. 1 , at step S301, a video data set in a target scene is collected. Based on the video data set, at step S302, the video data set is annotated, including a target detection frame and key points. For specific key points, please refer to the above Figure 1 After annotating the video data set, the annotated video data set is divided into a training set and a test set. At step S303, the training set is used to train the key point detection model, and then at step S304, the trained key point detection model is obtained. In some embodiments, the key point detection model can be, for example, a yolov8 model. By adding a small target detection module to the target feature layer of the yolov8 model, the detection accuracy of small targets can be improved. For more details about the small target detection module, please refer to the above Figure 1 The description made will not be repeated in this application.
[0049] Based on the trained key point detection model, at step S305, the video data to be detected is obtained, and the video data to be detected is input into the trained key point detection model for detection, so as to obtain key point information at step S306. In some embodiments, the key point information may include key point coordinates and visibility scores. Then, at step S307, the sitting posture score is calculated according to the key points. For more details about the sitting posture score, please refer to the above Figure 1 The description made will not be repeated in this application.
[0050] Furthermore, at step S308, based on the collected video data to be detected, the face recognition model can be used to extract the face features of the target, and at step S309, the face features are compared with the face features in the database to obtain the identity information of the target, so as to obtain the information matching results of each target. Finally, at step S310, the sitting posture score of each target is associated with the corresponding information matching result for subsequent query.
[0051] Based on the foregoing, it can be seen that the embodiment of the present application improves the key point detection model and adds a small target detection module, thereby improving the detection accuracy of small targets and reducing the missed detection of student targets that are far away from the camera in the target scene, so as to improve the accuracy of key point detection, and thus improve the accuracy of sitting posture scoring. Furthermore, the embodiment of the present application adds multiple upsampling layers and multiple depth-separable convolution layers after the target feature layer of the key point detection model to retain more detailed features, thereby facilitating the recognition of small targets. Furthermore, the embodiment of the present application also calculates the sitting posture score by combining key point information (such as coordinates and visibility scores), and quantifies the scores of corresponding situations through key point information, which can further improve the accuracy of sitting posture scoring.
[0052] Figure 4 is an exemplary structural block diagram of a device 400 for sitting posture scoring according to an embodiment of the present application. It is understood that the device 400 may include an apparatus of an embodiment of the present application, and the device implementing the solution of the present application may be a single device (such as a computing device) or a multifunctional device including various peripheral devices.
[0053] like Figure 4 As shown in , the device of the present application may also include a central processing unit or central processing unit ("CPU") 411, which may be a general-purpose CPU, a dedicated CPU, or other information processing and program execution unit. Further, the device 400 may also include a large-capacity memory 412 and a read-only memory ("ROM") 413, wherein the large-capacity memory 412 may be configured to store various types of data, including various video data sets, target detection boxes and key points, key point information, sitting posture scores, algorithm data, intermediate results, and various programs required to run the device 400. ROM 413 may be configured to store data and instructions required for power-on self-test of the device 400, initialization of various functional modules in the system, drivers for basic input / output of the system, and booting the operating system.
[0054] Optionally, the device 400 may also include other hardware platforms or components, such as the tensor processing unit ("TPU") 414, the graphics processing unit ("GPU") 415, the field programmable gate array ("FPGA") 416, and the machine learning unit ("MLU") 417 shown. It is understood that although a variety of hardware platforms or components are shown in the device 400, they are only exemplary and not restrictive, and those skilled in the art can add or remove corresponding hardware according to actual needs. For example, the device 400 may only include a CPU, related storage devices, and interface devices to implement the method for sitting posture scoring of the present application.
[0055] In some embodiments, in order to facilitate the transmission and interaction of data with an external network, the device 400 of the present application further includes a communication interface 418, so that it can be connected to a local area network / wireless local area network ("LAN / WLAN") 405 through the communication interface 418, and then connected to a local server 406 or to the Internet ("Internet") 407 through the LAN / WLAN. Alternatively or additionally, the device 400 of the present application can also be directly connected to the Internet or a cellular network through the communication interface 418 based on wireless communication technology, such as wireless communication technology based on the third generation ("3G"), the fourth generation ("4G") or the fifth generation ("5G") of the third generation ("3G"), the fourth generation ("4G") or the fifth generation ("5G") of the third generation ("3G"), the fourth generation ("4G") or the fifth generation ("5G") of the third generation ("3G"), the fourth generation ("4G") or the fifth generation ("5G") of the third generation ("3G") of the third generation ("3G"), the fourth generation ("4G") or the fifth generation ("5G") of the third generation ("3G") of the third generation ("3G") of the fourth generation ("4G") of the fourth generation ("4G") of the fourth generation ("4G") of the fourth generation ("5 ...
[0056] The peripheral devices of the device 400 may include a display device 402, an input device 403, and a data transmission interface 404. In one embodiment, the display device 402 may include, for example, one or more speakers and / or one or more visual displays, which are configured to perform voice prompts and / or image video display for the sitting posture scoring of the present application. The input device 403 may include other input buttons or controls such as a keyboard, a mouse, a microphone, a posture capture camera, etc., which are configured to receive input of audio data and / or user instructions. The data transmission interface 404 may include, for example, a serial interface, a parallel interface or a universal serial bus interface ("USB"), a small computer system interface ("SCSI"), a serial ATA, a FireWire ("FireWire"), a PCI Express, and a high-definition multimedia interface ("HDMI"), etc., which are configured for data transmission and interaction with other devices or systems. According to the solution of the present application, the data transmission interface 404 can receive video data collected from a camera installed in the target scene, and transmit data or results including a video data set or various other types to the device 400.
[0057] The CPU 411, mass storage 412, ROM 413, TPU 414, GPU 415, FPGA 416, MLU 417 and communication interface 418 of the device 400 of the present application can be interconnected through a bus 419, and data interaction with peripheral devices can be achieved through the bus. In one embodiment, through the bus 419, the CPU 411 can control other hardware components in the device 400 and its peripheral devices.
[0058] Combination of the above Figure 4The device for sitting posture scoring that can be used to implement the present application is described. It should be understood that the device structure or architecture here is only exemplary, and the implementation method and implementation entity of the present application are not limited thereto, but can be changed without departing from the spirit of the present application.
[0059] According to the above description in combination with the accompanying drawings, those skilled in the art can also understand that the embodiments of the present application can also be implemented by software programs. Therefore, the present application also provides a computer-readable storage medium, on which computer-readable instructions for sitting posture scoring are stored. When the computer-readable instructions are executed by one or more processors, they can be used to implement the present application in combination with the accompanying drawings. Figure 1 The method described for sitting posture scoring.
[0060] It should be noted that although the operations of the method of the present application are described in a specific order in the accompanying drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. On the contrary, the steps depicted in the flow chart can be performed in a different order. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.
[0061] It should be understood that when the terms "first", "second", "third" and "fourth" are used in the claims, the specification and the drawings of the present application, they are only used to distinguish different objects, rather than to describe a specific order. The terms "include" and "comprise" used in the specification and claims of the present application indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections.
[0062] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this application specification and claims, unless the context clearly indicates otherwise, the singular forms of "a", "an" and "the" are intended to include plural forms. It should also be further understood that the term "and / or" used in this application specification and claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.
[0063] Although the implementation methods of the present application are as above, the contents described are only examples adopted to facilitate the understanding of the present application, and are not intended to limit the scope and application scenarios of the present application. Any technician in the technical field described in the present application can make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in the present application, but the scope of patent protection of the present application shall still be subject to the scope defined in the attached claims.
[0064] In addition, the collection and acquisition of various data in this application complies with relevant laws and regulations and is authorized by the data provider. Any organization or individual that needs to obtain external data must obtain authorization in accordance with the law and ensure data security. It is not allowed to illegally collect, use, process, or transmit unauthorized or unprotected data, or to illegally buy, sell, provide, or disclose unauthorized or unprotected data.
Claims
1. A method for scoring sitting posture, comprising: Collect video datasets in target scenes; Annotating target detection frames and key points in the video data set as label data; Training a key point detection model based on the video data set and the label data, wherein the key point detection model includes a small target detection module for detecting small targets in the video data set; Acquire the video data to be detected and input the video data to be detected into a trained key point detection model for detection to obtain key point information; as well as A sitting posture score of the target in the target scene is calculated according to the key point information.
2. The method according to claim 1, wherein the target scenario at least includes classroom teaching.
3. The method according to claim 1, wherein the key points include at least the target's nose, eyes, ears, shoulders, elbows, wrists, head and neck.
4. The method according to claim 1, wherein the key point detection model comprises a yolov8 model.
5. A method according to claim 1 or 4, wherein the small target detection module is arranged after the target feature layer of the key point detection model, and the small target detection module includes multiple upsampling layers and multiple depth-separable convolutional layers, and each of the upsampling layers adopts bilinear interpolation. The method according to claim 1 , wherein the key point information comprises at least key point coordinates and visibility scores.
7. The method according to claim 6, wherein calculating the sitting posture score of the target in the target scene according to the key point information comprises: Determine whether the target lowers its head, turns its head, or turns its body according to the key point coordinates and the visibility score; Scoring is done based on the degree of head lowering, head turning, and body turning to obtain a target score; as well as The target score was subtracted from the initial score to obtain a sitting score.
8. The method according to claim 1, further comprising: Based on the video data to be detected, using a face recognition model to extract facial features of the target; Matching the facial features with the identity information of the target to obtain information matching results for each target; as well as Associate the sitting posture score of each target with the corresponding information matching result.
9. A device for scoring sitting posture, comprising: processor; as well as A memory having computer instructions for sitting posture scoring stored thereon, wherein when the computer instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.
10. A computer-readable storage medium, comprising computer program instructions for sitting posture scoring stored thereon, wherein when the computer program instructions are executed by one or more processors, the method according to any one of claims 1 to 8 is implemented.