Hand raising behavior detection method and device, equipment and storage medium
By calculating the relative position of the head and wrist and the relative angle between the arm and shoulder, combined with the comparison model of the length of the head and arm, the accurate detection of the hand-raising behavior is achieved, and the problem of low accuracy in the prior art is solved, and it is suitable for edge devices with insufficient computing power.
Patent Information
- Application Number
- CN202411710108.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-27
- Publication Date
- 2025-03-18
AI Technical Summary
The accuracy of the hand-raising detection method in the prior art is low, especially on edge devices with insufficient computing power, it is difficult to achieve efficient detection.
By obtaining the key points of the shoulders of the detection target, calculate the relative position between the head and the wrist and the relative angle between the arm and the shoulders of the arm and shoulders, select the relative angle threshold based on these parameters to determine whether the hand-raising behavior occurs. At the same time, the correctness of key points in the human body is checked through the comparison model of head length and arm length.
The accuracy of hand-raising detection is improved, and the demand for computing power is reduced, making this method suitable for more scenarios, especially edge devices.
Smart Images
Figure CN119649451A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of video image processing technology, and more specifically, to a method, apparatus, device, and storage medium for detecting hand-raising behavior. Background Art
[0002] With the development of information technology in education, it has become increasingly common to observe and statistically analyze teacher-student interactions in classrooms using automated and intelligent methods. This allows for the monitoring and objective evaluation of each student's learning status, leading to targeted improvements in teaching management and teacher quality. Raising one's hand is the most typical student behavior in the classroom, the most direct expression of active participation, and the most direct way for teachers and students to interact. It can serve as an effective method for evaluating student learning status and teacher teaching quality.
[0003] Currently, the application logic of pose recognition methods based on human keypoint detection in common teaching and research recording scenarios is generally as follows: ① Use human keypoint detection algorithms to detect the skeletal keypoints of all candidate targets. For example, in detecting hand-raising behavior, it is necessary to first detect keypoints such as the head, shoulder joints, elbow joints, and wrist joints; ② Use the detected keypoints to set different pose judgment methods according to different geometric arrangements; ③ Or use a pose classification model to directly classify the detected human targets. An important prerequisite for the above schemes is that the detection results of the human keypoint detection algorithm in step ① have a high accuracy rate. To meet this condition, common methods include: ① Using algorithm models with large parameters and computational load in single-stage methods, which generally have high accuracy; ② In two-stage methods, first use human detection algorithms to obtain the bounding boxes of the human body in the image, and then pass the targets in the bounding boxes to a high-precision single-target human keypoint detector; both of these methods can suppress the false detection rate to some extent. However, these schemes obviously have high computational requirements, which is not friendly to edge devices with insufficient computing power. Summary of the Invention
[0004] To overcome the low accuracy of current hand-raising detection methods, this invention provides a hand-raising behavior detection method, device, equipment, and storage medium. The technical solution adopted by this invention is as follows.
[0005] In a first aspect, the present invention provides a method for detecting hand-raising behavior, comprising:
[0006] Obtain the key shoulder points of the target object;
[0007] Calculate the relative positions between the head key points and the wrist key points;
[0008] Calculate the relative angle between the arm and shoulder, wherein the relative angle between the arm and shoulder is the angle formed by the intersection of the line connecting the wrist key point and the elbow key point and its extension with the line connecting the two shoulders and its extension.
[0009] Based on the relative position between the head key point and the wrist key point, a corresponding relative angle threshold is selected, and based on the comparison result of the selected relative angle threshold and the relative angle of the arm and shoulder, it is determined whether the target has raised its hand.
[0010] In one implementation, the process of selecting a corresponding relative angle threshold based on the relative position between the head key point and the wrist key point, and determining whether the target has raised its hand based on a comparison between the selected relative angle threshold and the relative angle between the arm and shoulder, includes:
[0011] When the wrist key point is lower than the head key point but higher than the neck key point, and the distance between the head key point and the wrist key point is greater than a preset distance threshold, a first relative angle threshold is selected.
[0012] When the relative angle between the arm and shoulder is greater than the first relative angle threshold, it is determined that the target has raised its hand.
[0013] In one implementation, the process of selecting a corresponding relative angle threshold based on the relative position between the head key point and the wrist key point, and determining whether the target has raised its hand based on a comparison between the selected relative angle threshold and the relative angle between the arm and shoulder, includes:
[0014] When the wrist key point is lower than the neck key point, and the distance between the head key point and the wrist key point is greater than a preset distance threshold, a second relative angle threshold is selected;
[0015] When the relative angle between the arm and shoulder is greater than the second relative angle threshold, it is determined that the target has raised its hand. The second relative angle threshold is greater than or equal to the first relative angle threshold.
[0016] In one implementation, the process of selecting a corresponding relative angle threshold based on the relative position between the head key point and the wrist key point, and determining whether the target has raised its hand based on a comparison between the selected relative angle threshold and the relative angle between the arm and shoulder, includes:
[0017] When the wrist key point is higher than the head key point, a first relative angle threshold is selected;
[0018] When the relative angle between the arm and shoulder is greater than the first relative angle threshold, it is determined that the target has raised its hand.
[0019] In one implementation, the preset distance threshold is greater than or equal to the distance from the head key point to the neck key point.
[0020] In one implementation, it further includes:
[0021] Acquire the head key points, neck key points, and elbow and wrist key points on the same side of the target to be detected;
[0022] The method of checking the detected human body key points is used to check the detected human body key points. If the conclusion that the detected human body key points are correct cannot be drawn, the use of the key points for the hand raising behavior detection is terminated.
[0023] The method for detecting key points on the human body includes:
[0024] Acquire key points of the head and neck of the target being detected;
[0025] Calculate the distance from the head key point to the neck key point, and record it as the head length;
[0026] Acquire key points of the elbow and wrist on the same side of the target being detected;
[0027] Calculate the distance from the elbow key point to the wrist key point, and record it as the length of one arm;
[0028] Based on a preset comparison model, the head length is compared with the length of the arm on one side to obtain a relative scale estimate.
[0029] When the relative scale estimate is within the preset scale threshold range, the detected human key points are determined to be correct.
[0030] In one embodiment, the preset comparison model in the method for examining detected key points of the human body is:
[0031] Where β is the relative scale estimate, and D H D is the head length. A The length is the arm length, and W and b are preset parameters.
[0032] In one embodiment, the method for examining detected key points on the human body further includes:
[0033] If the head length is not within the preset head length range, the human body key points of the current detection target are determined to be unusable;
[0034] If the length of the arm on one side is not within the preset arm length range, it is determined that the human body key points of the current detection target are unavailable;
[0035] If it is determined that the human body key points of the current detection target are unavailable, the detected human body key points are determined to be incorrect.
[0036] Secondly, the present invention provides a hand-raising behavior detection device, comprising:
[0037] The first acquisition module is used to acquire the key shoulder points of the target's shoulders;
[0038] The calculation module is used to calculate the relative position between the head key point and the wrist key point; and to calculate the arm-shoulder relative angle between the arm and the shoulder, wherein the arm-shoulder relative angle is the angle formed by the intersection of the line connecting the wrist key point and the elbow key point and its extension with the line connecting the shoulders and its extension.
[0039] The first judgment module is used to select a corresponding relative angle threshold based on the relative position between the head key point and the wrist key point, and to determine whether the target has made a hand-raising behavior based on the comparison result of the selected relative angle threshold and the relative angle of the arm and shoulder.
[0040] Thirdly, the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method of any of the above embodiments.
[0041] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that the program, when executed by a processor, implements the method of any of the above embodiments.
[0042] In this invention, for classroom teaching scenarios, different preset relative angle thresholds are selected and compared with the relative angle between the head and wrist key points based on their relative positions. The comparison results determine whether a hand-raising behavior has occurred. Simultaneously, the relationship between head length and arm length is used to check the accuracy of the detected human key points in a relatively simple way. Compared to existing methods, this invention requires less computing power, is more accurate, and is applicable to a wider range of scenarios. Attached Figure Description
[0043] Figure 1 This is a flowchart illustrating one of the embodiments of the present invention.
[0044] Figure 2 This is a flowchart illustrating the inspection method according to one embodiment of the present invention.
[0045] Figure 3 This is a schematic diagram of the overall structure of an embodiment of the second embodiment of the present invention.
[0046] Figure 4This is a schematic diagram of the overall structure of the inspection device according to Embodiment 2 of the present invention. Detailed Implementation
[0047] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
[0048] It should be noted that the terms "first, second, ..." used in the embodiments of the present invention are merely used to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, ..." can be interchanged in a specific order or sequence where permissible. It should be understood that the objects distinguished by "first, second, ..." can be interchanged where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein.
[0049] Example 1
[0050] See Figure 1 , Figure 1 This is a flowchart illustrating a hand-raising behavior detection method according to Embodiment 2 of the present invention. The method includes steps S110, S120, S130, and S140. It should be noted that steps S110, S120, S130, and S140 are merely reference numerals used to clearly explain the embodiment and the accompanying drawings. Figure 2 The correspondence is not intended to limit the order of steps in this embodiment.
[0051] Step S110: Obtain the key points of the shoulders of the target.
[0052] Step S120: Calculate the relative positions between the head key points and the wrist key points;
[0053] Step S130: Calculate the relative angle between the arm and shoulder, wherein the relative angle between the arm and shoulder is the angle formed by the intersection of the line connecting the wrist key point and the elbow key point and its extension with the line connecting the two shoulders and its extension.
[0054] Step S140: Select the corresponding relative angle threshold based on the relative position between the head key point and the wrist key point, and determine whether the target has raised its hand based on the comparison result of the selected relative angle threshold and the relative angle of the arm and shoulder.
[0055] This method is applicable to devices capable of analyzing human movements, such as recording and broadcasting hosts. For ease of explanation, the following description uses a recording and broadcasting host as an example. The recording and broadcasting host acquires the captured video stream from the recording and broadcasting terminal, and uses a human keypoint detection algorithm to perform full-image detection on the video frames in the video stream, obtaining the skeletal keypoints of all human bodies in the video frames and the corresponding human information.
[0056] In recorded classroom teaching and research scenarios, the main reference for detecting whether the target has made a gesture such as raising their hand is the relative position of the head and wrist, the relative position of the wrist and elbow, and the relative angle between the arm and shoulder. The relative position of the head and wrist includes their height relationship and distance. The distance between the key points of the head and the key points of the wrist is denoted as D. EH The angle formed by the intersection of the line connecting the wrist and elbow and its extension with the line connecting the shoulders and its extension is denoted as the relative angle between the arm and shoulder, i.e., the arm-shoulder relative angle A. AS .
[0057] To differentiate this from actions easily confused with raising one's hand, such as resting one's chin on one's hand or touching one's forehead, this method selects different preset relative angle thresholds based on the relative positions of key points on the head and wrist, for the relative angle A between the arm and shoulder. AS By comparing with [previous method], the most accurate hand-raising behavior detection results can be obtained.
[0058] In one embodiment, step S140 includes steps S141 and S142.
[0059] Step S141: When the wrist key point is lower than the head key point but higher than the neck key point, and the distance between the head key point and the wrist key point is greater than a preset distance threshold, a first relative angle threshold is selected.
[0060] Step S142: When the relative angle between the arm and shoulder is greater than the first relative angle threshold, it is determined that the target has raised its hand.
[0061] In this embodiment, when the wrist is below the head key point but above the neck key point, this indicates that the target is likely raising their hand, or performing an action easily confused with raising their hand, such as resting their chin or forehead on their hand. Therefore, it is necessary to further determine the distance between the head and the wrist. The determination criterion is the distance D between the head key point and the wrist key point. EH It must be greater than the preset distance threshold β EH After determining that the conditions are met, it is then determined whether the relative angle A between the arm and shoulder is satisfied. AS Is the angle greater than the preset first relative angle threshold δ1? If it is greater than the preset first relative angle threshold, then it is determined that the detected target has raised its hand.
[0062] In one embodiment, step S140 includes steps S143 and S144.
[0063] Step S143: When the wrist key point is lower than the neck key point and the distance between the head key point and the wrist key point is greater than a preset distance threshold, a second relative angle threshold is selected.
[0064] Step S144: When the relative angle between the arm and shoulder is greater than the second relative angle threshold, it is determined that the target has made a hand-raising action, wherein the second relative angle threshold is greater than or equal to the first relative angle threshold.
[0065] When the wrist keypoint is lower than the neck keypoint, this indicates that due to the camera's shooting angle, targets positioned in the foreground and corners will have their wrists lower relative to their heads because they are at the edge of the camera's field of view. The distance D between the head keypoint and the wrist keypoint will be... EH It must be greater than the preset distance threshold β EH Furthermore, the arm swing angle will be greater, therefore the judgment condition is that the relative angle A between the arm and the shoulder must be satisfied. AS It must be greater than the preset second relative angle threshold δ2, and the second relative angle threshold must be greater than the first relative angle threshold.
[0066] In one embodiment, step S140 includes steps S145 and S146.
[0067] Step S145: When the wrist key point is higher than the head key point, select the first relative angle threshold.
[0068] Step S146: When the relative angle between the arm and shoulder is greater than the first relative angle threshold, it is determined that the target has raised its hand.
[0069] When the wrist is higher than the head, it indicates that the target is likely raising their hand; therefore, the only condition for judgment is that the relative angle A between the arm and shoulder must be met. AS It must be greater than the first relative angle threshold δ1.
[0070] In one implementation, the preset distance threshold is greater than or equal to the distance from the head key point to the neck key point.
[0071] This implementation method specifically sets a preset distance threshold based on the length of the head, and also takes into account the actual situation.
[0072] This method combines the relative positions of key points on the head and wrist with different preset relative angle thresholds and compares them with the relative angle between the arm and shoulder. Based on the comparison results, it determines whether a hand-raising behavior has occurred. This method's judgment process is more consistent with application logic and has higher accuracy.
[0073] In one embodiment, the hand-raising behavior detection method further includes steps S150 and S160.
[0074] Step S150: Obtain the head key points, neck key points, and elbow key points and wrist key points on the same side of the target.
[0075] Step S160: Use the method of checking the detected human body key points to check the detected human body key points. If the conclusion that the detected human body key points are correct cannot be drawn, stop using the key points for hand raising behavior detection.
[0076] See Figure 2 , Figure 2 This is a flowchart illustrating a method for detecting key points on the human body according to Embodiment 1 of the present invention. The method includes steps S210, S220, S230, S240, S250, and S260. It should be noted that steps S210, S220, S230, S240, S250, and S260 are merely reference numerals used to clearly explain the embodiments and accompanying drawings. Figure 2 The correspondence is not intended to limit the order of steps in this embodiment.
[0077] Step S210: Obtain the head key points and neck key points of the target being detected;
[0078] Step S220: Calculate the distance from the head key point to the neck key point, and record it as the head length;
[0079] Step S230: Obtain the elbow key points and wrist key points on the same side of the target being detected;
[0080] Step S240: Calculate the distance from the elbow key point to the wrist key point, and record it as the length of one arm;
[0081] Step S250: Based on a preset comparison model, the head length is compared with the length of the arm on one side to obtain a relative scale estimate.
[0082] Step S260: When the relative scale estimate is within the preset scale threshold range, it is determined that the detected human body key points are correct.
[0083] Steps S150 and S160 first determine the correctness of the key points of the target. If they are incorrect, these key points will not be used for hand-raising behavior detection. It should be noted that not using these key points does not mean that the target object will not be detected. It may be that the key points of the target object will be recaptured, or a different target object will be selected for detection. Because this hand-raising behavior detection method is designed for a classroom scenario, there may be multiple people raising their hands in a classroom scenario. Therefore, if the key points of the current target are incorrect, another target can be selected.
[0084] The method for detecting human key points in this examination is applicable to devices capable of analyzing human movements, such as recording and broadcasting hosts. For ease of explanation, the following description uses a recording and broadcasting host as an example. The recording and broadcasting host acquires the captured video stream from the recording terminal and uses a human key point detection algorithm to perform full-image detection on the video frames in the video stream, obtaining the skeletal key points of all human bodies in the video frames and the corresponding human information. Specifically:
[0085] (1) The key points of the human body include: head, neck, shoulders, elbows and wrists, a total of 8 key points;
[0086] (2) In the classroom scene, the human target can generally only be seen in the upper body. Therefore, the six key points of the lower body, namely the two hips, two knees, and two ankles, are not considered. Only the key points of the upper body are calculated and decoded, which can reduce the computational overhead.
[0087] Common pose recognition methods based on skeletal keypoints often overlook a crucial prerequisite: the accuracy of the keypoints detected by the human keypoint detection algorithm. Since this algorithm cannot guarantee the correctness of every detected keypoint, a set of criteria is needed for initial screening. This method provides such a criterion for this filtering process.
[0088] In the recorded classroom teaching and research analysis scenario, the head is a relatively less obscured part of the human body. Therefore, in human keypoint detection algorithms, the probability of error is lower than for other keypoints due to less occlusion. Thus, in steps S210 and S220, using the length of the detected target's head as the scale standard for that target is appropriate. The distance between the head keypoint and the neck keypoint is calculated to obtain D. H The length of the head of the target being tested at the current scale (panoramic or close-up shots).
[0089] Compared to the head, the arms are often mis-estimated or under-estimated by human keypoint detection algorithms due to their smaller area in the image or easy occlusion. This severely affects the accuracy of subsequent pose recognition based on skeletal keypoints. Steps S230 and S240 initially assume that all human keypoints are accurately detected. If the elbow and wrist keypoints on the same side can be detected by the human keypoint detection algorithm, the distance between the elbow and wrist keypoints is calculated to obtain D. A The length of the left / right arm of the target being tested at the current scale.
[0090] In step S250, the arm length D is... A and head length D H The relative scale estimate β between the head and arm is calculated and compared using a preset comparison model formula. In step S260, the relative scale estimate β is compared with a preset scale threshold range. If the value is within the preset scale threshold range, the detected human key points are determined to be correct.
[0091] The following are some situations where targets do not meet the conditions of steps S250 and S260: First, even if the human keypoint detection algorithm correctly detects all keypoints, the target does not make a detectable behavioral pose; second, false detection of human keypoints leads to unreliable length calculation data; third, when the camera is shooting from different angles, the target's arms and other parts may appear too small or too large due to angle differences. All three situations severely affect the accuracy of subsequent keypoint-based pose recognition methods. Therefore, only when the above conditions are met can the detected human keypoints be considered correct.
[0092] In one implementation, the preset comparison model is:
[0093] Where β is the relative scale estimate, and D H D is the head length. A The length is the arm length, and W and b are preset parameters.
[0094] Here, W and b can be parameters obtained through training. Using an image dataset with labeled human key points, W and b are trained by utilizing the image scale information in the training data and the position information of each labeled key point.
[0095] In one embodiment, the method further includes steps S270, S280, and S290.
[0096] Step S270: If the head length is not within the preset head length range, determine that the human body key points of the current detection target are unavailable;
[0097] Step S280: If the length of the arm on one side is not within the preset arm length range, it is determined that the human body key points of the current detection target are unavailable.
[0098] Step S290: When it is determined that the human body key points of the current detection target are unavailable, it is determined that the detected human body key points are incorrect.
[0099] The head length and arm length are compared with their respective preset length threshold ranges. If either the head length or arm length is outside the preset length threshold range, the human keypoint of the currently detected target is considered unusable. This is because there are two situations where a target does not meet this condition: first, the target is very small in the image and does not meet the minimum standard for detection and recognition; second, false detections of human keypoints lead to unreliable length calculation data.
[0100] It should be noted that steps S270 and S280 determine whether the detected human key points are usable. If they are unusable, it means that the human key points are incorrect. Therefore, these two steps are preferably performed before steps S250 and S260. If they are determined to be unusable, there is no need to perform the subsequent steps.
[0101] In this method for detecting key human body points, specifically for classroom teaching scenarios, the relationship between head length and arm length is used to check the accuracy of the detected key human body points in a relatively simple way. Compared to existing methods, this method requires less computing power and is applicable to a wider range of scenarios.
[0102] Example 2
[0103] Corresponding to the method in Embodiment 1, such as Figure 3 As shown, the present invention also provides a hand-raising behavior detection device 3, including: a first acquisition module 310, a calculation module 320 and a first judgment module 330.
[0104] The first acquisition module 310 is used to acquire the key shoulder points of the target's shoulders;
[0105] The calculation module 320 is used to calculate the relative position between the head key point and the wrist key point; and to calculate the arm-shoulder relative angle between the arm and the shoulder, wherein the arm-shoulder relative angle is the angle formed by the intersection of the line connecting the wrist key point and the elbow key point and its extension with the line connecting the shoulders and its extension.
[0106] The first judgment module 330 is used to select a corresponding relative angle threshold based on the relative position between the head key point and the wrist key point, and to determine whether the target has made a hand-raising behavior based on the comparison result of the selected relative angle threshold and the relative angle of the arm and shoulder.
[0107] In one implementation, the first judgment module, in performing the process of selecting a corresponding relative angle threshold based on the relative position between the head key point and the wrist key point, and judging whether the target has raised its hand based on the comparison result of the selected relative angle threshold and the relative angle between the arm and shoulder, includes:
[0108] When the wrist key point is lower than the head key point but higher than the neck key point, and the distance between the head key point and the wrist key point is greater than a preset distance threshold, a first relative angle threshold is selected.
[0109] When the relative angle between the arm and shoulder is greater than the first relative angle threshold, it is determined that the target has raised its hand.
[0110] In one implementation, the first judgment module, in performing the process of selecting a corresponding relative angle threshold based on the relative position between the head key point and the wrist key point, and judging whether the target has raised its hand based on the comparison result of the selected relative angle threshold and the relative angle between the arm and shoulder, includes:
[0111] When the wrist key point is lower than the neck key point, and the distance between the head key point and the wrist key point is greater than a preset distance threshold, a second relative angle threshold is selected;
[0112] When the relative angle between the arm and shoulder is greater than the second relative angle threshold, it is determined that the target has raised its hand. The second relative angle threshold is greater than or equal to the first relative angle threshold.
[0113] In one implementation, the first judgment module, in performing the process of selecting a corresponding relative angle threshold based on the relative position between the head key point and the wrist key point, and judging whether the target has raised its hand based on the comparison result of the selected relative angle threshold and the relative angle between the arm and shoulder, includes:
[0114] When the wrist key point is higher than the head key point, a first relative angle threshold is selected;
[0115] When the relative angle between the arm and shoulder is greater than the first relative angle threshold, it is determined that the target has raised its hand.
[0116] In one implementation, the preset distance threshold is greater than or equal to the distance from the head key point to the neck key point.
[0117] In one embodiment, the hand-raising behavior detection device further includes:
[0118] The second acquisition module is used to acquire the head key points, neck key points, and elbow key points and wrist key points on the same side of the target being detected.
[0119] The second judgment module is used to examine the detected human body key points using a device that checks the detected human body key points. If it cannot conclude that the detected human body key points are correct, it will terminate the use of the key points for hand-raising behavior detection.
[0120] like Figure 4 As shown, the device 4 for checking and detecting key points of the human body includes: an acquisition module 410, a calculation module 420, a comparison module 430, and a judgment module 440.
[0121] The acquisition module 410 is used to acquire the head key points and neck key points of the target being detected; and to acquire the elbow key points and wrist key points on the same side of the target being detected.
[0122] The calculation module 420 is used to calculate the distance from the head key point to the neck key point, which is recorded as the head length; and to calculate the distance from the elbow key point to the wrist key point, which is recorded as the length of one arm.
[0123] The comparison module 430 is used to compare the head length with the length of the arm on one side based on a preset comparison model to obtain a relative scale estimate.
[0124] The judgment module 440 is used to determine that the detected human key points are correct when the relative scale estimate is within the range of a preset scale threshold.
[0125] In one embodiment, the preset comparison model of the device for examining detected key points on the human body is:
[0126] Where β is the relative scale estimate, and D H D is the head length. A The length is the arm length, and W and b are preset parameters.
[0127] In one embodiment, the judgment module of the device for checking the detected human body key points is further used to determine that the human body key points of the current detection target are unusable if the head length is not within a preset head length range; determine that the human body key points of the current detection target are unusable if the length of one arm is not within a preset arm length range; and determine that the detected human body key points are incorrect when it is determined that the human body key points of the current detection target are unusable.
[0128] In this device, for classroom teaching scenarios, different preset relative angle thresholds are selected based on the relative positions of key points on the head and wrist, and compared with the relative angles of the arm and shoulder. Based on the comparison results, it is determined whether a hand-raising behavior has occurred. Simultaneously, the relationship between head length and arm length is used to check the accuracy of the detected human body key points in a relatively simple way. Compared to existing judgment methods, this invention has lower computational requirements, higher accuracy, and wider applicability.
[0129] Example 3
[0130] This invention also provides a storage medium storing computer instructions that, when executed by a processor, implement the hand-raising behavior detection method of any of the above embodiments.
[0131] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, random access memory (RAM), read-only memory (ROM), magnetic disks, or optical disks.
[0132] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, terminal, or network device, etc.) to execute all or part of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, RAM, ROM, magnetic disks, or optical disks.
[0133] Corresponding to the computer storage medium described above, one embodiment also provides a computer device, which includes a memory, an encoder, and a computer program stored in the memory and executable on the encoder, wherein the encoder executes the program to implement any of the hand-raising behavior detection methods described in the above embodiments.
[0134] The aforementioned computer device, designed for classroom teaching scenarios, combines the relative positions of key points on the head and wrist with different preset relative angle thresholds and compares them with the relative angles of the arm and shoulder. Based on the comparison results, it determines whether a hand-raising behavior has occurred. Simultaneously, it further checks the accuracy of the detected human key points using a relatively simple method based on the relationship between head length and arm length. Compared to existing methods, this invention requires less computing power, has higher accuracy, and is applicable to a wider range of scenarios.
[0135] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0136] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.
Claims
1. A hand raising behavior detection method, characterized in that: include: Obtain the shoulder key points of both shoulders of the detection target; Calculate the relative position between the head key points and the wrist key points; Calculate the arm-shoulder relative angle between the arm and the shoulder, wherein the arm-shoulder relative angle is the angle formed by the intersection of the line between the wrist key point and the elbow key point and its extension line and the line between the shoulders and its extension line; According to the relative position between the key points of the head and the key points of the wrist, the corresponding relative angle threshold is selected, and according to the comparison result of the selected relative angle threshold and the relative angle of the arm and shoulder, it is judged whether the detection target raises the hand.
2. The hand raising behavior detection method according to claim 1, characterized in that: The process of selecting a corresponding relative angle threshold according to the relative position between the key point of the head and the key point of the wrist, and judging whether the detection target has raised the hand according to the comparison result of the selected relative angle threshold and the relative angle of the arm and shoulder, includes: When the wrist key point is lower than the head key point but higher than the neck key point, and the distance between the head key point and the wrist key point is greater than a preset distance threshold, selecting a first relative angle threshold; When the arm-shoulder relative angle is greater than a first relative angle threshold, it is determined that the detection target has raised his hand.
3. The hand raising behavior detection method according to claim 2, characterized in that: The process of selecting a corresponding relative angle threshold according to the relative position between the key point of the head and the key point of the wrist, and judging whether the detection target has raised the hand according to the comparison result of the selected relative angle threshold and the relative angle of the arm and shoulder, includes: When the wrist key point is lower than the neck key point and the distance between the head key point and the wrist key point is greater than a preset distance threshold, selecting a second relative angle threshold; When the arm-shoulder relative angle is greater than a second relative angle threshold, the detection target is judged to have raised the hand, wherein the second relative angle threshold is greater than or equal to the first relative angle threshold.
4. The hand raising behavior detection method according to claim 3, characterized in that: The process of selecting a corresponding relative angle threshold according to the relative position between the key point of the head and the key point of the wrist, and judging whether the detection target has raised the hand according to the comparison result of the selected relative angle threshold and the relative angle of the arm and shoulder, includes: When the wrist key point is higher than the head key point, selecting a first relative angle threshold; When the arm-shoulder relative angle is greater than a first relative angle threshold, it is determined that the detection target has raised his hand.
5. The hand raising behavior detection method according to any one of claims 1 to 4, characterized in that: The preset distance threshold is greater than or equal to the distance from the head key point to the neck key point.
6. The hand raising behavior detection method according to any one of claims 1 to 4, characterized in that: Also includes: Obtain the key points of the head and neck of the detected target, as well as the key points of the elbow and wrist on the same side; The detected human key points are checked using a method for checking the detected human key points. If a correct conclusion cannot be drawn about the detected human key points, the hand raising behavior detection using the key points is terminated; Wherein, the method for checking the detected key points of the human body includes: Obtain the key points of the head and neck of the detection target; Calculate the distance from the key point of the head to the key point of the neck, and record it as the head length; Obtain the elbow key points and wrist key points on the same side of the detection target; Calculate the distance from the elbow key point to the wrist key point, and record it as the length of one side of the arm; Based on a preset comparison model, the head length is compared with the arm length on one side to obtain a relative scale estimation value; When the relative scale estimation value is within a preset scale threshold range, it is determined that the detected human body key points are correct.
7. The hand raising behavior detection method according to claim 6, characterized in that: The preset comparison model in the method for checking the detected key points of the human body is: Among them, β is the relative scale estimate, D H is the head length, D A is the arm length, W and b are preset parameters.
8. The hand raising behavior detection method according to claim 7, characterized in that: The method for checking the detected key points of the human body also includes: If the head length is not within the preset head length range, it is determined that the human body key points of the current detection target are unavailable; If the arm length on one side is not within the preset arm length range, it is determined that the human body key point of the current detection target is unavailable; When it is determined that the human body key points of the current detection target are unavailable, it is determined that the detected human body key points are incorrect.
9. A hand raising behavior detection device, characterized in that: include: The first acquisition module is used to acquire the shoulder key points of both shoulders of the detection target; A calculation module, used for calculating the relative position between the key points of the head and the key points of the wrist; Calculate the arm-shoulder relative angle between the arm and the shoulder, wherein the arm-shoulder relative angle is the angle formed by the intersection of the line between the wrist key point and the elbow key point and its extension line and the line between the shoulders and its extension line; The first judgment module is used to select the corresponding relative angle threshold according to the relative position between the head key point and the wrist key point, and judge whether the detection target raises the hand according to the comparison result of the selected relative angle threshold and the arm-shoulder relative angle.
10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 8 is implemented.
11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.