Human body posture determination method and device, electronic device, and storage medium

Abstract

Embodiments of the present application relate to the technical field of image processing, and disclose a human body posture determination method and device, electronic equipment and a storage medium. The method comprises the following steps: determining a head region corresponding to a human head and a hand region corresponding to a human hand from a target image, the target image being an image comprising a human body; and if an overlap condition of the head region and the hand region meets a first condition, determining that the human body is in an illegal posture. By implementing the embodiments of the present application, it can be determined whether a user is in an illegal posture.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to a method and apparatus for determining human posture, electronic equipment, and storage medium. Background Technology

[0002] With the rapid development of electronic devices, a wide variety of functions have emerged, such as learning assistance, video viewing, and gaming entertainment. These functions often attract users to sit in front of electronic devices for extended periods of time.

[0003] In practice, it has been found that users inevitably adopt improper postures such as supporting their heads with their hands when using electronic devices for extended periods. Maintaining these improper postures for long periods can lead to problems such as vision loss and spinal misalignment. Therefore, identifying whether a user is in an improper posture has become an urgent problem to be solved. Summary of the Invention

[0004] This application discloses a method and apparatus for determining human posture, an electronic device, and a storage medium, which can determine whether a user is in an illegal posture.

[0005] The first aspect of this application discloses a method for determining human posture, comprising:

[0006] The head region corresponding to the human head and the hand region corresponding to the human hand are determined from the target image, wherein the target image is an image that includes the human body;

[0007] If the overlap between the head region and the hand region meets the first condition, then the human body is determined to be in an illegal posture.

[0008] The second aspect of this application discloses a device for determining human posture, comprising:

[0009] The first determining unit is used to determine the head region corresponding to the human head and the hand region corresponding to the human hand from the target image, wherein the target image is an image including the human body;

[0010] The second determining unit is used to determine that the human body is in an illegal posture if the overlap between the head region and the hand region meets the first condition.

[0011] The third aspect of this application discloses an electronic device, including:

[0012] Memory containing executable program code;

[0013] A processor coupled to the memory;

[0014] The processor calls the executable program code stored in the memory to execute the human posture determination method disclosed in the first aspect of the embodiments of this application.

[0015] The fourth aspect of this application discloses a computer-readable storage medium storing a computer program, wherein the computer program causes a computer to execute the method for determining human posture disclosed in the first aspect of this application.

[0016] Compared with related technologies, the embodiments of this application have the following beneficial effects:

[0017] In this embodiment, the head region corresponding to the human head and the hand region corresponding to the human hand can be determined from a target image including a human body. Further, if the overlap between the head region and the hand region in the target image satisfies a first condition, it indicates that the human head and hand may be in contact, i.e., a hand-supporting-head posture. Therefore, it can be determined that the human body in the target image is in an illegal posture, where an illegal posture is one that poses a danger to the human body. Thus, this embodiment solves the technical problem in the prior art of being unable to identify whether a user is in an illegal posture. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of a hand-supported head posture disclosed in an embodiment of this application;

[0020] Figure 2 This is a flowchart illustrating a method for determining human posture disclosed in an embodiment of this application;

[0021] Figure 3 This is a schematic diagram of a head region and a hand region disclosed in an embodiment of this application;

[0022] Figure 4 This is a schematic diagram of another hand-supported head disclosed in an embodiment of this application;

[0023] Figure 5 This is a flowchart illustrating another method for determining human posture disclosed in an embodiment of this application;

[0024] Figure 6 This is a schematic diagram disclosed in an embodiment of the present application for illustrating the misalignment of the hand and head;

[0025] Figure 7This is a flowchart illustrating another method for determining human posture disclosed in an embodiment of this application;

[0026] Figure 8 This is a schematic diagram of a key point in the head disclosed in an embodiment of this application;

[0027] Figure 9 This is a schematic diagram of a first sub-region disclosed in an embodiment of this application;

[0028] Figure 10 This is a schematic diagram of a key hand point disclosed in an embodiment of this application;

[0029] Figure 11 This is a flowchart illustrating another method for determining human posture disclosed in an embodiment of this application;

[0030] Figure 12 This is a schematic diagram of the structure of a human posture determination device disclosed in an embodiment of this application;

[0031] Figure 13 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application. Detailed Implementation

[0032] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0033] It should be noted that the terms "first," "second," "third," and "fourth," etc., used in the specification and claims of this application are used to distinguish different objects, not to describe a specific order. The terms "comprising" and "having," and any variations thereof, in the embodiments of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or devices.

[0034] This application discloses a method and apparatus for determining human posture, an electronic device, and a storage medium, which can determine whether a user is in an illegal posture.

[0035] The technical solution of this application will be described in detail below with reference to specific embodiments.

[0036] To more clearly illustrate the method, apparatus, electronic device, and storage medium for determining human posture disclosed in this application, the application scenarios applicable to this method are first introduced. Optionally, this method can be applied to various electronic devices with image processing capabilities, such as portable electronic devices like mobile phones and tablets, wearable devices like smartwatches and smart bracelets, or desktop devices like desktop computers and desktop TVs, without limitation.

[0037] To address the technical problem in related technologies that cannot identify whether a user is in an illegal posture, the human posture determination method provided in this application can identify a target image including a human body using an electronic device to determine the head region corresponding to the human head and the hand region corresponding to the human hand. Furthermore, if the overlap between the head region and the hand region in the target image satisfies a first condition, it indicates that the human head and hand may be in contact, i.e., a hand-supporting-head posture. Please refer to [link to relevant documentation]. Figure 1 , Figure 1 This is a schematic diagram of a hand-supported head posture disclosed in an embodiment of this application. In this posture, the hand 110 contacts and supports the head 120. It can be seen that the human spine is in a crooked state when in this posture, and the head may also be in a crooked state. Therefore, maintaining the hand-supported head posture for a long time will easily lead to problems such as vision decline and spinal misalignment.

[0038] In response, electronic devices can determine that the human body in the target image is in an illegal posture if the overlap between the head region and the hand region in the target image meets the first condition, thereby solving the technical problem in the prior art of not being able to identify whether the user is in an illegal posture.

[0039] In one optional embodiment, the method can be applied to learning-aid electronic devices, such as learning machines and tutoring machines, without limitation. Optionally, the learning-aid electronic device may have image acquisition and image processing functions, thereby enabling it to monitor whether the user is in an illegal posture during the learning process, and then remind the user to correct the user's posture, thereby protecting the user's physical health.

[0040] Based on this, the method for determining human posture disclosed in the embodiments of this application will be described below.

[0041] Please see Figure 2 , Figure 2 This is a flowchart illustrating a method for determining human posture disclosed in an embodiment of this application. Optionally, this method can be applied to the aforementioned electronic device with image processing capabilities, or other execution entities, and is not limited thereto. This application embodiment uses the aforementioned electronic device as an example for illustration and should not be construed as limiting the scope of this application embodiment.

[0042] Optionally, the method may include the following steps:

[0043] 202. Determine the head region corresponding to the human head and the hand region corresponding to the human hand from the target image.

[0044] In this embodiment, the electronic device may also have an image acquisition function. Optionally, the target image may be an image acquired by the electronic device through the image acquisition function. In other optional embodiments, the target image may also be an image obtained by the electronic device from other electronic devices or the Internet, which is not limited here. This embodiment uses the example of the target image being acquired by the electronic device for illustration, and should not be construed as limiting the embodiments of this application.

[0045] Optionally, the target image can be an image including a human body. Optionally, the target image can be any frame from multiple image frames included in the target video data, wherein the target video data can be video data recording the process of a human body using an electronic device while in front of it.

[0046] Please refer to further information. Figure 3 , Figure 3 This is a schematic diagram of a head region and a hand region disclosed in an embodiment of this application. Optionally, the electronic device can recognize the target image using an image recognition method to determine the head region 320 corresponding to the human head and the hand region 330 corresponding to the human hand based on the recognition result. Optionally, the image recognition method may include, but is not limited to, statistical image recognition methods, structural image recognition methods, and fuzzy image recognition methods.

[0047] Among them, such as Figure 3 As shown, after image recognition processing, the electronic device adds a first rectangular frame 340 to the head region 320 in the target image. This first rectangular frame 340 includes the head region 320, thus marking the head region. Similarly, a second rectangular frame 350 is added to the hand region 330, which includes the hand region 330. For ease of explanation, this embodiment uses the area included by the first rectangular frame 340 as the head region 320 and the area included by the second rectangular frame 350 as the hand region 330, and this should not be construed as limiting the scope of this embodiment.

[0048] It is understandable that the human body usually has two hands, so the hand region 330 can include two regions. However, if only one hand is captured at certain shooting angles, the resulting image can only identify one hand region 330. Figure 3 The target image in the example only includes a hand region 330 and should not be construed as limiting the embodiments of this application.

[0049] 204. If the overlap between the head area and the hand area meets the first condition, then the human body is determined to be in an illegal posture. An illegal posture is a posture that poses a danger to the human body.

[0050] In this embodiment of the application, the electronic device can determine whether the head region and the hand region overlap. If it is determined that the head region and the hand region are in an overlapping state, it can be determined that the overlap between the head region and the hand region satisfies the first condition.

[0051] Optionally, the electronic device can determine the overlap between the head region and the hand region by determining a target distance between them, and if the target distance is less than a distance threshold, determine that the overlap between the head region and the hand region satisfies the first condition. Optionally, the target distance may include Euclidean distance, which refers to the true distance between two points in M-dimensional space; the distance threshold can be set by developers based on extensive development experience and is not limited here. Wherein, when the target distance between the head region and the hand region is less than the distance threshold, it indicates that the distance between the head region and the hand region is quite close, thus determining that the overlap between the head region and the hand region satisfies the first condition.

[0052] Please refer to further information. Figure 4 , Figure 4 This is a schematic diagram of another hand-supported head posture disclosed in an embodiment of this application. It is understood that if the head region 410 and the hand region 420 are only in contact but not overlapping, the human body may still be in an illegal hand-supported head posture. Optionally, the electronic device can directly determine whether the human body is in an illegal posture based on the target distance between the head region 410 and the hand region 420. Alternatively, determining whether the human body is in an illegal posture based on distance can be used as a supplementary means to further determine whether the head region 410 and the hand region 420 are in contact when it is determined that they are not overlapping, thus avoiding misjudgment.

[0053] It should be further noted that illegal postures, in addition to the hand-supporting-head posture, can also include other postures that are harmful to the human body. This application uses the hand-supporting-head posture as an example for illustration, and should not be construed as limiting the scope of this application.

[0054] By implementing the methods disclosed in the above embodiments, the head region corresponding to the human head and the hand region corresponding to the human hand can be determined from a target image including a human body. Further, if the overlap between the head region and the hand region in the target image satisfies a first condition, it indicates that the human head and hand may be in contact, i.e., a hand-supporting-head posture. Therefore, it can be determined that the human body in the target image is in an illegal posture, where an illegal posture is a posture that poses a danger to the human body. It is evident that the embodiments of this application solve the technical problem in the prior art of being unable to identify whether a user is in an illegal posture.

[0055] Please see Figure 5 , Figure 5 This is a flowchart illustrating another method for determining human posture disclosed in this application. Optionally, this method can be applied to the aforementioned electronic device with image processing capabilities, or other execution entities, and is not limited thereto. This application uses the aforementioned electronic device as an example for illustration and should not be construed as limiting the scope of this application. Optionally, the method may include the following steps:

[0056] 502. Determine the head region corresponding to the human head and the hand region corresponding to the human hand from the target image.

[0057] Please see Figure 6 , Figure 6 This is a schematic diagram disclosed in an embodiment of the present application to illustrate the misalignment of the hand and head. In the diagram, the human head 610 and the human hand 620 are positioned one in front of the other. However, due to the misalignment, the human head 610 and the human hand 620 appear to be touching in the image captured by the camera 630. In reality, the human head 610 and the human hand 620 are not in contact, meaning there is no situation where the hand supports the head.

[0058] It should be further explained that, because the camera 630 is in Figure 6 In the target image obtained by shooting from the specified shooting position, the posture of the human head 610 is always abnormal, such as tilting the head back. Optionally, after determining the head region corresponding to the human head from the target image, the electronic device can further determine whether the target posture corresponding to the human head is in an abnormal state. If the target posture corresponding to the human head is in an abnormal state, the subsequent steps can be omitted to avoid misjudgment and save power consumption of the electronic device.

[0059] If the target posture corresponding to the human head is not in an abnormal state, it means that there is no misalignment between the human head and the human hand. In this case, the electronic device can determine the hand region corresponding to the human hand from the target image.

[0060] By implementing the above method, the electronic device can proceed with subsequent steps only after confirming that the human head is not in an abnormal state, so as to avoid misjudgment due to misalignment; in addition, for target images that may be misjudged, subsequent steps are not performed after the head region is determined, so as to save the power consumption of the electronic device.

[0061] As an optional implementation, the electronic device can determine the target angle corresponding to the Euler angle of the human head. If the target angle corresponding to the Euler angle is within the target threshold range, it can be determined that the target posture corresponding to the human head is not in an abnormal state; if the target angle corresponding to the Euler angle is not within the target threshold range, it can be determined that the target posture corresponding to the human head is in an abnormal state.

[0062] Euler angles are angular parameters used to determine the position of a rigid body rotating at a fixed point. In this embodiment, the posture of the human head can be determined using the Euler angles of the human head. Optionally, the Euler angles may include yaw angles, and the target angle may include a first angle corresponding to the yaw angle. If the absolute value of the first angle is greater than a yaw angle threshold, it is determined that the target angle corresponding to the Euler angle is not within the target threshold range, and the target posture of the human head is determined to be in an abnormal state. If the absolute value of the first angle is less than or equal to the yaw angle threshold, it is determined that the target angle corresponding to the Euler angle is within the target threshold range, and the target posture of the human head is determined to be not in an abnormal state.

[0063] The yaw angle threshold can be set by developers based on extensive development experience. Typical values ​​may include 25 degrees, 26 degrees, or 28 degrees, etc., and are not limited here.

[0064] Optionally, the Euler angle may include the pitch angle, and the target angle may include a second angle corresponding to the pitch angle; then, if the second angle is less than the pitch angle threshold, it is determined that the target angle corresponding to the Euler angle is not within the target threshold range, and the target attitude corresponding to the human head is determined to be in an abnormal state; if the second angle is greater than or equal to the yaw angle threshold, it is determined that the target angle corresponding to the Euler angle is within the target threshold range, and the target attitude corresponding to the human head is determined to be in an abnormal state.

[0065] The pitch angle threshold can be set by developers based on extensive development experience. Typical values ​​may include -15 degrees, -16 degrees, or -18 degrees, etc., and are not limited here.

[0066] Optionally, the electronic device can also determine whether the human head is in an abnormal state based on the first angle corresponding to the yaw angle of the human head and the second angle corresponding to the pitch angle, which is not limited here.

[0067] Optionally, Euler angles may also include roll angle. The method of determining whether the human head is in an abnormal state by using roll angle is similar to the method of determining whether the human head is in an abnormal state by using yaw angle and pitch angle, and will not be described again here.

[0068] By implementing the above method, electronic devices can determine whether a human head is in an abnormal state by measuring the Euler angles of the human head. Since the Euler angles of the human head are relatively easy to obtain, the implementation difficulty and cost of this method are reduced. In addition, the Euler angles of the human head can also reflect the posture of the human head well, thereby improving the accuracy of subsequent determination of whether the human head is in an abnormal state.

[0069] As an optional implementation, the electronic device can use a region recognition model to determine the head region corresponding to the human head and the hand region corresponding to the human hand from the target image. Optionally, the region recognition model can be trained from multiple sample images including human bodies and used to identify the head region and hand region in the image.

[0070] Optionally, the region identification model can be a model obtained through compression operations, which may include, but are not limited to, pruning operations, quantization operations, and distillation operations.

[0071] By implementing the above method, the region identification model can be compressed, thereby reducing the amount of data in the region identification model. This allows the region identification model to be configured in electronic devices and also saves storage space in the electronic devices.

[0072] 504. If the first ratio of the first region and the second region is greater than the first threshold, then the overlap between the head region and the hand region satisfies the first condition, and the human body is determined to be in an illegal posture. The first region includes the intersection of the hand region and the head region, and the second region includes the hand region, the head region, or the first union, which is the union of the hand region and the head region.

[0073] In this embodiment of the application, the electronic device can take the intersection of the hand region and the head region as the first region, take the hand region as the second region, and determine a first ratio between the first region and the second region; if the first ratio is greater than a first threshold, it can be determined that the overlap between the head region and the hand region satisfies the first condition.

[0074] Optionally, the electronic device can use the intersection of the hand region and the head region as the first region, and the head region as the second region, and determine a first ratio between the first region and the second region; then, if the first ratio is greater than a first threshold, it can be determined that the overlap between the head region and the hand region satisfies the first condition.

[0075] Optionally, the electronic device can take the intersection of the hand region and the head region as the first region, the union of the hand region and the head region as the second region, and determine a first ratio between the first region and the second region; then, if the first ratio is greater than a first threshold, it can be determined that the overlap between the head region and the hand region satisfies the first condition.

[0076] It should be noted that the denominators of the first ratios determined by the three methods mentioned above are different, and the first thresholds used for threshold judgment in these three methods are also different. The specific value of the first threshold can be set by the developers based on extensive development experience, and is not limited here.

[0077] By implementing the above method, the intersection of the hand and head regions—that is, the overlapping area of ​​the hand and head regions—can be used as the numerator, and the union of the hand, head, or hand and head regions can be used as the denominator to determine the first ratio. If the first ratio is greater than a first threshold, it indicates that the overlap between the hand and head regions is significant, confirming that the human hand and head are adjacent. Therefore, the overlap between the head and hand regions satisfies the first condition. Furthermore, electronic devices can use the hand, head, or hand and head regions as the denominator to determine the first ratio, which improves the flexibility and compatibility of the method.

[0078] As an optional implementation, the electronic device can take the intersection of the hand region and the head region as the first region and the hand region as the second region, and determine a first ratio between the first region and the second region; if the first ratio is greater than a first threshold, it can be determined that the overlap between the head region and the hand region satisfies a first condition.

[0079] Optionally, the first threshold may include 0.25, 0.26, or 0.3, etc. The specific value can be set by the developers based on extensive development experience, and is not limited here.

[0080] It should be noted that when a person is in a hand-supported-head posture, the hand usually obscures the head, which may lead to inaccurate determination of the head area. However, when the hand is in front of the head, there is no obstruction issue, and the determined hand area is usually more accurate. Therefore, the denominator can be set to the accurate hand area to improve the accuracy of the subsequently determined first ratio, thereby improving the accuracy of subsequent determinations of whether the person is in an illegal posture.

[0081] As an optional implementation, when the electronic device determines that a human body is in an illegal posture, it can output a prompt message to inform the user that they are currently in an illegal posture. Optionally, the prompt message may include voice prompts, text prompts, etc., and is not limited thereto.

[0082] By implementing the above method, when it is determined that the human body is in an illegal posture, a prompt message can be output to remind the user, so as to prompt the user to change his or her posture, thereby achieving the effect of protecting the user's physical health.

[0083] Optionally, after displaying the prompt message, if it is determined that the human body is still in an illegal posture, a corresponding restriction operation can be performed. Optionally, the restriction operation may include controlling the electronic device to switch to a locked screen state or controlling the electronic device to power off. Optionally, if it is determined that the human body is not in an illegal posture, the restriction operation can be lifted.

[0084] By implementing the above method, if the prompting information fails to prompt the user to change their posture, the user's continued use of the electronic device can be restricted by controlling the electronic device to switch to a locked screen state or controlling the electronic device to shut down, thereby prompting the user to change their posture and thus protecting the user's health.

[0085] As an optional implementation, if it is determined that the human body is in an illegal posture, the target duration of the user's illegal posture can be further monitored. If the target duration is greater than or equal to the duration threshold, a prompt message is output.

[0086] Optionally, the target duration can be set by the developers based on their development experience or by the users based on their usage needs. Typical values ​​may include 5 minutes, 10 minutes, etc., and are not limited here.

[0087] It's understandable that even briefly touching one's head could be considered an illegal posture, but since a brief illegal posture doesn't pose a threat to the user, the aforementioned method only outputs a warning message when the user is confirmed to be in an illegal posture for an extended period. This avoids the electronic device frequently outputting warning messages, which could disrupt normal use and improve the user experience.

[0088] The methods disclosed in the above embodiments can be implemented only after determining that the human head is not in an abnormal state, thus avoiding misjudgment due to misalignment. Furthermore, for target images where misjudgment is possible, subsequent steps are not performed after the head region is identified, saving power consumption of the electronic device. Additionally, the Euler angles of the human head can be used to determine whether the human head is in an abnormal state; since the Euler angles of the human head are relatively easy to obtain, the implementation difficulty and cost of this method are reduced. Moreover, the Euler angles of the human head can also reflect the posture of the human head well, thereby improving the accuracy of subsequent determinations of whether the human head is in an abnormal state. Furthermore, the region recognition model can be compressed to reduce the amount of data in the region recognition model, allowing it to be configured in the electronic device and saving storage space. Finally, if it is determined that the human body is in an illegal posture, a prompt message can be output to remind the user to change their posture, thereby protecting the user's health.

[0089] Please see Figure 7 , Figure 7 This is a flowchart illustrating another method for determining human posture disclosed in this application. Optionally, this method can be applied to the aforementioned electronic device with image processing capabilities, or other execution entities, and is not limited thereto. This application uses the aforementioned electronic device as an example for illustration and should not be construed as limiting the scope of this application. Optionally, the method may include the following steps:

[0090] 702. Determine the head region corresponding to the human head and the hand region corresponding to the human hand from the target image.

[0091] 704. If the overlap between the head region and the hand region satisfies the first condition, then determine the first sub-region corresponding to the head region and the second sub-region corresponding to the hand region.

[0092] like Figure 3 As shown, although the head region determined by the electronic device includes the human head, the head region is larger than the head; similarly, the hand region is also larger than the hand. Therefore, if the overlap between the head region and the hand region meets the first condition, it may only be that the areas of the head region other than the head overlap with the areas of the hand region other than the hand, but the head and hand do not overlap or touch, which means that a misjudgment may occur.

[0093] Optionally, the electronic device can further reduce the head region and hand region, that is, the electronic device can determine a first sub-region from the head region, wherein the first sub-region is located within the head region and the area of ​​the first sub-region is smaller than the area of ​​the head region; and determine a second sub-region from the hand region, wherein the second sub-region is located within the hand region and the area of ​​the second sub-region is smaller than the area of ​​the hand region.

[0094] Furthermore, subsequent electronic devices can determine that the human body is in an illegal posture only when the overlap between the first sub-region and the second sub-region meets the second condition, thereby improving the accuracy of determining whether the human body is in an illegal posture.

[0095] By implementing the above method, the electronic device can determine a first sub-region that better represents the true position of the human head and a second sub-region that better represents the true position of the human hand. The device only determines that the human body is in an illegal posture when the overlap between the first and second sub-regions meets the second condition, thereby improving the accuracy of determining whether the human body is in an illegal posture.

[0096] As an alternative implementation, the electronic device can identify multiple head key points in the head region, and then determine a first sub-region corresponding to the head region based on the multiple head key points.

[0097] Please see Figure 8 , Figure 8 This is a schematic diagram of a head key point disclosed in an embodiment of this application. There may be multiple head key points 800. Optionally, the head key points 800 may be located on the head outline, mouth outline, nose outline, and eye outline. Optionally, the number of head key points 800 may be 46, 48, or 50. The specific number can be set by the developers based on extensive development experience and is not limited here.

[0098] Optionally, head keypoints can be added manually by developers or by electronic devices using a head keypoint detection model; this is not limited to either method. The head keypoint detection model can be trained from multiple sample images including the head and its keypoints, and is used to determine the head keypoints in the image.

[0099] By implementing the above method, since the head key points are all located within the head area, the head key points can more accurately reflect the true position of the head within the head area. Therefore, the first sub-region determined based on the head key points can more accurately reflect the true position of the head.

[0100] Optionally, the electronic device can determine a first sub-region corresponding to the head region based on a first target key point included among multiple head key points, wherein the first sub-region is the smallest region containing the first target key point.

[0101] Optionally, the first target key point may include key points of facial features located on the facial feature contour. Optionally, the key points of facial features may include key points of the mouth corresponding to the mouth contour, key points of the nose corresponding to the nose contour, or key points of the eyes corresponding to the eye contour.

[0102] Please refer to it again. Figure 8 In this process, the electronic device can use the key points of the mouth located on the mouth contour as the first target key points, and then determine the mouth sub-region 810 as the first sub-region corresponding to the head region based on the first target key points.

[0103] Optionally, the electronic device can also use the key points of the nose located on the nose contour as the first target key points, and then determine the nose sub-region 820 as the first sub-region corresponding to the head region based on the first target key points.

[0104] Optionally, the electronic device can also use the eye key points located on the eye contour as the first target key points, and then determine the eye sub-region 830 as the first sub-region corresponding to the head region based on the first target key points.

[0105] By implementing the above method, the electronic device can determine the first sub-region that can represent the position of the head by some key points of the head. For example, the mouth sub-region can be determined as the first sub-region corresponding to the head region. Since the mouth sub-region is located within the head area, it is more accurate to determine whether the hand and the head overlap based on the mouth sub-region, and it can prevent misjudgment.

[0106] Optionally, the electronic device can determine, among the first target key points including multiple head key points, the first key point with the smallest ordinate, the second key point with the largest ordinate, the third key point with the smallest abscissa, and the fourth key point with the largest abscissa; and then determine the first sub-region corresponding to the head region based on the first key point, the second key point, the third key point, and the fourth key point.

[0107] Optionally, the electronic device can generate a first sub-region of a rectangle, wherein a first keypoint is located on the first side of the first sub-region of the rectangle, a second keypoint is located on the second side of the first sub-region of the rectangle, a third keypoint is located on the third side of the first sub-region of the rectangle, and a fourth keypoint is located on the fourth side of the first sub-region of the rectangle.

[0108] Please see Figure 9 , Figure 9This is a schematic diagram of a first sub-region disclosed in an embodiment of this application. The electronic device can identify a first key point 910 with the smallest ordinate, a second key point 920 with the largest ordinate, a third key point 930 with the smallest abscissa, and a fourth key point 940 with the largest abscissa among the key points of the mouth. Then, based on the first key point 910, the second key point 920, the third key point 930, and the fourth key point 940, it can determine the first sub-region 950 corresponding to the head region.

[0109] By implementing the above method, the electronic device can determine the smallest region containing the first target key point through the four extreme key points: the first key point with the smallest ordinate, the second key point with the largest ordinate, the third key point with the smallest abscissa, and the fourth key point with the largest abscissa. This method is simple to implement and has a high accuracy rate.

[0110] As an alternative implementation, the electronic device can identify multiple key hand points in the hand region, and then determine a second sub-region corresponding to the hand region based on the multiple key hand points.

[0111] Please see Figure 10 , Figure 10 This is a schematic diagram of key hand points disclosed in an embodiment of this application. There can be multiple key hand points 1000. Optionally, the key hand points 1000 can be located on the knuckles of the fingers, at the base of the fingers, or at the base of the palm. Optionally, the number of key hand points 1000 can be 21, 22, or 25. The specific number can be set by the developers based on extensive development experience and is not limited here.

[0112] Optionally, hand keypoints can be added manually by developers or by electronic devices using a hand keypoint detection model; this is not limited to either method. The hand keypoint detection model can be trained from multiple sample images including the hand and its keypoints, and is used to determine the hand keypoints in the image.

[0113] By implementing the above method, since the key points of the hand are all located within the hand area, the key points of the hand can more accurately reflect the true position of the hand within the hand area. Therefore, the second sub-region determined based on the key points of the hand can more accurately reflect the true position of the hand.

[0114] Optionally, the electronic device can determine a second sub-region corresponding to the hand region based on the second target key point included among multiple hand key points, wherein the second sub-region is the smallest region containing the second target key point.

[0115] Optionally, the second target key point may include a finger key point located on a finger joint; optionally, the fingers may include the index finger, middle finger, ring finger, and little finger.

[0116] Please refer to it again. Figure 10 Optionally, the electronic device can determine the fifth key point 1010 with the smallest ordinate, the sixth key point 1020 with the largest ordinate, the seventh key point 1030 with the smallest abscissa, and the eighth key point 1040 with the largest abscissa from among the second target key points including multiple hand key points; then the electronic device can determine the second sub-region 1050 corresponding to the hand region based on the fifth key point 1010, the sixth key point 1020, the seventh key point 1030, and the eighth key point 1040.

[0117] By implementing the above method, the electronic device can determine the smallest region containing the second target key point through the four extreme key points: the fifth key point with the smallest ordinate, the sixth key point with the largest ordinate, the seventh key point with the smallest abscissa, and the eighth key point with the largest abscissa. This method is simple to implement and has a high accuracy rate.

[0118] Optionally, the head keypoint detection model and the hand keypoint detection model mentioned above can be models obtained through compression operations. Optionally, compression operations can include pruning operations, quantization operations, and distillation operations.

[0119] By implementing the above method, the amount of data in the head keypoint detection model and the hand keypoint detection model can be reduced, thereby enabling the head keypoint detection model and the hand keypoint detection model to be configured in electronic devices, and also saving storage space in electronic devices.

[0120] 706. If the overlap between the first sub-region and the second sub-region satisfies the second condition, then the human body is determined to be in an illegal posture.

[0121] As an optional implementation, the electronic device can determine that the overlap between the first sub-region and the second sub-region satisfies the second condition when the second ratio between the third region and the fourth region is greater than the second threshold, and determine that the human body is in an illegal posture. The third region includes the intersection of the first sub-region and the second sub-region, and the fourth region includes the first sub-region, the second sub-region, or the second union, where the second union is the union of the first sub-region and the second sub-region.

[0122] It should be noted that the denominators for determining the second ratio are different in the three methods mentioned above, and the second threshold used for threshold judgment is also different in these three methods. The specific value of the second threshold can be set by the developers based on extensive development experience, and is not limited here.

[0123] By implementing the above method, the intersection of the first and second sub-regions (i.e., the overlapping area of ​​the first and second sub-regions) can be used as the numerator, and the first sub-region, the second sub-region, or the union of the first and second sub-regions can be used as the denominator to determine the second ratio. If the second ratio is greater than a second threshold, it indicates a large overlap between the first and second sub-regions, thus confirming that the human hand and head are close together. Furthermore, electronic devices can use the first sub-region, the second sub-region, or the union of the first and second sub-regions as the denominator to determine the second ratio, further improving the flexibility and compatibility of the method.

[0124] As an optional implementation, the electronic device can take the intersection of the first sub-region and the second sub-region as the third region and the second sub-region as the fourth region, and determine a second ratio between the third region and the fourth region; if the second ratio is greater than a second threshold, it can be determined that the overlap between the first sub-region and the second sub-region satisfies the second condition.

[0125] It should be noted that when a person is in a hand-supported-head posture, the hand usually obscures the head, which may lead to inaccuracy in the first sub-region. However, since the hand is in front of the head, there is no obstruction issue, and the second sub-region determined in this case is usually more accurate. Therefore, setting the denominator to the accurate second sub-region can improve the accuracy of the subsequently determined second ratio, thereby improving the accuracy of determining whether the person is in an illegal posture.

[0126] As an optional implementation, the first sub-region may include an eye sub-region corresponding to the eye key point, a nose sub-region corresponding to the nose key point, and a mouth sub-region corresponding to the mouth key point; then, if the overlap between the mouth sub-region and the second sub-region meets the second condition, the electronic device determines that the human body is in an illegal state; if the overlap between the mouth sub-region and the second sub-region does not meet the second condition, it determines whether the overlap between the nose sub-region and the second sub-region meets the second condition.

[0127] Furthermore, if the overlap between the nose sub-region and the second sub-region satisfies the second condition, then the human body is determined to be in an illegal state; if the overlap between the nose sub-region and the second sub-region does not satisfy the second condition, then it is determined whether the overlap between the eye sub-region and the second sub-region satisfies the second condition.

[0128] Furthermore, if the overlap between the eye sub-region and the second sub-region satisfies the second condition, then the human body is determined to be in an illegal state.

[0129] It's understandable that when a person is in a hand-supporting-head position, their hand is generally in contact with their chin, and sometimes even their mouth. Electronic devices could first determine if the mouth, which is closest to the chin, is in contact with the hand, and so on, then determine if the nose and eyes are in contact with the hand. Implementing this method could potentially eliminate the need to separately determine whether the mouth, nose, and eyes are in contact with the hand, thus reducing the computational load and power consumption of the electronic device.

[0130] As an optional implementation, the first sub-region may include an eye sub-region corresponding to an eye key point, a nose sub-region corresponding to a nose key point, and a mouth sub-region corresponding to a mouth key point; then, if the electronic device determines that at least one of the eye sub-region, nose sub-region, and mouth sub-region overlaps with the second sub-region to satisfy a second condition, it determines that the human body is in an illegal state.

[0131] By implementing the above method, the electronic device can determine that the human body is in an illegal state if at least one of the eye sub-region, nose sub-region, and mouth sub-region overlaps with the second sub-region to satisfy the second condition. This method saves more time and improves efficiency compared to judging the second sub-region one by one by the eye sub-region, nose sub-region, and mouth sub-region.

[0132] The methods disclosed in the above embodiments can determine a first sub-region representing the head position using some head key points. For example, the mouth sub-region can be determined as the first sub-region corresponding to the head region. Since the mouth sub-region is located within the head area, subsequent determination of whether the hand and head overlap based on the mouth sub-region is more accurate and can prevent misjudgment. Furthermore, the smallest region containing the second target key points can be determined using the four extreme key points among the second target key points: the fifth key point with the smallest ordinate, the sixth key point with the largest ordinate, the seventh key point with the smallest abscissa, and the eighth key point with the largest abscissa. This method is simple to implement and has high accuracy. Furthermore, the second ratio can be determined by using the first sub-region, the second sub-region, or the union of the first and second sub-regions as the denominator, which improves the flexibility and compatibility of the method. Also, it may not be necessary to separately determine whether the mouth, nose, and eyes are in contact with the hand, thereby reducing the computational load and power consumption of electronic devices. Moreover, if at least one of the eye sub-region, nose sub-region, and mouth sub-region overlaps with the second sub-region and satisfies the second condition, it can be determined that the human body is in an illegal state. This saves more judgment time and improves the efficiency of the method compared to judging the eye sub-region, nose sub-region, and mouth sub-region with the second sub-region one by one.

[0133] Please see Figure 11 , Figure 11This is a flowchart illustrating another method for determining human posture disclosed in this application. Optionally, this method can be applied to the aforementioned electronic device with image processing capabilities, or other execution entities, and is not limited thereto. This application uses the aforementioned electronic device as an example for illustration and should not be construed as limiting the scope of this application. Optionally, the method may include the following steps:

[0134] 1102. Determine the head region corresponding to the human head and the hand region corresponding to the human hand from the target image.

[0135] 1104. If the overlap between the head region and the hand region satisfies the first condition, then determine the first sub-region corresponding to the head region and the second sub-region corresponding to the hand region.

[0136] In this embodiment, the method for determining the first sub-region corresponding to the head region and the second sub-region corresponding to the hand region can be similar to the determination method disclosed in the above embodiments, and will not be repeated here.

[0137] 1106. If the overlap between the first sub-region and the second sub-region does not meet the third condition, then the human body is determined to be in an illegal posture.

[0138] Please refer to it again. Figure 1 When a person is in an unnatural head-supporting posture, their hands are usually in contact with their chin in order to support their head.

[0139] In this embodiment, the first sub-region may include an eye sub-region corresponding to the eye key point, a nose sub-region corresponding to the nose key point, and a mouth sub-region corresponding to the mouth key point. If the overlap between the first and second sub-regions satisfies a third condition, it indicates that at least one of the eye, nose, or mouth sub-regions is in contact with the second sub-region corresponding to the hand. That is, the hand is not in contact with the chin of the head, thus confirming that the human body is not in an illegal hand-supporting-the-head posture, but rather in a legal posture. For example, a user touching their nose or mouth can confirm that the human body is in a legal posture.

[0140] Conversely, if the overlap between the first and second sub-regions does not satisfy the third condition, i.e., the hand is not in contact with the eyes, nose, and mouth, then the hand is most likely in contact with the chin of the head, which can confirm that the human body is in an illegal posture.

[0141] In an optional embodiment, the electronic device may determine that the overlap between the first sub-region and the second sub-region does not meet the third condition if the second ratio between the third region and the fourth region is less than or equal to the third threshold, and determine that the human body is in an illegal posture. The third region includes the intersection of the first sub-region and the second sub-region, and the fourth region includes the first sub-region, the second sub-region, or the second union, wherein the second union is the union of the first sub-region and the second sub-region.

[0142] Optionally, the third region can be the intersection of the first and second sub-regions, and the fourth region can be the second sub-region.

[0143] Optionally, if the electronic device determines that the second ratio between the third region and the fourth region is greater than the third threshold, it determines that the overlap between the first sub-region and the second sub-region satisfies the third condition, and determines that the human body is in a legal posture.

[0144] In another optional embodiment, if the electronic device determines that the overlap between the mouth sub-region and the second sub-region does not meet the third condition, it can determine whether the overlap between the nose sub-region and the second sub-region meets the third condition; and if the overlap between the nose sub-region and the second sub-region does not meet the third condition, it can further determine whether the overlap between the eye sub-region and the second sub-region meets the third condition; if the overlap between the eye sub-region and the second sub-region does not meet the third condition, it is determined that the human body is in an illegal posture.

[0145] Optionally, if the electronic device determines that at least one of the eye sub-region, nose sub-region, and mouth sub-region overlaps with the second sub-region to satisfy the third condition, it can stop the judgment step and determine that the human body is in a legal state.

[0146] For example, if an electronic device determines that the overlap between the nose sub-region and the second sub-region meets the third condition, it can skip the subsequent step of "determining whether the overlap between the eye sub-region and the second sub-region meets the third condition" and directly determine that the human body is in a legal state. This reduces the computational load and power consumption of the electronic device.

[0147] It should be further clarified that the above-mentioned legal postures are those that, other than illegal postures, do not pose a threat to the body. For example, touching one's nose or sitting upright are both legal postures, and no specific restrictions are imposed here.

[0148] Optionally, if the electronic device determines that the human body is in a legal posture, it may not output any prompts, thereby avoiding the frequent output of prompts that would affect the user's normal use of the electronic device.

[0149] By implementing the method disclosed in the above embodiments, it is possible to determine that the human body is not in an illegal posture of supporting the head with the hand when the human hand is in contact with the eyes, nose or mouth. This can avoid misjudgment by electronic devices and thus avoid frequent output of prompt information by electronic devices. This reduces the power consumption of electronic devices and improves the user experience.

[0150] Please see Figure 12 , Figure 12 This is a schematic diagram of a human posture determination device disclosed in an embodiment of this application. Optionally, this device can be applied to the aforementioned electronic device with image processing function, or other execution entities, and is not limited thereto. This application embodiment uses the aforementioned electronic device as an example for description and should not be construed as limiting the scope of this application embodiment. Optionally, the device may include a first determining unit 1202 and a second determining unit 1204, wherein:

[0151] The first determining unit 1202 is used to determine the head region corresponding to the human head and the hand region corresponding to the human hand from the target image, wherein the target image is an image including the human body.

[0152] The second determining unit 1204 is used to determine that the human body is in an illegal posture if the overlap between the head region and the hand region meets the first condition. An illegal posture is a posture that causes harm to the human body.

[0153] By implementing the above-described device, the head region corresponding to the human head and the hand region corresponding to the human hand can be determined from a target image including a human body. Furthermore, if the overlap between the head region and the hand region in the target image satisfies a first condition, it indicates that the human head and hand may be in contact, i.e., a hand-supporting-head posture. Therefore, it can be determined that the human body in the target image is in an illegal posture, where an illegal posture is one that poses a danger to the human body. Thus, the embodiments of this application solve the technical problem in the prior art of being unable to identify whether a user is in an illegal posture.

[0154] As an optional implementation, the second determining unit 1204 is further configured to determine that the overlap between the head region and the hand region satisfies a first condition when the first ratio between the first region and the second region is greater than a first threshold, and to determine that the human body is in an illegal posture. The first region includes the intersection of the hand region and the head region, and the second region includes the hand region, the head region, or a first union, wherein the first union is the union of the hand region and the head region.

[0155] By implementing the above-described device, the first ratio can be determined by using the hand region, the head region, or the union of the hand region and the head region as the denominator, which also improves the flexibility and compatibility of the method.

[0156] As an alternative implementation, the first region is the intersection of the hand region and the head region, and the second region is the hand region.

[0157] When implementing the above device, since the hands usually cover the head when the human body is in a hand-supported-head posture, the determined head area may be inaccurate. However, since the hands are in front of the head, there is no problem of obstruction, and the determined hand area is usually more accurate. Therefore, the denominator can be set to the accurate hand area to improve the accuracy of the subsequently determined first ratio, thereby improving the accuracy of subsequent determination of whether the human body is in an illegal posture.

[0158] As an optional implementation method, Figure 12 The apparatus shown may further include a third determining unit and a fourth determining unit (not shown), wherein:

[0159] The third determining unit is used to determine the first sub-region corresponding to the head region after the overlap between the head region and the hand region meets the first condition. The first sub-region is located within the head region and the area of ​​the first sub-region is smaller than the area of ​​the head region.

[0160] The fourth determining unit is used to determine the second sub-region corresponding to the hand region, wherein the second sub-region is located within the hand region and the area of ​​the second sub-region is smaller than the area of ​​the hand region;

[0161] Furthermore, the second determining unit 1204 is also used to determine that the human body is in an illegal posture when the overlap between the first sub-region and the second sub-region satisfies the second condition.

[0162] By implementing the above device, the electronic device can determine a first sub-region that better represents the true position of the human head and a second sub-region that better represents the true position of the human hand. The human body is determined to be in an illegal posture only when the overlap between the first and second sub-regions meets the second condition, thereby improving the accuracy of determining whether the human body is in an illegal posture.

[0163] As an optional implementation method, Figure 12 The apparatus shown may further include a fifth determining unit (not shown), wherein:

[0164] The fifth determining unit is used to determine multiple head key points in the head region before determining the first sub-region corresponding to the head region;

[0165] Furthermore, the third determining unit is also used to determine the first sub-region corresponding to the head region based on multiple head key points.

[0166] By implementing the above device, since the key points of the head are all located within the head area, the key points of the head can more accurately reflect the true position of the head within the head area. Therefore, the first sub-region determined based on the key points of the head can more accurately reflect the true position of the head.

[0167] As an optional implementation, the third determining unit is further configured to determine a first sub-region corresponding to the head region based on the first target key point included among the multiple head key points, wherein the first sub-region is the smallest region containing the first target key point.

[0168] By implementing the above device, a first sub-region that can represent the position of the head can be determined by some key points of the head. For example, the mouth sub-region can be determined as the first sub-region corresponding to the head region. Since the mouth sub-region is located within the head region, it is more accurate to determine whether the hand and the head overlap based on the mouth sub-region, and it can prevent misjudgment.

[0169] As an optional implementation, the third determining unit is further configured to determine, among the first target key points including the plurality of head key points, the first key point with the smallest ordinate, the second key point with the largest ordinate, the third key point with the smallest abscissa, and the fourth key point with the largest abscissa; and to determine the first sub-region corresponding to the head region based on the first key point, the second key point, the third key point, and the fourth key point.

[0170] By implementing the above device, the smallest region containing the first target key point can be determined by the four extreme key points: the first key point with the smallest ordinate, the second key point with the largest ordinate, the third key point with the smallest abscissa, and the fourth key point with the largest abscissa. This method is simple to implement and has a high accuracy rate.

[0171] As an optional implementation, the first target key point includes key points of facial features located on the facial feature contour, including key points of the mouth corresponding to the mouth contour, key points of the nose corresponding to the nose contour, or key points of the eyes corresponding to the eye contour.

[0172] As an optional implementation method, Figure 12 The apparatus shown may further include a sixth determining unit (not shown), wherein:

[0173] The sixth determining unit is used to determine multiple key hand points in the hand region before determining the second sub-region corresponding to the hand region;

[0174] Furthermore, the fourth determining unit is also used to determine the second sub-region corresponding to the hand region based on multiple hand key points.

[0175] By implementing the above device, since the key points of the hand are all located within the hand area, the key points of the hand can more accurately reflect the true position of the hand within the hand area. Therefore, the second sub-region determined based on the key points of the hand can more accurately reflect the true position of the hand.

[0176] As an optional implementation, the fourth determining unit is further configured to determine a second sub-region corresponding to the hand region based on the second target key point included among the multiple hand key points, wherein the second sub-region is the smallest region containing the second target key point.

[0177] By implementing the above device, a second sub-region that is more representative of the hand position can be determined by some key points of the hand. This makes it more accurate to determine whether the hand and head overlap based on the second sub-region and can prevent misjudgment.

[0178] As an optional implementation, the fourth determining unit is further configured to determine, among the second target key points including multiple hand key points, a fifth key point with the smallest ordinate, a sixth key point with the largest ordinate, a seventh key point with the smallest abscissa, and an eighth key point with the largest abscissa; and to determine a second sub-region corresponding to the hand region based on the fifth key point, the sixth key point, the seventh key point, and the eighth key point.

[0179] By implementing the above device, the smallest region containing the second target key point can be determined by the four extreme key points: the fifth key point with the smallest ordinate, the sixth key point with the largest ordinate, the seventh key point with the smallest abscissa, and the eighth key point with the largest abscissa. This method is simple to implement and has a high accuracy rate.

[0180] As an alternative implementation, the second target key point includes finger key points located on the finger joints, the fingers including the index finger, middle finger, ring finger, and little finger.

[0181] As an optional implementation, the second determining unit 1204 is further configured to determine that the overlap between the first sub-region and the second sub-region satisfies the second condition if the second ratio between the third region and the fourth region is greater than the second threshold, and to determine that the human body is in an illegal posture. The third region includes the intersection of the first sub-region and the second sub-region, and the fourth region includes the first sub-region, the second sub-region, or the second union, wherein the second union is the union of the first sub-region and the second sub-region.

[0182] By implementing the above-described apparatus, the second ratio can be determined by using the first sub-region, the second sub-region, or the union of the first and second sub-regions as the denominator, which also improves the flexibility and compatibility of the method.

[0183] As an optional implementation, the third region is the intersection of the first sub-region and the second sub-region, and the fourth region is the second sub-region.

[0184] When implementing the above device, since the hands usually cover the head when the human body is in a hand-supported-head posture, the first sub-region may be inaccurate. However, since the hands are in front of the head, there is no problem of obstruction, and the second sub-region determined in this way is usually more accurate. Therefore, setting the denominator to the accurate second sub-region can improve the accuracy of the subsequently determined second ratio, thereby improving the accuracy of the subsequent determination of whether the human body is in an illegal posture.

[0185] As an optional implementation, the first sub-region includes an eye sub-region corresponding to the eye key point, a nose sub-region corresponding to the nose key point, and a mouth sub-region corresponding to the mouth key point; the second determining unit 1204 is further configured to determine that the human body is in an illegal state if the overlap between the mouth sub-region and the second sub-region meets the second condition, and if the overlap between the mouth sub-region and the second sub-region does not meet the second condition, then determine whether the overlap between the nose sub-region and the second sub-region meets the second condition.

[0186] Furthermore, if the overlap between the nose sub-region and the second sub-region satisfies the second condition, then the human body is determined to be in an illegal state; if the overlap between the nose sub-region and the second sub-region does not satisfy the second condition, then it is determined whether the overlap between the eye sub-region and the second sub-region satisfies the second condition.

[0187] Furthermore, if the overlap between the eye sub-region and the second sub-region satisfies the second condition, then the human body is determined to be in an illegal state.

[0188] Implementing the above device may eliminate the need to separately determine whether the mouth, nose, and eyes are in contact with the hand, thereby reducing the computational load and power consumption of electronic devices.

[0189] As an optional implementation, the first sub-region includes an eye sub-region corresponding to an eye key point, a nose sub-region corresponding to a nose key point, and a mouth sub-region corresponding to a mouth key point; the second determining unit 1204 is further configured to determine that the human body is in an illegal state if at least one of the eye sub-region, nose sub-region, and mouth sub-region overlaps with the second sub-region and satisfies a second condition.

[0190] By implementing the above-described device, if at least one of the eye sub-region, nose sub-region, and mouth sub-region overlaps with the second sub-region to satisfy the second condition, it can be determined that the human body is in an illegal state. Compared with judging the second sub-region one by one using the eye sub-region, nose sub-region, and mouth sub-region, this method can save more judgment time and improve efficiency.

[0191] As an optional implementation, the first determining unit 1202 is further configured to determine the head region corresponding to the human head from the target image, and determine whether the target posture corresponding to the human head is in an abnormal state; and if the target posture corresponding to the human head is not in an abnormal state, then determine the hand region corresponding to the human hand from the target image.

[0192] By implementing the above device, subsequent steps can be performed only after confirming that the human head is not in an abnormal state, so as to avoid misjudgment due to misalignment; in addition, for target images that may be misjudged, subsequent steps are not performed after the head region is determined, so as to save power consumption of electronic devices.

[0193] As an optional implementation, the first determining unit 1202 is further configured to determine the target angle corresponding to the Euler angle of the human head; and if the target angle is within the target threshold range, determine that the target posture corresponding to the human head is not in an abnormal state; and if the target angle is not within the target threshold range, determine that the target posture corresponding to the human head is in an abnormal state.

[0194] By implementing the above device, it is possible to determine whether the human head is in an abnormal state by measuring the Euler angles of the human head. Since the Euler angles of the human head are relatively easy to obtain, the implementation difficulty and cost of this method are reduced. In addition, the Euler angles of the human head can also reflect the posture of the human head well, thereby improving the accuracy of subsequent determination of whether the human head is in an abnormal state.

[0195] As an optional implementation, Euler angles include yaw angles, the target angle includes a first angle corresponding to the yaw angle, and / or Euler angles include pitch angles, the target angle includes a second angle corresponding to the pitch angle.

[0196] As an optional implementation, the first determining unit 1202 is further configured to determine the head region corresponding to the human head and the hand region corresponding to the human hand from the target image through a region recognition model. The region recognition model is a model obtained through compression operations, including pruning operations, quantization operations, and distillation operations.

[0197] By implementing the above-mentioned device, the region identification model can be compressed, thereby reducing the amount of data in the region identification model. This allows the region identification model to be configured into electronic devices, while also saving storage space in the electronic devices.

[0198] As an optional implementation method, Figure 12 The illustrated device may also include an output unit (not shown), wherein:

[0199] The output unit is used to output a prompt message after determining that the human body is in an illegal posture. The prompt message is used to inform the user that the current posture is illegal.

[0200] By implementing the above device, when it is determined that the human body is in an illegal posture, a prompt message can be output to remind the user, so as to prompt the user to change his or her posture, thereby achieving the effect of protecting the user's physical health.

[0201] As an optional implementation, the second determining unit 1204 is further configured to determine that the human body is in an illegal posture if the overlap between the first sub-region and the second sub-region does not meet the third condition.

[0202] Optionally, the second determining unit 1204 is further configured to determine that the human body is in an illegal posture if the second ratio between the third region and the fourth region is less than or equal to the third threshold, and the overlap between the first sub-region and the second sub-region does not meet the third condition. The third region includes the intersection of the first sub-region and the second sub-region, and the fourth region includes the first sub-region, the second sub-region, or the second union, wherein the second union is the union of the first sub-region and the second sub-region.

[0203] Optionally, the second determining unit 1204 is further configured to determine whether the overlap between the nose sub-region and the second sub-region satisfies the third condition if the overlap between the mouth sub-region and the second sub-region does not satisfy the third condition; and if the overlap between the nose sub-region and the second sub-region does not satisfy the third condition, determine whether the overlap between the eye sub-region and the second sub-region satisfies the third condition; and if the overlap between the eye sub-region and the second sub-region does not satisfy the third condition, determine that the human body is in an illegal posture.

[0204] Optional, Figure 12 The apparatus shown may further include a seventh determining unit (not shown), wherein:

[0205] The seventh determining unit is used to determine that the human body is in a legal posture when the overlap between the first sub-region and the second sub-region satisfies the third condition.

[0206] Optionally, the seventh determining unit is also used to determine that the human body is in a legal state if the overlap between at least one of the eye sub-region, nose sub-region, and mouth sub-region and the second sub-region satisfies the third condition.

[0207] Please see Figure 13 , Figure 13 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application. For example... Figure 13 As shown, the electronic device may include:

[0208] Memory 1301 storing executable program code;

[0209] Processor 1302 coupled to memory 1301;

[0210] The processor 1302 calls the executable program code stored in the memory 1301 to execute the human posture determination method disclosed in the above embodiments.

[0211] This application discloses a computer-readable storage medium storing a computer program that causes a computer to execute the human posture determination method disclosed in the above embodiments.

[0212] This application also discloses an application publishing platform, which is used to publish computer program products. When the computer program products are run on a computer, the computer performs some or all of the steps of the methods described in the above method embodiments.

[0213] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Those skilled in the art should also recognize that the embodiments described in the specification are optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0214] In the various embodiments of this application, it should be understood that the sequence number of each process does not necessarily imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0215] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they can be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0216] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0217] If the aforementioned integrated units are implemented as software functional units and sold or used as independent products, they can be stored in a computer-accessible memory. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several requests to cause a computer device (which can be a personal computer, server, or network device, specifically a processor in the computer device) to execute some or all of the steps of the methods described in the various embodiments of this application.

[0218] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0219] The foregoing has provided a detailed description of a method and apparatus for determining human posture, an electronic device, and a storage medium disclosed in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for determining human posture, characterized in that, The method includes: The head region corresponding to the human head and the hand region corresponding to the human hand are determined from the target image, wherein the target image is an image that includes the human body; If the first ratio of the first region and the second region is greater than a first threshold, then the overlap between the head region and the hand region satisfies a first condition, and a first sub-region corresponding to the head region is determined, wherein the first sub-region is located within the head region and the area of ​​the first sub-region is smaller than the area of ​​the head region; a second sub-region corresponding to the hand region is determined, wherein the second sub-region is located within the hand region and the area of ​​the second sub-region is smaller than the area of ​​the hand region; if the overlap between the first sub-region and the second sub-region satisfies a second condition, then the human body is determined to be in an illegal posture, wherein the first region includes the intersection of the hand region and the head region, and the second region includes the hand region, the head region, or a first union, wherein the first union is the union of the hand region and the head region; And, before determining the second sub-region corresponding to the hand region, the method further includes: Multiple key hand points were identified in the hand region; And, determining the second sub-region corresponding to the hand region includes: The second sub-region corresponding to the hand region is determined based on the plurality of hand key points.

2. The method according to claim 1, characterized in that, The first region is the intersection of the hand region and the head region, and the second region is the hand region.

3. The method according to claim 1, characterized in that, Before determining the first sub-region corresponding to the head region, the method further includes: Multiple key points of the head were identified in the head region; And, determining the first sub-region corresponding to the head region includes: The first sub-region corresponding to the head region is determined based on the multiple head key points.

4. The method according to claim 3, characterized in that, The step of determining the first sub-region corresponding to the head region based on the plurality of head key points includes: Based on the first target key point included in the plurality of head key points, a first sub-region corresponding to the head region is determined, and the first sub-region is the smallest region containing the first target key point.

5. The method according to claim 4, characterized in that, The first target key point includes key points of facial features located on the facial feature contour, including key points of the mouth corresponding to the mouth contour, key points of the nose corresponding to the nose contour, or key points of the eyes corresponding to the eye contour.

6. The method according to claim 1, characterized in that, The step of determining the second sub-region corresponding to the hand region based on the plurality of hand key points includes: Based on the second target key point included in the plurality of hand key points, a second sub-region corresponding to the hand region is determined, and the second sub-region is the smallest region containing the second target key point.

7. The method according to claim 6, characterized in that, The second target key point includes finger key points located on the finger joints, the fingers including the index finger, middle finger, ring finger and little finger.

8. The method according to claim 1, characterized in that, If the overlap between the first sub-region and the second sub-region satisfies the second condition, then determining that the human body is in an illegal posture includes: If the second ratio of the third region and the fourth region is greater than the second threshold, then the overlap between the first sub-region and the second sub-region satisfies the second condition, and the human body is determined to be in an illegal posture. The third region includes the intersection of the first sub-region and the second sub-region, and the fourth region includes the first sub-region, the second sub-region, or the second union, where the second union is the union of the first sub-region and the second sub-region.

9. The method according to claim 8, characterized in that, The third region is the intersection of the first sub-region and the second sub-region, and the fourth region is the second sub-region.

10. The method according to claim 1, characterized in that, The method further includes: If the overlap between the first sub-region and the second sub-region satisfies the third condition, then the human body is determined to be in a legal posture.

11. The method according to claim 10, characterized in that, The first sub-region includes the eye sub-region corresponding to the eye key point, the nose sub-region corresponding to the nose key point, and the mouth sub-region corresponding to the mouth key point; and, if the overlap between the first sub-region and the second sub-region satisfies a third condition, then determining that the human body is in a legal pose includes: If at least one of the eye sub-region, nose sub-region, and mouth sub-region overlaps with the second sub-region in a manner that satisfies the third condition, then the human body is determined to be in a legal state.

12. The method according to claim 1, characterized in that, The step of determining the head region corresponding to the human head and the hand region corresponding to the human hand from the target image includes: Determine the head region corresponding to the human head from the target image, and determine whether the target pose corresponding to the human head is in an abnormal state; If the target posture corresponding to the human head is not in an abnormal state, then the hand region corresponding to the human hand is determined from the target image.

13. The method according to claim 12, characterized in that, Determining whether the target pose corresponding to the human head is in an abnormal state includes: Determine the target angle corresponding to the Euler angle of the human head; If the target angle is within the target threshold range, then it is determined that the target posture corresponding to the human head is not in an abnormal state; If the target angle is not within the target threshold range, then the target posture corresponding to the human head is determined to be in an abnormal state.

14. The method according to claim 13, characterized in that, The Euler angle includes the yaw angle, the target angle includes a first angle corresponding to the yaw angle, and / or the Euler angle includes the pitch angle, the target angle includes a second angle corresponding to the pitch angle.

15. The method according to claim 1, characterized in that, The step of determining the head region corresponding to the human head and the hand region corresponding to the human hand from the target image includes: The region recognition model is used to determine the head region corresponding to the human head and the hand region corresponding to the human hand from the target image. The region recognition model is a model obtained through compression operations, which include pruning, quantization and distillation operations.

16. The method according to claim 1, characterized in that, When applied to an electronic device, after determining that the human body is in an illegal posture, the method further includes: Output a prompt message and / or perform a restriction operation. The prompt message is used to inform the user that the current posture is illegal. The restriction operation includes controlling the electronic device to switch to a locked screen state or a powered-off state.

17. A device for determining human posture, characterized in that, The device includes: The first determining unit is used to determine the head region corresponding to the human head and the hand region corresponding to the human hand from the target image, wherein the target image is an image including the human body; The second determining unit is configured to, when the first ratio of the first region and the second region is greater than a first threshold, determine that the overlap between the head region and the hand region satisfies a first condition, and determine a first sub-region corresponding to the head region, wherein the first sub-region is located within the head region and the area of ​​the first sub-region is smaller than the area of ​​the head region; determine a second sub-region corresponding to the hand region, wherein the second sub-region is located within the hand region and the area of ​​the second sub-region is smaller than the area of ​​the hand region; if the overlap between the first sub-region and the second sub-region satisfies a second condition, then determine that the human body is in an illegal posture, wherein the first region includes the intersection of the hand region and the head region, and the second region includes the hand region, the head region, or a first union, wherein the first union is the union of the hand region and the head region; Furthermore, the device further includes: a sixth determining unit, configured to determine a plurality of key hand points in the hand region before determining the second sub-region corresponding to the hand region; The fourth determining unit is used to determine the second sub-region corresponding to the hand region based on the plurality of hand key points.

18. An electronic device, characterized in that, The method includes a memory storing executable program code and a processor coupled to the memory; wherein the processor invokes the executable program code stored in the memory to perform the method as described in any one of claims 1 to 16.

19. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 16.

Images