Method, device, electronic device and storage medium for posture detection

Abstract

The present disclosure relates to a method, device, electronic device, and storage medium for posture detection, and relates to the technical field of posture detection. The method comprises: obtaining an image to be detected containing a target object; determining, based on the image to be detected, posture parameters of the target object and position information of key points of the target object, wherein the key points are used to indicate a designated position of the target object; and determining, based on the posture parameters and position information, whether the posture of the target object is abnormal. The present disclosure performs posture detection by separately obtaining the posture parameters of the target object and the position information of the key points from the image to be detected, and combining the posture parameters and the position information of the key points to improve the accuracy of detecting the posture of the target object and avoid misjudging whether the posture of the target object is abnormal.

Description

Technical Field

[0001] The present disclosure relates to the technical field of posture detection, and in particular to a posture detection method, device, electronic device, and storage medium. Background Art

[0002] Currently, more and more people work and study in incorrect postures for a long time, which will affect their physical and mental health. Therefore, reminding and correcting incorrect postures has important practical significance.

[0003] Related technologies primarily use neural network image recognition technology to perform simple posture classification training to detect incorrect postures, or use sensors (such as angle sensors and infrared sensors) to obtain relevant posture information for posture detection. However, these two methods have low accuracy and are prone to misjudging whether the user's posture is abnormal. Summary of the Invention

[0004] To overcome the problems existing in the related art, the present disclosure provides a method, device, electronic device and storage medium for posture detection.

[0005] According to a first aspect of an embodiment of the present disclosure, a method for posture detection is provided, the method comprising:

[0006] Obtain an image to be detected containing a target object;

[0007] Determining, based on the image to be detected, posture parameters of the target object and position information of key points of the target object; the key points are used to indicate designated positions of the target object;

[0008] Determine whether the posture of the target object is abnormal based on the posture parameter and the position information.

[0009] Optionally, determining the posture parameters of the target object and the position information of key points of the target object according to the image to be detected includes:

[0010] According to the image to be detected, the posture parameters and the position information are determined by using a pre-trained posture detection model.

[0011] Optionally, the posture detection model includes a posture parameter determination model and a key point recognition model; determining the posture parameters and the position information based on the image to be detected by using a pre-trained posture detection model includes:

[0012] Determine the posture parameters by using the posture parameter determination model according to the image to be detected;

[0013] The image to be detected is input into the key point recognition model to obtain the position information of each key point.

[0014] Optionally, the posture parameter is a posture angle of a designated part of the target object, and the posture angle includes a pitch angle, a yaw angle, and a roll angle of the designated part; and determining the posture parameter by using the posture parameter determination model according to the image to be detected includes:

[0015] Acquiring a target image containing the designated part from the image to be detected;

[0016] The target image is input into the posture parameter determination model to obtain the posture angle.

[0017] Optionally, determining whether the posture of the target object is abnormal based on the posture parameter and the position information includes:

[0018] Determining, based on the position information, a plurality of key point distances corresponding to each preset key point set, wherein each preset key point set includes at least three key points;

[0019] Determine whether the posture of the target object is abnormal according to the posture parameter and the key point distance.

[0020] Optionally, determining whether the posture of the target object is abnormal based on the posture parameter and the key point distance includes:

[0021] For each of the preset key point sets, determining at least one distance ratio corresponding to the preset key point set according to a plurality of key point distances corresponding to the preset key point set; the distance ratio is a ratio between the distances between two key points;

[0022] If the posture parameter is outside the preset parameter range, and / or the distance ratio corresponding to at least one of the preset key point sets is not within the preset ratio range, it is determined that there is an abnormality in the posture of the target object.

[0023] Optionally, the posture parameter determination model is trained in the following manner:

[0024] Acquire a training sample set; the training sample set includes a training image and training posture parameters corresponding to the training image;

[0025] The preset model is trained according to the training sample set to obtain the posture parameter determination model.

[0026] According to a second aspect of an embodiment of the present disclosure, a device for posture detection is provided, the device comprising:

[0027] An acquisition module is configured to acquire an image to be detected containing a target object;

[0028] a parameter determination module configured to determine, based on the image to be detected, the posture parameters of the target object and the position information of the key points of the target object; the key points are used to indicate the designated positions of the target object;

[0029] The abnormality determination module is configured to determine whether there is an abnormality in the posture of the target object according to the posture parameter and the position information.

[0030] Optionally, the parameter determination module is configured to determine the posture parameters and the position information according to the image to be detected by using a pre-trained posture detection model.

[0031] Optionally, the parameter determination module includes:

[0032] a parameter determination submodule, configured to determine the posture parameters according to the image to be detected by using the posture parameter determination model;

[0033] The processing submodule is configured to input the image to be detected into the key point recognition model to obtain the position information of each key point.

[0034] Optionally, the posture parameter is a posture angle of a designated part of the target object, and the posture angle includes a pitch angle, a yaw angle, and a roll angle of the designated part; and the parameter determination submodule is configured to:

[0035] Acquiring a target image containing the designated part from the image to be detected;

[0036] The target image is input into the posture parameter determination model to obtain the posture angle.

[0037] Optionally, the abnormality determination module includes:

[0038] a distance determination submodule configured to determine, based on the position information, a plurality of key point distances corresponding to each preset key point set; each of the preset key point sets includes at least three key points;

[0039] The abnormality determination submodule is configured to determine whether there is an abnormality in the posture of the target object based on the posture parameters and the key point distance.

[0040] Optionally, the abnormality determination submodule is configured to:

[0041] For each of the preset key point sets, determining at least one distance ratio corresponding to the preset key point set according to a plurality of key point distances corresponding to the preset key point set; the distance ratio is a ratio between the distances between two key points;

[0042] If the posture parameter is outside the preset parameter range, and / or the distance ratio corresponding to at least one of the preset key point sets is not within the preset ratio range, it is determined that there is an abnormality in the posture of the target object.

[0043] Optionally, the posture parameter determination model is trained in the following manner:

[0044] Acquire a training sample set; the training sample set includes a training image and training posture parameters corresponding to the training image;

[0045] The preset model is trained according to the training sample set to obtain the posture parameter determination model.

[0046] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0047] processor;

[0048] a memory for storing processor-executable instructions;

[0049] The processor is configured to execute the steps of the posture detection method provided in the first aspect of the present disclosure.

[0050] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the posture detection method provided in the first aspect of the present disclosure are implemented.

[0051] The technical solutions provided by the embodiments of the present disclosure may have the following beneficial effects:

[0052] The present disclosure first obtains an image to be detected containing a target object, then determines the target object's posture parameters and the position information of key points based on the image to be detected, where the key points are used to indicate a specified location of the target object. Finally, based on the posture parameters and position information, it is determined whether the target object's posture is abnormal. The present disclosure performs posture detection by separately obtaining the target object's posture parameters and the position information of key points from the image to be detected, and combining the posture parameters and the position information of the key points to improve the accuracy of detecting the target object's posture and avoid misjudging whether the target object's posture is abnormal.

[0053] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0055] Figure 1 The figure is a flow chart showing a method for gesture detection according to an exemplary embodiment.

[0056] Figure 2 is based on Figure 1 The illustrated embodiment shows a flow chart of step 102 .

[0057] Figure 3 is a schematic diagram showing a posture parameter according to an exemplary embodiment.

[0058] Figure 4 is based on Figure 1 The illustrated embodiment shows a flow chart of step 103 .

[0059] Figure 5 is a schematic diagram showing an abnormal sitting posture according to an exemplary embodiment.

[0060] Figure 6 is a schematic diagram showing another abnormal sitting posture according to an exemplary embodiment.

[0061] Figure 7 The figure is a flowchart of a training posture parameter determination model according to an exemplary embodiment.

[0062] Figure 8 It is a block diagram showing a device for gesture detection according to an exemplary embodiment.

[0063] Figure 9 is based on Figure 8 The illustrated embodiment shows a block diagram of a parameter determination module.

[0064] Figure 10 is based on Figure 8 The illustrated embodiment shows a block diagram of an abnormality determination module.

[0065] Figure 11 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0066] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0067] Before introducing the method, device, electronic device and storage medium for posture detection provided by the present disclosure, the application scenarios involved in each embodiment of the present disclosure are first introduced. The application scenario can be any scenario in which the posture of the target object needs to be detected. Among them, the target object can be a person, an animal, or an instrument (such as an intelligent robot) that can present a certain posture. The present disclosure does not make specific limitations on this. For example, when people are reading or watching the content displayed on the screen of a terminal device (such as taking an online class, watching a movie, etc.), they often involuntarily get too close to the book or screen, causing eye fatigue, or sitting in a bad posture, causing scoliosis, affecting physical and mental health. It is necessary to detect whether the person's sitting posture is abnormal, so that when the person's sitting posture is abnormal, a prompt can be given to restore the normal sitting posture in time, thereby ensuring the person's physical and mental health. For another example, in scenes such as fitness and dancing, it is also possible to detect whether a person's movement posture is abnormal (that is, whether the person's movement posture is standard).

[0068] Figure 1 FIG. 1 is a flow chart showing a method for posture detection according to an exemplary embodiment. Figure 1 As shown, the method may include the following steps:

[0069] In step 101, an image to be detected containing a target object is obtained.

[0070] For example, when the posture of the target object is detected, it is necessary to set an image acquisition device in advance at a designated position. Among them, the image acquisition device can be a device capable of acquiring images, such as a camera, a video camera, or an infrared image sensor, and the designated position can be any position where the image acquisition device can capture the complete posture of the target object. Then, the image acquisition device can be used to capture an image to be detected containing the target object. For example, in the case where the target object is a person, when the target object is reading or studying, the camera can be set on a desk lamp to capture an image to be detected containing the upper body of the target object when the target object sits in front of the desk lamp, and the image to be detected is used to detect whether the sitting posture of the target object is abnormal (i.e., whether the posture of the target object is abnormal). For another example, in the case where the target object is a person, when the target object watches the content displayed on the screen of a terminal device (such as a smart phone, tablet computer, and laptop computer, etc.), the camera of the terminal device can be used as an image acquisition device to capture an image to be detected containing the upper body of the target object when the target object sits in front of the terminal device, and the image to be detected is used to detect whether the user's sitting posture is abnormal.

[0071] In step 102, the posture parameters of the target object and the position information of the key points of the target object are determined based on the image to be detected. The key points are used to indicate the specified positions of the target object.

[0072] For example, to accurately detect the posture of a target object, quantifiable parameters can be extracted from the image to be detected and used to determine whether the target object's posture is abnormal. Specifically, the target object's posture parameters and the position information of key points of the target object can be selected as quantifiable parameters to determine whether the target object's posture is abnormal. A posture detection model is pre-trained to extract the posture parameters and the position information of key points from the image to be detected. After acquiring the image to be detected, the posture parameters and position information are determined based on the image to be detected using the pre-trained posture detection model. The posture parameters are measurable parameters that can reflect the posture of the target object, such as the posture angle, height, or movement speed of a specified part of the target object. Key points are used to indicate the specified location of the target object. For example, if the target object is a person, the key points can be multiple pre-set specific human body parts of the target object (such as the eyes, ears, nose, corners of the mouth, neck, shoulders, center of the chest, knees, feet, elbows, hands, and other human body parts).

[0073] In step 103, it is determined whether the posture of the target object is abnormal based on the posture parameters and position information.

[0074] For example, when the target object's posture is abnormal, the distances between certain key points of the target object are different from when the target object's posture is normal. Taking the target object as an example, when the target object is reading or studying, the distance between the key point corresponding to the tip of the nose and the key point corresponding to the top of the shoulder is smaller when the sitting posture is incorrect than when the sitting posture is correct. Therefore, the key point distances between the key points can be used to determine whether the target object's posture is abnormal. At the same time, the abnormal posture parameters of the target object when the target object's posture is abnormal can be determined in advance based on a large number of experiments, and a preset parameter range for determining whether the target object's posture is abnormal can be set based on the abnormal posture parameters.

[0075] Furthermore, after determining the posture parameters and position information, the key point distance between the key points of the target object can be determined based on the position information of the key points, and then the presence of an abnormality in the posture of the target object can be determined based on the key point distance. Simultaneously, the presence of an abnormality in the posture of the target object can also be determined in combination with the posture parameters of the target object. If the posture parameters of the target object are outside a preset parameter range, the posture of the target object is considered to be abnormal; otherwise, the posture of the target object is considered to be normal.

[0076] Then, if it is determined that the target object's posture is abnormal, a prompt message can be issued to prompt the target object to restore a normal posture in a timely manner. For example, in the case where the target object is a person, when the target object is viewing the content displayed on the screen of the terminal device, if it is determined that the target object's posture is abnormal, a corresponding prompt text (such as "Please adjust your sitting posture") can be displayed on the screen of the terminal device, or an indicator light on the terminal device can be controlled to flash in a preset manner (such as controlling the indicator light to flash at a preset frequency and color), or a voice prompt can be issued by controlling the speaker on the terminal device to prompt the user to restore a normal posture, thereby achieving the effect of preventing myopia and scoliosis.

[0077] It should be noted that in order to improve the accuracy and stability of posture detection, multiple images to be detected containing the target object can be first obtained, and then steps 102 to 103 can be performed on each image to be detected to determine whether the posture of the target object in each image to be detected is abnormal. The first number of images to be detected in which the posture of the target object is determined to be abnormal and the second number of images to be detected in which the posture of the target object is determined to be normal can be recorded. Finally, whether the posture of the target object is abnormal can be determined based on the first number and the second number. For example, if the first number is greater than or equal to the second number, the posture of the target object is determined to be abnormal; otherwise, the posture of the target object is determined to be normal.

[0078] In summary, the present disclosure first obtains an image to be detected containing a target object, then determines the target object's posture parameters and the position information of key points based on the image to be detected, wherein the key points are used to indicate a specified position of the target object, and finally determines whether the target object's posture is abnormal based on the posture parameters and position information. The present disclosure performs posture detection by separately obtaining the target object's posture parameters and the position information of key points from the image to be detected, and combining the posture parameters and the position information of the key points to improve the accuracy of detecting the target object's posture and avoid misjudging whether the target object's posture is abnormal.

[0079] Figure 2 is based on Figure 1 The embodiment shown is a flow chart of step 102. Figure 2 As shown, the posture detection model includes a posture parameter determination model and a key point recognition model, and step 102 may include the following steps:

[0080] In step 1021, a model is determined by determining the posture parameters based on the image to be detected.

[0081] For example, a posture detection model may include a posture parameter determination model and a key point recognition model. These models can be independent of each other or the same model. The posture parameter determination model is used to determine the posture parameters of the target object based on the image to be detected, while the key point recognition model is used to determine the position information of the key points of the target object based on the image to be detected.

[0082] In one scenario, the posture parameter can be the posture angle of a specified part of the target object, and the posture angle can include the pitch angle, yaw angle, and roll angle of the specified part. Specifically, the posture parameter determination model can be trained in advance based on training images marked with different posture angles. After the posture parameter determination model is trained, the target image containing the specified part can be obtained from the image to be detected. For example, in the scenario of detecting whether a person's sitting posture is abnormal, the specified part can be the person's head, and the posture angle can be the pitch angle, yaw angle, and roll angle of the person's head, such as Figure 3 As shown, Figure 3 In this case, pitch is the pitch angle, yaw is the yaw angle, and roll is the roll angle. These posture parameters actually represent the 3D posture of the human head. After acquiring the image to be detected, face detection can be performed on the image to locate the target face. The face image can then be cropped from the image to be detected (i.e., an image containing the target head is obtained). The face image is then scaled (for example, to a size of 112x112), and pixel values ​​of the scaled face image are normalized to obtain the target image.

[0083] After obtaining the target image, the target image can be input into a posture parameter determination model to obtain the posture angle, that is, the pitch angle, yaw angle and roll angle of the head of the target object output by the posture parameter determination model.

[0084] In step 1022, the image to be detected is input into a key point recognition model to obtain the position information of each key point.

[0085] For example, while determining the posture parameters, the image to be detected can also be input into the key point recognition model, and the key point recognition model outputs the position information of each key point. The position information of the key point can be the position coordinates of the key point. For example, in a scenario where a person's sitting posture is detected to see if there is any abnormality, the key point can be a skeletal point of the human body (such as the tip of the nose, chin, left and right eyes, left and right ears, and left and right shoulders). In this case, the key point recognition model can be a skeletal point detection model. After acquiring the image to be detected, the image to be detected can be input into the skeletal point detection model to obtain the position coordinates of each skeletal point output by the skeletal point detection model (i.e., the position information of each key point).

[0086] Figure 4 is based on Figure 1 The embodiment shown is a flow chart of step 103. Figure 4 As shown, step 103 may include the following steps:

[0087] In step 1031, multiple key point distances corresponding to each preset key point set are determined based on the position information, wherein each preset key point set includes at least three key points.

[0088] For example, a comparative analysis can be performed beforehand between the distances between key points when the target object's posture is abnormal and the distances between key points when the target object's posture is normal, thereby selecting a specified number of preset key point sets. Each preset key point set includes at least three key points that can be used to determine whether the target object's posture is abnormal. Based on the position information of each key point, the multiple key point distances corresponding to each preset key point set can then be determined.

[0089] In step 1032, it is determined whether the posture of the target object is abnormal based on the posture parameters and the key point distances.

[0090] Furthermore, for each preset key point set, at least one distance ratio corresponding to the preset key point set can be determined based on the distances of multiple key points corresponding to the preset key point set. The distance ratio is the ratio of the distances between two key points. If the posture parameters are outside the preset parameter range, and / or the distance ratio corresponding to at least one preset key point set is not within the preset ratio range, it is determined that the posture of the target object is abnormal.

[0091] For example, in the scenario of detecting whether a person's sitting posture is abnormal, if the posture parameters are the pitch angle, yaw angle and roll angle of the person's head, the preset parameter ranges corresponding to the pitch angle, yaw angle and roll angle can be pre-set when the person's sitting posture is normal. After obtaining the pitch angle, yaw angle and roll angle of the target object's head output by the posture parameter determination model, judge whether the pitch angle, yaw angle and roll angle of the target object's head meet the preset parameter range (that is, judge whether the pitch angle, yaw angle and roll angle are within the corresponding preset parameter range). If yes, it is determined that the person's sitting posture is normal; if not, it is determined that the person's sitting posture is abnormal. At the same time, when the person's sitting posture is normal, the distances between the tip of the nose and the top of the left shoulder and the top of the right shoulder are basically the same. When the person's sitting posture is abnormal, the person's head may appear to be biased to one side of the shoulder, such as Figure 5 As shown, if this situation persists for a long time, it will cause scoliosis in the user. The distance between the nose tip and the left and right shoulders can be used to determine whether the person's sitting posture is abnormal. Therefore, the three key points corresponding to the nose tip, the top of the left shoulder, and the top of the right shoulder can be used as a preset key point set, and a preset ratio range (e.g., 0.9-1.1) can be pre-set. The preset ratio range is used to characterize the range of the ratio between the key point distance between the key point corresponding to the nose tip and the key point corresponding to the left shoulder top, and the key point distance between the key point corresponding to the nose tip and the key point corresponding to the right shoulder top when the person's sitting posture is normal. After obtaining the position information of each key point output by the key point recognition model, the key point distance between the key point corresponding to the nose tip and the key point corresponding to the left shoulder top, and the key point distance between the key point corresponding to the nose tip and the key point corresponding to the right shoulder top can be calculated. The distance ratio between these two key point distances is then calculated, and it is determined whether the distance ratio is within the preset ratio range. If it is, it is determined that the person's sitting posture is normal. If not, it is determined that the person's sitting posture is abnormal.

[0092] Furthermore, the key point distance can be not only the distance between the lines of two key points, but also the distance between the two key points in the vertical direction. For example, in the scenario of detecting whether a person's sitting posture is abnormal, when the head posture angle is normal and the distance between the head and the shoulders is symmetrical, there are other abnormal sitting postures (such as a person lying on a desk, which can easily cause the person to hunch back). The following is an example of an abnormal sitting posture of a person lying on a desk. Figure 6 As shown. If the head posture angle is normal (i.e. the head is not pitching), the distance from the eyes to the nose tip is constant. However, if a person lies on a desk, the distance from the nose tip to the top of the shoulder (left shoulder top or right shoulder top) will be shortened. At this time, the ratio of the distance from the eyes to the nose tip to the distance from the nose tip to the top of the shoulder will exceed the normal range. Therefore, the three key points corresponding to the eyes, nose tip, and shoulder top can be used as a preset key point set, and a preset ratio range can be pre-set. The preset ratio range is used to characterize the key point distances in the vertical direction between the key points corresponding to the eyes and the key points corresponding to the nose tip, and the range of the ratios between the key point distances in the vertical direction between the key points corresponding to the nose tip and the key points corresponding to the shoulder top when there is no abnormality in the sitting posture of the person. After obtaining the position information of each key point output by the key point recognition model, the key point distances in the vertical direction between the key points corresponding to the eyes and the key points corresponding to the nose tip, and the key point distances in the vertical direction between the key points corresponding to the nose tip and the key points corresponding to the shoulder top can be calculated. Then the distance ratio between the two key point distances is calculated, and it is determined whether the distance ratio is within the preset ratio range. If so, it is determined that there is nothing abnormal in the person's sitting posture; if not, it is determined that there is something abnormal in the person's sitting posture.

[0093] It should be noted that there can be multiple preset key point sets, and each preset key point set can calculate a corresponding distance ratio. As long as the distance ratio corresponding to one preset key point set is not within the preset ratio range, it can be determined that there is an abnormality in the posture of the target object.

[0094] In addition, during actual use, the target object may not be facing the image acquisition device, and the pitch angle, yaw angle and roll angle may vary with different application devices. At the same time, the differences in the target objects may also cause the preset parameter range and the preset ratio range to not be completely suitable. Therefore, during actual use, a parameter adjustment function can be provided for the target object (that is, the preset parameter range and the preset ratio range can be adjusted) to improve the accuracy and adaptability of posture detection. For example, in order to improve the accuracy of abnormal sitting posture detection, the user can make a correction in the first posture detection with a normal sitting posture and the commonly used placement position of the image acquisition device to update the preset parameter range and the preset ratio range.

[0095] Figure 7 FIG. 1 is a flow chart showing a training posture parameter determination model according to an exemplary embodiment. Figure 7 As shown in Figure 2, the posture parameter determination model is trained through the following steps:

[0096] In step 201, a training sample set is obtained, where the training sample set includes training images and training posture parameters corresponding to the training images.

[0097] For example, when training a posture parameter determination model, an image acquisition device can be fixed in a position facing the target object, and a large number of images (e.g., 10,000) of different target objects in different postures can be collected as training images. The training posture parameters corresponding to each training image are annotated. The training images need to cover different postures. Each training image and the training posture parameters corresponding to the training image can then be used as a training sample set.

[0098] For example, in the scenario of detecting whether a person's sitting posture is abnormal, the training images can include various postures such as head upright, head lowered, head raised, head tilted to the left, and head tilted to the right. After obtaining the training images, the training images can be subjected to a face detection algorithm (e.g., using the CenterFace network to perform face detection on the training images) to locate the faces in the training images and crop out clear and complete face images. Then, the key points of the face (e.g., 21 points such as the left and right corners of the eyes, the tip of the nose, and the left and right corners of the mouth) can be manually annotated. Based on the annotated key points of the face, the solvePnP projection function of the Opencv image library is used to fit a general 68-point average face model (e.g., a BFM three-dimensional face model), and the fitting parameters are solved to obtain the 3D posture corresponding to each training image and the training posture parameters corresponding to the 3D posture.

[0099] In step 202, a preset model is trained based on a training sample set to obtain a posture parameter determination model.

[0100] Specifically, N training images can be taken as training samples each time, where N is the number of images in each iteration of the training process, for example, N = 256. Then, for each training image, the size of the training image can be uniformly scaled to 112x112 by bilinear interpolation, and the pixel values ​​of each training image can be normalized using the normalization formula. The normalization formula can be expressed as: Norm =(I-127.5) / 127.5, I Norm Represents the normalized training image, and I represents the original training image. The normalized training image can then be used as the input of the preset model, and the training posture parameters corresponding to the training image can be used as the output of the preset model to train the preset model. During the training process, the preset model is updated using a loss function until the preset model meets the preset training conditions, thereby obtaining a posture parameter determination model. The preset training conditions can, for example, be completing a training run of the preset model using all training images, and the loss function can use the smooth L1 loss function, which can be expressed as: x represents the difference between the pose parameters estimated by the preset model and the training pose parameters.

[0101] It should be noted that the preset model can use a convolutional neural network, which is mainly divided into two parts: the backbone network and the head network. The backbone network can adopt the lightweight network structure MobileNet to extract the semantic features of the image, and the head network can be composed of a 7*7 pooling layer and a 256*3-dimensional fully connected layer to fuse the features extracted by the backbone network and output 3D posture parameters. For example, when the posture parameters are pitch angle, yaw angle and roll angle, the head network can output a three-dimensional posture parameter vector, denoted as Pose. Pose can be expressed as: Pose = {pitch, yaw, roll}, where pitch, yaw, roll respectively represent the pitch angle, yaw angle and roll angle of the specified part of the target object. In addition, the post-processing formula can be used to post-process Pose to convert Pose from radians to common angle units. Among them, the post-processing formula can be expressed as:

[0102] In summary, the present disclosure first obtains an image to be detected containing a target object, then determines the target object's posture parameters and the position information of key points based on the image to be detected, wherein the key points are used to indicate a specified position of the target object, and finally determines whether the target object's posture is abnormal based on the posture parameters and position information. The present disclosure performs posture detection by separately obtaining the target object's posture parameters and the position information of key points from the image to be detected, and combining the posture parameters and the position information of the key points to improve the accuracy of detecting the target object's posture and avoid misjudging whether the target object's posture is abnormal.

[0103] Figure 8 FIG. 1 is a block diagram of a device for gesture detection according to an exemplary embodiment. Figure 8 As shown, the device 300 includes an acquisition module 301 , a parameter determination module 302 and an abnormality determination module 303 .

[0104] The acquisition module 301 is configured to acquire an image to be detected containing a target object.

[0105] The parameter determination module 302 is configured to determine the posture parameters of the target object and the position information of the key points of the target object based on the image to be detected. The key points are used to indicate the specified positions of the target object.

[0106] The abnormality determination module 303 is configured to determine whether there is an abnormality in the posture of the target object according to the posture parameters and the position information.

[0107] Optionally, the parameter determination module is configured to determine the posture parameters and position information according to the image to be detected by using a pre-trained posture detection model.

[0108] Figure 9is based on Figure 8 The embodiment shown is a block diagram of a parameter determination module. Figure 9 As shown, the parameter determination module 302 includes:

[0109] The parameter determination submodule 3021 is configured to determine the posture parameters according to the image to be detected by using the posture parameter determination model.

[0110] The processing submodule 3022 is configured to input the image to be detected into the key point recognition model to obtain the position information of each key point.

[0111] Optionally, the posture parameter is the posture angle of a specified part of the target object, and the posture angle includes the pitch angle, yaw angle and roll angle of the specified part. The parameter determination submodule 3021 is configured to:

[0112] From the image to be detected, obtain the target image containing the specified part.

[0113] The target image is input into the attitude parameter determination model to obtain the attitude angle.

[0114] Figure 10 is based on Figure 8 The embodiment shown is a block diagram of an abnormality determination module. Figure 10 As shown, the abnormality determination module 303 includes:

[0115] The distance determination submodule 3031 is configured to determine the distances of multiple key points corresponding to each preset key point set according to the position information. Each preset key point set includes at least three key points.

[0116] The abnormality determination submodule 3032 is configured to determine whether there is any abnormality in the posture of the target object based on the posture parameters and the key point distance.

[0117] Optionally, the abnormality determination submodule 3032 is configured to:

[0118] For each preset key point set, at least one distance ratio corresponding to the preset key point set is determined based on the multiple key point distances corresponding to the preset key point set, wherein the distance ratio is the ratio between the distances between two key points.

[0119] When the posture parameter is outside the preset parameter range, and / or the distance ratio corresponding to at least one preset key point set is not within the preset ratio range, it is determined that there is an abnormality in the posture of the target object.

[0120] Optionally, the pose parameter determination model is trained by:

[0121] A training sample set is obtained, wherein the training sample set includes training images and training posture parameters corresponding to the training images.

[0122] The preset model is trained according to the training sample set to obtain the posture parameter determination model.

[0123] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0124] In summary, the present disclosure first obtains an image to be detected containing a target object, then determines the target object's posture parameters and the position information of key points based on the image to be detected, wherein the key points are used to indicate a specified position of the target object, and finally determines whether the target object's posture is abnormal based on the posture parameters and position information. The present disclosure performs posture detection by separately obtaining the target object's posture parameters and the position information of key points from the image to be detected, and combining the posture parameters and the position information of the key points to improve the accuracy of detecting the target object's posture and avoid misjudging whether the target object's posture is abnormal.

[0125] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon. When the program instructions are executed by a processor, the steps of the posture detection method provided by the present disclosure are implemented.

[0126] Figure 11 8 is a block diagram of an electronic device according to an exemplary embodiment. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0127] Reference Figure 11 , the electronic device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0128] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the gesture detection method described above. In addition, the processing component 802 may include one or more modules to facilitate interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

[0129] The memory 804 is configured to store various types of data to support operations on the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0130] The power component 806 provides power to the various components of the electronic device 800. The power component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 800.

[0131] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor can not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0132] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), which is configured to receive external audio signals when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0133] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as a keyboard, click wheel, buttons, etc. These buttons may include but are not limited to: a home button, volume buttons, a start button, and a lock button.

[0134] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the electronic device 800. For example, the sensor assembly 814 can detect the open / closed state of the electronic device 800, the relative positioning of components, such as the display and keypad of the electronic device 800. The sensor assembly 814 can also detect changes in the position of the electronic device 800 or a component of the electronic device 800, the presence or absence of user contact with the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and temperature changes of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0135] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0136] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-mentioned posture detection method.

[0137] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions. The instructions can be executed by the processor 820 of the electronic device 800 to perform the above-mentioned gesture detection method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0138] In another exemplary embodiment, a computer program product is further provided. The computer program product includes a computer program executable by a programmable device, and has a code portion for performing the above-mentioned gesture detection method when executed by the programmable device.

[0139] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the present disclosure. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0140] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for posture detection, characterized in that: The method comprises: Obtain an image to be detected containing a target object; Determining, based on the image to be detected, posture parameters of the target object and position information of key points of the target object; the key points are used to indicate designated positions of the target object; Determining whether the posture of the target object is abnormal based on the posture parameter and the position information; The determining, based on the posture parameter and the position information, whether the posture of the target object is abnormal includes: Determining, based on the position information, a plurality of key point distances corresponding to each of a plurality of preset key point sets; the key point distances comprising a vertical distance between two key points; and each of the preset key point sets comprising at least three key points for determining whether a posture of the target object is abnormal. For each of the preset key point sets, determining at least one distance ratio corresponding to the preset key point set according to a plurality of key point distances corresponding to the preset key point set; the distance ratio is a ratio between the distances between two key points; When the posture parameter is outside the preset parameter range, and / or the distance ratio corresponding to at least one of the preset key point sets is not within the preset ratio range, determining that the posture of the target object is abnormal; The preset parameter range and the preset ratio range are obtained by correction based on the normal sitting posture of the target object and the commonly used placement position of the image acquisition device.

2. The method according to claim 1, characterized in that The step of determining the posture parameters of the target object and the position information of the key points of the target object according to the image to be detected includes: According to the image to be detected, the posture parameters and the position information are determined by using a pre-trained posture detection model.

3. The method according to claim 2, characterized in that The posture detection model includes a posture parameter determination model and a key point recognition model; the posture parameters and the position information are determined based on the image to be detected by using a pre-trained posture detection model, including: Determine the posture parameters by using the posture parameter determination model according to the image to be detected; The image to be detected is input into the key point recognition model to obtain the position information of each key point.

4. The method according to claim 3, characterized in that The posture parameter is a posture angle of a designated part of the target object, and the posture angle includes a pitch angle, a yaw angle, and a roll angle of the designated part; determining the posture parameter by using the posture parameter determination model according to the image to be detected, including: Acquiring a target image containing the designated part from the image to be detected; The target image is input into the posture parameter determination model to obtain the posture angle.

5. The method according to claim 3 or 4, characterized in that The posture parameter determination model is trained in the following way: Acquire a training sample set; the training sample set includes a training image and training posture parameters corresponding to the training image; The preset model is trained according to the training sample set to obtain the posture parameter determination model.

6. A posture detection device, characterized in that: The device comprises: An acquisition module is configured to acquire an image to be detected containing a target object; a parameter determination module configured to determine, based on the image to be detected, posture parameters of the target object and position information of key points of the target object; the key points are used to indicate a specified position of the target object; the posture parameters are measurable parameters reflecting the posture of the target object; an abnormality determination module, configured to determine whether there is an abnormality in the posture of the target object based on the posture parameter and the position information; The abnormality determination module is configured to determine, based on the position information, a plurality of key point distances corresponding to each preset key point set in a plurality of preset key point sets; the key point distance includes a distance between two key points in a vertical direction; each of the preset key point sets includes at least three key points for determining whether there is an abnormality in the posture of the target object; for each of the preset key point sets, determine, based on the plurality of key point distances corresponding to the preset key point set, at least one distance ratio corresponding to the preset key point set; the distance ratio is the ratio between the distances between two key points; if the posture parameter is outside a preset parameter range, and / or the distance ratio corresponding to at least one of the preset key point sets is not within a preset ratio range, it is determined that there is an abnormality in the posture of the target object; The preset parameter range and the preset ratio range are obtained by correction based on the normal sitting posture of the target object and the commonly used placement position of the image acquisition device.

7. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to execute the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images