Attitude detection method and equipment

Abstract

The embodiment of the invention provides a posture detection method and device, and relates to the technical field of electronics. The method can be applied to a first device, the first device can obtain a first image, and the first image comprises at least one shooting object. Then, the first equipment can perform posture detection on the first image, determine human body joint point position data corresponding to the at least one shooting object, and determine posture parameters according to the human body joint point position data; the posture parameters comprise joint angles corresponding to the human body joint points, and the posture parameters are used for representing the back posture corresponding to the at least one shooting object. Then, when the first equipment determines that the back posture of the shooting object is an abnormal posture according to the posture parameters, first information is output; the first information is used for prompting to adjust the back posture of the shot object. A user does not need to depend on a worn correction product when using the electronic equipment, and a more humanized posture detection function is provided for the user.

Description

Technical Field

[0001] The present application relates to the field of electronic technology, and in particular to a posture detection method and device. Background Art

[0002] With the rapid popularization of electronic devices such as mobile phones, computers and televisions, people's lives have been brought a lot of convenience and enjoyment. As users spend more and more time watching electronic devices, problems such as cervical spine problems, tilted head, hunchback and strabismus caused by watching electronic devices with bad posture are becoming increasingly prominent.

[0003] Usually, see Figure 1 , users will use posture correction products to constrain their posture during the viewing process. For example, teenagers will wear correction products before watching electronic devices so that the correction products can remind and correct their viewing posture. Then, after watching, the teenagers need to take off the correction products. In other words, users need to rely on correction products every time they watch electronic devices, and they need to wear and take off repeatedly, which has certain usage limitations. This reduces the user experience. Summary of the invention

[0004] The embodiment of the present application provides a posture detection method and device, which can perform posture detection based on the collected image and determine the posture parameter corresponding to at least one photographed object. When it is determined that the back posture of the photographed object is abnormal according to the posture parameter, a posture reminder is given. The user does not need to rely on correction products for posture constraints, which simplifies the user's operation steps.

[0005] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0006] In a first aspect, an embodiment of the present application provides a posture detection method, which is applied to a first device. The method includes: the first device can obtain a first image, and the first image includes at least one photographed object.

[0007] Next, the first device may perform posture detection on the first image, determine the human joint point position data corresponding to at least one photographed object, and determine the posture parameters according to the human joint point position data. The posture parameters include the joint angles corresponding to the human joint points, and the posture parameters are used to characterize the back posture corresponding to at least one photographed object.

[0008] Then, when the first device determines that the back posture of the subject is abnormal according to the posture parameters, it outputs the first information; the first information is used to prompt the subject to adjust the back posture.

[0009] It can be seen that the embodiment of the present application can analyze the human body joint point position data from the image, and can restore the specific orientation and posture of the current user. It is understandable that the human body joint point position data includes the joint point position corresponding to at least one joint. In other words, the embodiment of the present application can use joint angles that are richer than bone information to evaluate the user's back posture. It can improve the accuracy of detecting the user's posture and make the classification of posture more refined. At the same time, there is no need for users to rely on wearable correction products when using electronic devices, providing users with a more humane posture detection function.

[0010] In an implementation of the first aspect, the first device may acquire the first image while playing the target media resource during the process of acquiring the first image, and may pause playing the target media resource when the first device determines, based on the posture parameter, that the back posture of the photographed subject is abnormal.

[0011] Then, the first device may acquire the second image, and when it is determined that the back posture of at least one photographed object is a normal posture according to the posture parameters obtained from the second image, the target media resource is resumed to be played.

[0012] It can be seen that the use scenario of the embodiment of the present application can be a scenario of playing the target media resource. For example, when a teenager uses the first device to watch the target media resource, the first device can suspend the playback of the target media resource when the teenager's back posture is determined to be abnormal according to the posture parameter, so as to restrict the teenager's posture. At the same time, it will also prevent the teenager from affecting the viewing of the target media resource during the viewing of the first information.

[0013] Then, the first device can continue to capture the second image, and when it is determined that the teenager's back posture is normal according to the posture parameters obtained from the second image, the target media resource can be resumed to be played. In other words, the first device can perform posture detection and posture reminder on the user by itself, and when it is determined that the user's back posture is abnormal, the target media resource can be suspended to be played, and when it is determined that the user's back posture is adjusted from abnormal to normal, the target media resource can be resumed to be played. Furthermore, the user can be constrained to adjust the current abnormal posture to normal posture, which has a strong constraining effect.

[0014] In an implementation method of the first aspect, the first information includes first prompt information; when the first device determines that the back posture of the subject is abnormal according to the posture parameters, in the process of outputting the first information, when it is determined that there is a target subject among N subjects according to preset conditions, the first prompt information is output; the first prompt information is used to prompt the adjustment of the back posture of the target subject, and N is a positive integer.

[0015] It can be seen that in the embodiment of the present application, when multiple shooting objects are in abnormal postures, if there is a target shooting object among the multiple shooting objects, the first prompt information corresponding to the target shooting object can be output. That is to say, when multiple shooting objects are in abnormal postures, not only posture reminders can be given to multiple shooting objects, but also posture reminders can be given to one or some shooting objects among the multiple shooting objects, so as to realize the process of accurate reminders for shooting objects. A more humanized posture reminder function is provided to users, improving the user experience.

[0016] In an implementation method of the first aspect, the first information includes first prompt information; when the first device determines that the back posture of a subject is abnormal according to the posture parameters, in the process of outputting the first information, when it is determined that the back posture of N subjects is abnormal according to the posture parameters, the posture detection result is sent to the second device; wherein N is a positive integer, the posture detection result includes the abnormal postures corresponding to the N subjects, and the second device is associated with the first device.

[0017] Next, the first device may receive a trigger instruction sent by the second device; the trigger instruction is used to instruct to remind the target subject among the N subjects of the photographing, and output first prompt information according to the trigger instruction; the first prompt information is used to prompt to adjust the back posture of the target subject of the photographing.

[0018] It can be seen that in the embodiment of the present application, when multiple photographed subjects are in abnormal postures, the posture detection results are sent to other devices. Then, a trigger instruction from the other device is received, and the trigger instruction is used to instruct to perform a posture reminder for the target photographed subject among the multiple photographed subjects. And the first prompt information corresponding to the target photographed subject is output.

[0019] That is to say, when multiple subjects are in abnormal postures, not only can posture reminders be given to multiple subjects, but other devices can also be used to select one or some of the multiple subjects for posture reminders, so as to achieve a process of accurate reminders for the subjects. The embodiments of the present application can provide users with more humane and richer posture reminder functions through information linkage between devices, thereby improving the user's usage experience.

[0020] In an implementation manner of the first aspect, the first device may acquire a target three-dimensional skeleton model during the process of outputting the first prompt information, where the target three-dimensional skeleton model is a three-dimensional skeleton model of a human body corresponding to a normal posture.

[0021] Next, a dynamic image is generated based on the human 3D skeleton model corresponding to the target photographed object and the target 3D skeleton model; the dynamic image presents the process of the human 3D skeleton model corresponding to the target photographed object changing to the target 3D skeleton model.

[0022] Then, the first device outputs a first prompt message, the first prompt message includes a dynamic image, and the first prompt message also includes a voice prompt message, the voice prompt message is used to prompt the target subject to follow the dynamic image to adjust the back posture to a normal posture.

[0023] It can be seen that in the process of reminding the user of posture, the embodiment of the present application can use a combination of audio and video prompts to remind the user of posture. The first device can obtain the three-dimensional skeleton model of the human body corresponding to the normal posture and the three-dimensional skeleton model of the human body corresponding to the abnormal posture of the current user, and generate a dynamic image. The dynamic image is used to guide the user how to adjust the current abnormal posture to a normal posture. At the same time, the first device can also output voice prompt information in the process of outputting the dynamic image, guiding the target subject to follow the dynamic image to adjust the back posture to a normal posture. It can allow the user to perceive how to correctly adjust the back posture, and can also constrain the user's posture.

[0024] In an implementation manner of the first aspect, the first information includes one or more of pop-up prompt information, text prompt information, voice prompt information, or video prompt information.

[0025] In an implementation method of the first aspect, after the first device determines that the back posture of the subject is an abnormal posture based on the posture parameters, it outputs the posture statistics corresponding to each subject; the posture statistics include the abnormal posture and the posture holding time corresponding to the abnormal posture.

[0026] It can be seen that the embodiment of the present application can support the output of the posture statistics corresponding to each photographed object. It can allow the user to perceive which abnormal posture he is in and how long the posture is maintained in the abnormal posture. It provides users with a more humane posture detection function and improves the user's experience.

[0027] In an implementation manner of the first aspect, the first device may perform target detection on the first image to obtain at least one first region during the process of performing posture detection on the first image and determining the human joint point position data corresponding to at least one photographed object. Then, the first device may perform posture detection on the at least one first region to determine the human joint point position data corresponding to at least one photographed object.

[0028] It can be seen that the embodiment of the present application can perform target detection on the first image to obtain at least one first area. Then, posture detection is performed on the at least one first area. It can be understood that the first area refers to an image area selected from the first image, which is the focus of image analysis. By delineating the area including the target features as a prerequisite for further processing of the image, the image processing time can be reduced and the image processing accuracy can be improved.

[0029] In an implementation manner of the first aspect, during the process of performing gesture detection on at least one first area, the first device may determine J second areas in the at least one first area based on a preset rule, where J is a positive integer, and perform gesture detection on the J second areas.

[0030] The preset rule includes determining, in at least one first area, areas other than the area where the photographed subject holding the preset item is located as J second areas;

[0031] Alternatively, the preset rule includes determining the area where the photographed subjects whose age value is less than or equal to the age threshold are located in at least one first area as J second areas;

[0032] Alternatively, the preset rule includes determining, in the at least one first area, an area where the photographed object closest to the image center of the first image is located as the J second areas.

[0033] It can be seen that in the embodiment of the present application, when multiple people use the first device, the first device can not only perform posture detection for each user, but also perform posture detection only for pre-set users.

[0034] For example, gesture detection and gesture reminder are performed on users other than the remote control holder.

[0035] For another example, posture detection and posture reminders are performed on teenagers or children.

[0036] For another example, posture detection and posture reminder are performed on the user who is close to the center of the image.

[0037] Users can set users who need to perform posture detection at any time and anywhere, and can also switch users who need posture detection at any time and anywhere. It can provide users with more humanized posture detection functions and enhance the user experience.

[0038] In an implementation manner of the first aspect, the abnormal posture includes a hunchback posture, a side-lying posture or a backward leaning posture; and the posture parameters include a forward leaning angle, a trunk backward bending angle and a lateral tilt angle corresponding to the photographed object.

[0039] In the process of the first device determining that the back posture of the subject is abnormal based on the posture parameters, it can be determined that the back posture of the subject is a hunchback posture when the forward leaning angle is positive and greater than the first preset angle, the absolute value of the trunk backward bending angle is greater than the second preset angle, and the absolute value of the lateral tilt angle is less than the third preset angle.

[0040] The first device may determine that the back posture of the subject is a side-lying posture when the absolute value of the lateral tilt angle is greater than or equal to a third preset angle.

[0041] The first device may determine that the back posture of the subject is a backward posture when the forward tilt angle is a negative value and is smaller than a fourth preset angle, and the fourth preset angle is smaller than the first preset angle.

[0042] And in the process of determining that the back posture of the subject is lying on its side when the absolute value of the lateral inclination angle is greater than or equal to the third preset angle, it can be determined that the back posture of the subject is lying on its side when the forward inclination angle is a positive value and less than or equal to the first preset angle or the forward inclination angle is a negative value and greater than or equal to the fourth preset angle, and / or the absolute value of the trunk backward bending angle is less than or equal to the second preset angle, and the absolute value of the lateral inclination angle is greater than or equal to the third preset angle.

[0043] And in the process of determining that the back posture of the subject is a leaning back posture when the forward leaning angle is a negative value and is smaller than the fourth preset angle, it can be determined that the back posture of the subject is a leaning back posture when the absolute value of the trunk backward bending angle is greater than the second preset angle, and / or the absolute value of the lateral tilt angle is smaller than the third preset angle, and the forward leaning angle is a negative value and is smaller than the fourth preset angle.

[0044] It can be seen that in the embodiment of the present application, if the forward tilt angle is a positive value and the larger the angle is, the larger the forward tilt angle of the human body's back is. If the forward tilt angle is a negative value and the smaller the angle is, the larger the backward leaning angle of the human body's back is. If the absolute value of the backward bending angle of the torso is larger, it can be reflected that the human body's back is more curved. If the absolute value of the lateral tilt angle is larger, it can be reflected that the human body's back is more inclined toward the side of the human body. In this way, when the degree to which the human body's back tilts toward the side of the human body is smaller, the degree to which the human body's back tilts forward is larger and the degree of curvature is larger, it can be indicated that the user's back posture is a hunchback posture. When the degree to which the human body's back tilts toward the side of the human body is larger, it can be indicated that the user's back posture is a side-lying posture. The more curved the human body's back is, the greater the degree of backward leaning is, and then the user's back posture can be determined to be a backward leaning posture.

[0045] And, in judging the side-lying posture process, the lateral inclination angle can also be combined with the forward tilt angle and / or the trunk backward bending angle for comprehensive judgment. In judging the backward posture process, the forward tilt angle can also be combined with the lateral inclination angle and / or the trunk backward bending angle for comprehensive judgment.

[0046] Therefore, when determining that the back posture of the user is abnormal, the embodiment of the present application can specifically classify the abnormal posture, and can determine that the back posture of the user is a hunchback posture, a side-lying posture, or a backward posture among the abnormal postures, thereby improving the precision of abnormal posture classification.

[0047] In one possible implementation of the first aspect, in the process of determining the posture parameters according to the position data of the human joint points, the first device can generate a three-dimensional human skeleton model corresponding to the photographed object in the current posture according to the position data of the human joint points. Then, the position data corresponding to the target joint points in the three-dimensional human skeleton model are obtained; the target joint points include the back joint points, the hip joint points corresponding to the back, and the head and neck joint points. Finally, the first device can determine the posture parameters according to the position data corresponding to the target joint points.

[0048] It can be seen that the embodiment of the present application can perform three-dimensional skeleton modeling based on the human body joint point position data, and present a three-dimensional skeleton model corresponding to the human body in three-dimensional space. Then, the position data corresponding to the back joint points, the upper and lower hip joint points corresponding to the back, and the head and neck joint points in the human body joint point position data can be used to output posture parameters. In this way, the joint angles corresponding to the human body joint points can be used to evaluate the user's back posture. Furthermore, the accuracy of detecting the user's posture can be improved.

[0049] In a second aspect, an embodiment of the present application provides a posture detection method, which is applied to a second device, and the method includes: the second device receives a posture detection result from a first device, and the first device is associated with the second device. The posture detection result includes abnormal postures of N photographed objects corresponding to back postures, where N is a positive integer.

[0050] The second device can output second information according to the posture detection result; the second information is used to prompt that the back posture of the N subjects is abnormal. Then, the second device can detect the trigger operation for the target subject among the N subjects, and send a trigger instruction to the first device, the trigger instruction is used to instruct the target subject to be reminded of the posture.

[0051] It can be seen that in the embodiment of the present application, when multiple photographed subjects are in abnormal postures, the second device can receive the posture detection result from the first device. Then, according to the posture detection result, the second information is output, and the second information is used to prompt that the back posture of the multiple photographed subjects is an abnormal posture. Then, the second device can detect the trigger operation for the target photographed subject among the multiple photographed subjects, and send a trigger instruction to the first device, and the trigger instruction is used to instruct the target photographed subject to perform a posture reminder.

[0052] That is to say, when multiple subjects are in abnormal postures, other devices can be used to select one or some of the subjects to be reminded of their postures, so as to achieve a precise reminder process for the subjects. The embodiments of the present application can provide users with more humane and richer posture reminder functions through information linkage between devices, thereby improving the user's experience.

[0053] In a third aspect, an embodiment of the present application provides an interactive system, which includes a first device and a second device.

[0054] The first device is configured to acquire a first image, wherein the first image includes at least one photographed object;

[0055] The first device is further configured to perform posture detection on the first image, determine the human body joint point position data of the photographed object, and determine the posture parameters according to the human body joint point position data;

[0056] The first device is further configured to send a posture detection result to the second device when it is determined according to the posture parameter that the back postures of the N photographed subjects are abnormal postures; wherein the posture detection result includes the abnormal postures corresponding to the N photographed subjects;

[0057] The second device is configured to receive the posture detection result and output second information according to the posture detection result; the second information is used to prompt that the back posture of the N photographed subjects is an abnormal posture;

[0058] The second device is further configured to detect a trigger operation for a target photographing object among the N photographing objects, and send a trigger instruction to the first device, where the trigger instruction is used to instruct to perform a posture reminder for the target photographing object;

[0059] The first device is also configured to output first prompt information according to the trigger instruction; the first prompt information is used to prompt the target subject to adjust his back posture.

[0060] In a fourth aspect, an embodiment of the present application provides a display device, the display device comprising: a smart camera and a display screen, the smart camera comprising a processor.

[0061] The processor is configured to acquire a first image, wherein the first image includes at least one photographed object;

[0062] The processor is further configured to perform posture detection on the first image, determine the position data of the human joint points of the photographed object, and determine the posture parameters according to the position data of the human joint points; the posture parameters include the joint angles corresponding to the human joint points; the posture parameters are used to characterize the back posture corresponding to at least one photographed object;

[0063] The processor is further configured to control the display screen to output first information when it is determined according to the posture parameter that the back posture of the photographed subject is abnormal;

[0064] The display screen is configured to output first information, where the first information is used to prompt the subject to adjust his back posture.

[0065] In the fifth aspect, an embodiment of the present application provides an electronic device, which includes: a memory, a camera and a display screen, wherein the camera includes a processor, the memory and the display screen are coupled to the processor, the display screen is used to output first information or second information, and the memory is used to store computer-readable instructions. When the processor reads the computer-readable instructions from the memory, the electronic device executes the posture detection method described in the first aspect or the second aspect above.

[0066] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, which includes a computer program. When the computer program runs on an electronic device, the electronic device executes the posture detection method described in the first or second aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 A schematic diagram of a related technology scenario provided by an embodiment of the present application;

[0068] Figure 2 A schematic diagram of a process for detecting a user's posture provided in an embodiment of the present application;

[0069] Figure 3 A schematic diagram of the hardware structure of a smart screen provided in an embodiment of the present application;

[0070] Figure 4 A schematic diagram of the hardware structure of a smart screen provided in an embodiment of the present application;

[0071] Figure 5 A schematic diagram of an interactive system provided in an embodiment of the present application;

[0072] Figure 6 A flowchart of a posture detection method provided in an embodiment of the present application;

[0073] Figure 7 A schematic diagram of an interface for target detection information provided in an embodiment of the present application;

[0074] Figure 8 A schematic diagram of a process for determining posture parameters provided in an embodiment of the present application;

[0075] Fig. 9 A schematic diagram of calculating the degree of lateral tilt of the entire back of a human body provided in an embodiment of the present application;

[0076] Fig.10 A schematic diagram of an interface for determining an abnormal posture provided in an embodiment of the present application;

[0077] Fig.11 A schematic diagram of a first information interface provided in an embodiment of the present application Figure 1 ;

[0078] Fig.12 A schematic diagram of a first information interface provided in an embodiment of the present application Figure 2 ;

[0079] Fig.13 A schematic diagram of a first information interface provided in an embodiment of the present application Figure 3 ;

[0080] Fig.14 A schematic diagram of a first information interface provided in an embodiment of the present application Figure 4 ;

[0081] Fig.15 A schematic diagram of an interface of a smart screen provided in an embodiment of the present application;

[0082] Fig.16 A schematic diagram of an interface of a smart life application provided in an embodiment of the present application;

[0083] Fig.17 A schematic diagram of a flow chart of interaction between a smart screen and a mobile phone provided in an embodiment of the present application;

[0084] Fig.18 A schematic diagram of an interface of second information provided in an embodiment of the present application;

[0085] Fig.19 A schematic diagram of a posture detection management interface in a mobile phone provided in an embodiment of the present application;

[0086] Fig. 20 A schematic diagram of a posture detection management interface in a smart screen provided in an embodiment of the present application;

[0087] Fig.21 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0088] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. Among them, in the description of the present application, unless otherwise specified, " / " indicates that the objects associated before and after are in an "or" relationship, for example, A / B can represent A or B; "and / or" in the present application is only a kind of association relationship describing the associated objects, indicating that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. And, in the description of the present application, unless otherwise specified, "multiple" refers to two or more than two. "At least one of the following" or its similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple. In addition, in order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish the same or similar items with basically the same functions and effects.

[0089] Those skilled in the art will appreciate that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit the difference. At the same time, in some embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way for easy understanding.

[0090] In addition, the network architecture and business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. A person of ordinary skill in the art can appreciate that with the evolution of the network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are equally applicable to similar technical problems.

[0091] With the rapid popularization of electronic devices such as mobile phones, computers and televisions, people's lives have been brought a lot of convenience and enjoyment. Users spend more and more time watching electronic devices in their daily lives. As a result, problems such as cervical spine problems, tilted head, hunchback and strabismus caused by watching electronic devices with bad postures are becoming increasingly prominent. Especially among teenagers, bad sitting posture, unhealthy use of eyes and excessive use of eyes may affect the development of teenagers' vision. As teenagers spend more time using electronic devices, parents pay more attention to their children's eye use and sitting habits.

[0092] Usually, users will use posture correction products to constrain their posture during the viewing process. For example, teenagers will wear correction products before watching electronic devices so that the correction products can remind and correct their viewing posture. Then, after watching, teenagers need to take off the correction products. In other words, teenagers need to rely on correction products every time they watch electronic devices, and they need to repeatedly wear and take off the products, which has certain usage limitations. This reduces the user experience.

[0093] In addition, the related art also provides another solution for detecting user posture, see Figure 2 In (A), this solution can collect the user's sitting posture image through a device set on a desk lamp, and then input the preprocessed sitting posture image data into the sitting posture classification model to obtain the sitting posture classification result. Compare the current sitting posture classification result with the pre-trained correct sitting posture. If it is not a correct sitting posture, give a reminder. It can be understood that this solution can only simply classify the sitting posture of a single user into a correct sitting posture and an incorrect sitting posture. The classification of sitting posture is low in precision and the flexibility of detecting user posture is low.

[0094] Another method for detecting user gestures is provided in the related art, see Figure 2 In (B), this solution can collect video data through a binocular camera, perform face detection on the video data, and determine whether the face head is rotated left or right. Then determine whether the user's sitting posture is correct. It can be understood that this solution only replaces the body's orientation angle by identifying the left and right orientation angles of the user's head, and cannot reflect the sitting posture of the body. There will be cases of wrong judgment, and the accuracy of judging the user's posture is low.

[0095] From the above content, it can be seen that in the current solutions for detecting user posture, there are problems of low flexibility in detecting user posture and reduced detection accuracy, thereby affecting the user's experience.

[0096] Based on the above content, an embodiment of the present application provides a posture detection method and device, which can determine the human joint point position data corresponding to at least one photographed object in the first image by performing posture detection on the collected first image. Then, the posture parameters corresponding to at least one photographed object can be determined based on the human joint point position data corresponding to at least one photographed object; the posture parameters are used to characterize the back posture corresponding to at least one photographed object. Exemplarily, the posture parameters may include the forward tilt angle, the trunk backward bend angle, and the lateral tilt angle corresponding to the user. Among them, the forward tilt angle is used to characterize the degree to which the user's back tilts forward, the trunk backward bend angle is used to characterize the degree of back bending, and the lateral tilt angle is used to characterize the degree to which the user's back tilts to the side. Then, when it is determined according to the posture parameters that the back posture of the photographed object is an abnormal posture, the first information can be output, and the first information is used to prompt the adjustment of the back posture of the photographed object.

[0097] That is to say, the embodiment of the present application can analyze the human joint point position data from the image, and fit the human skeleton posture according to the human joint point position data, present the three-dimensional skeleton model corresponding to the human body in the three-dimensional space, and can restore the specific orientation and posture of the current user. Among them, the human joint point position data includes the joint point position corresponding to at least one joint. Then, the posture parameters can be obtained according to the corresponding human key point position data in the three-dimensional skeleton model, and when the back posture of the user is determined to be an abnormal posture according to the posture parameters, the first information is output.

[0098] It can be seen that when users such as teenagers use electronic devices, the electronic devices can collect images by themselves, and the images include the teenagers who are using the electronic devices. Then, the electronic devices can detect the posture of the teenagers and provide subsequent posture reminders based on the collected images. There is no need for teenagers to rely on the correction products they wear to perform some tedious operations to constrain their posture. It provides users with a more humane posture detection function and simplifies the user's operation process. This improves the user's experience.

[0099] Moreover, in the embodiment of the present application, joint angles, which are richer than bone information, are used to evaluate the back posture of the user. It is understandable that each bone needs joints to be connected, and joints enable the user to make various movements. The position of the joint points can more accurately characterize the corresponding posture of the user. In other words, the embodiment of the present application can accurately identify the posture of the user and improve the accuracy of detecting the user's posture. At the same time, the embodiment of the present application can also specifically classify abnormal postures, such as hunchback posture, side-lying posture and backward posture. The classification of postures is more refined, which improves the flexibility of posture detection.

[0100] The posture detection method provided in the embodiment of the present application can be applied to electronic devices. In some embodiments, the electronic device may include, for example, a smart screen, a mobile phone, a tablet computer, a personal computer (PC), a personal digital assistant (PDA), a smart watch, a netbook, a wearable device, an augmented reality (AR) device, a virtual reality (VR) device, a vehicle-mounted device, and a smart car, etc. The embodiment of the present application does not impose any special restrictions on the specific form of the electronic device.

[0101] The operating system installed on the electronic device includes but is not limited to Or other operating systems. This application does not limit the specific type of electronic equipment and the type of operating system when an operating system is installed.

[0102] For example, taking the electronic device as a smart screen as an example, Figure 3 (A) in the figure shows a schematic diagram of the structure of the smart screen 100.

[0103] The smart screen 100 may include a processor 110, an external memory interface 120, an internal memory 121, an antenna, a wireless communication module 130, a charging management module 140, a power management module 141, a battery 142, an audio module 150, a speaker 150A, a sensor module 160, an artificial intelligence (AI) camera 170 and a display screen 180. Among them, the sensor module 160 includes an image sensor 160A.

[0104] It is understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the smart screen 100. In other embodiments of the present application, the smart screen 100 may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure can be implemented in hardware, software, or a combination of software and hardware.

[0105] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0106] Among them, the controller can be the nerve center and command center of the smart screen 100. The controller can generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0107] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0108] The wireless communication function of the smart screen 100 can be realized through an antenna, a wireless communication module 130, a modem processor, and a baseband processor.

[0109] The wireless communication module 130 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), infrared technology (IR), etc., which can be applied to the smart screen 100. The wireless communication module 130 can be one or more devices that integrate at least one communication processing module. The wireless communication module 130 receives electromagnetic waves via an antenna, modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 130 can also receive the signal to be sent from the processor 110, modulate the frequency, amplify it, and convert it into electromagnetic waves for radiation through the antenna.

[0110] The smart screen 100 implements the display function through a GPU, a display screen 180, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 180 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0111] The display screen 180 is used to display images, videos, etc. The display screen 180 includes a display panel. In some embodiments, the smart screen 100 may include 1 or N display screens 180, where N is a positive integer greater than 1.

[0112] The smart screen 100 can realize the shooting function through ISP, AI camera 170, video codec, GPU, display screen 180 and application processor.

[0113] The ISP is used to process the data fed back by the AI ​​camera 170. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. The ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP can be set in the AI ​​camera 170.

[0114] The AI ​​camera 170 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the smart screen 100 may include 1 or N AI cameras 170, where N is a positive integer greater than 1.

[0115] Here, the AI ​​camera 170 can be set in the smart screen 100. Of course, it can also be a part of the electronic device. In some implementations, the camera can also be set outside the electronic device and connected to the electronic device by wired or wireless means. For example, the camera and the electronic device can be connected to the electronic device via Bluetooth or a mobile hotspot. The electronic device can control the camera by sending or receiving instructions to the camera.

[0116] The AI ​​camera 170 may also include a camera module, which may be disposed inside the camera. Of course, it may also be disposed at other locations inside the smart screen 100. The camera module includes a lens, a focus motor, a base, a circuit board, and a photosensitive element.

[0117] The base is fixedly connected to one side of the circuit board. The focus motor is located on the side of the base away from the circuit board and is fixedly connected to the periphery of the base. The lens is installed in the middle of the focus motor. The photosensitive element is fixed to the side of the circuit board facing the lens.

[0118] The lens is used to collect light signals reflected by the subject. The focus motor is used to drive the lens to move in a direction parallel to the optical axis. The optical axis refers to a line passing through the center of the lens. In some embodiments, the smart screen 100 can control the focus motor to move the lens to a focus position, thereby completing the focusing process.

[0119] In some embodiments, the focus motor may be a voice coil motor (VCM), a shape memory alloy (SMA) motor, a piezo motor (PM), a stepper motor (STM), etc.

[0120] In some embodiments of the present application, the AI ​​camera 170 may also include an AI processor. In one implementable manner, the AI ​​processor may be used for face and / or human figure recognition, three-dimensional skeletal joint recognition, posture detection, posture reminders, and the like.

[0121] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the smart screen 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, music, video and other files are saved in the external memory card.

[0122] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the smart screen 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area.

[0123] The smart screen 100 can implement audio functions such as music playback and recording through the audio module 150, the speaker 150A, and the application processor.

[0124] In other embodiments, the image sensor 160A can be used to detect objects within the range captured by the camera, and each photosensitive unit corresponds to an image point in the image sensor. The image sensor 160A may include a first color (red green blue, RGB) image sensor, a second color (red yellow blue, RYB) image sensor, a black and white image sensor, and an infrared image sensor, etc., which is not limited in the embodiments of the present application. The image sensor 160A is used to collect images within the range captured by the AI ​​camera 170, wherein the image may include an RGB image, a RYB image, a black and white image, and an infrared image, etc.

[0125] The image sensor 160A may also be disposed in one or more AI cameras 170. The embodiment of the present application does not specifically limit the arrangement of components.

[0126] Typically, the smart screen 100 may also be equipped with a remote controller. The remote controller is used to control the smart screen 100. Figure 3(B) in the figure shows a schematic diagram of the structure of a remote control 190. The remote control 190 may include a plurality of buttons, such as an up button 191, a down button 192, a left button 193, a right button 194, an OK button 195, a power button 196, etc. The buttons on the remote control 190 may be mechanical buttons or touch buttons. The remote control 190 may receive key inputs, generate key signal inputs related to user settings and function control of the smart screen 100, and send corresponding signals to the smart screen 100 to control the smart screen 100. For example, when a user presses the up button 191, the down button 192, the left button 193, the right button 194, the OK button 195 or the power button 196, the button may send a corresponding signal; and send a signal to the smart screen 100 via Bluetooth, infrared, etc. The smart screen 100 receives the signal corresponding to the button through the wireless communication module 130, and may perform corresponding operations according to the signal.

[0127] It is understandable that the remote controller 190 may also include other buttons and components, such as a volume button, a Bluetooth interface, an infrared interface, a battery storage cavity (for installing batteries to power the remote controller), etc. The embodiments of the present application will not be described in detail.

[0128] Figure 4 This is a software structure block diagram of the smart screen 100 of an embodiment of the present application.

[0129] The layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. The system is divided into four layers, from top to bottom: application layer, application framework layer, Runtime (Android runtime) and system libraries and kernel layer.

[0130] The application layer can include a series of application packages.

[0131] like Figure 4 As shown, the application package may include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications.

[0132] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0133] like Figure 4 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0134] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0135] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0136] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0137] The phone manager is used to provide communication functions for smart watches, such as the management of call status (including answering, hanging up, etc.).

[0138] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0139] The notification manager allows applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a reminder sound is emitted, a smart watch vibrates, and an indicator light flashes.

[0140] Runtime includes core libraries and virtual machines. Runtime is responsible for the scheduling and management of the Android system.

[0141] The core library consists of two parts: one is the function that the Java language needs to call, and the other is The core library.

[0142] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0143] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0144] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0145] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0146] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0147] A 2D graphics engine is a drawing engine for 2D drawings.

[0148] The kernel layer is a layer between hardware and software. The kernel layer includes at least a display driver, a camera driver, an audio driver, and a sensor driver. In some embodiments, the camera driver in the kernel layer is used to support image acquisition, and perform gesture detection and gesture reminder based on the acquired image. The display driver is used to support the display of prompt information corresponding to the gesture reminder process.

[0149] The posture detection method provided in the embodiment of the present application can be applied to the interactive system. Figure 5 An interactive system provided in an embodiment of the present application includes a first device 100 (such as a user's smart screen) and a second device 101 (such as a user's mobile phone).

[0150] In some embodiments, a communication connection may be established between the first device 100 and the second device 101. The communication connection may include a wireless communication connection. For example, the wireless communication connection may include a short-range wireless communication connection such as a Bluetooth communication connection, a near-field communication connection, and a wireless local area network communication connection.

[0151] In some other embodiments, the interactive system provided in the embodiments of the present application may further include a third device 102 (a user's smart watch). A communication connection is established between the third device 102 and the first device 100 and the second device 101. The communication connection may be a wireless communication connection.

[0152] Among them, the wireless communication technology for establishing a wireless communication connection includes but is not limited to at least one of the following: wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT) (for example, traditional Bluetooth or bluetooth low energy (BLE) Bluetooth), near field communication (NFC), Zigbee, frequency modulation (FM), infrared (IR), etc.

[0153] For ease of description, the first device 100 is taken as a smart screen as an example. The smart screen is different from traditional TV products because it has the function of wireless signal transmission and reception, and will assume more roles in the family. The smart screen is not only a home audio-visual entertainment center, but also an information sharing center, a control management center, and a multi-device interaction center. The smart screen can establish connections with a variety of devices to achieve information interaction.

[0154] Exemplarily, the smart screen 100 can exchange information with the mobile phone 101 and the smart watch 102. The smart screen 100 can send the posture detection result generated after posture detection to the mobile phone 101 or the smart watch 102, and the mobile phone 101 and / or the smart watch 102 can control the smart screen 100 to output the prompt information corresponding to the posture detection result according to the posture detection result. In other words, while teenagers are watching the smart screen 100, parents can use the mobile phone 101 or the smart watch 102 to control the posture reminder of one or some teenagers in the posture detection results. Then, parents can control the smart screen 100 to perform posture reminders by operating the mobile phone 101 or the smart watch 102 to prompt one or some teenagers to adjust their current posture.

[0155] The interactive system provided in the embodiment of the present application may also include other electronic devices in addition to the first device 100, the second device 101 and the third device 102. The interactive system provided in the embodiment of the present application includes but is not limited to the interaction between three devices, and may also be an interaction between one device and multiple devices. Those skilled in the art can determine the type and quantity of electronic devices according to actual needs, and these designs do not exceed the protection scope of the embodiments of the present application.

[0156] The following embodiments will be combined with the accompanying drawings, taking the device as an example Figure 3 Taking the smart screen with the structure shown in the figure as an example, the posture detection method provided by the embodiment of the present application is described. Figure 6 In (A), the method may include:

[0157] S601, the smart screen obtains a first image.

[0158] In some embodiments of the present application, an AI camera with shooting and control functions is installed in the smart screen, and the AI ​​camera can perform tasks such as taking pictures and videos, posture detection, and posture reminder. It should be noted that the AI ​​camera can enter the working state after detecting the start operation input by the user.

[0159] In one possible implementation, the startup operation input by the user in this solution may include a click operation by the user to turn on the smart screen; for example, a click operation by the user on the corresponding startup control of the smart screen. After that, the AI ​​camera enters the startup state and obtains the user's posture in front of the smart screen by collecting image data. Alternatively, the user clicks on the target application icon. For example, the user clicks on the video application icon.

[0160] After that, the AI ​​camera enters the startup state and acquires the user's posture in front of the smart screen by collecting image data. It is understandable that when a user clicks on the target application icon, it indicates that the user will use the smart screen within a certain period of time in the future. In this way, the AI ​​camera can be turned on and collect image data, which is convenient for subsequent posture detection tasks.

[0161] In some embodiments of the present application, the AI ​​camera may include an image sensor, and the image sensor may collect image data in front of the smart screen. The image data collected by the image sensor includes N frames of images, where N is a positive integer greater than or equal to 1. Exemplarily, the image sensor in the AI ​​camera may collect a first image, and the first image includes at least one photographed object, so that subsequent posture detection tasks can be performed based on the first image.

[0162] S602: The smart screen performs posture detection on the first image to determine posture parameters of the subject.

[0163] In some embodiments of the present application, the smart screen can perform posture detection on the first image, determine the human joint point position data corresponding to at least one photographed object, and determine the posture parameters based on the human joint point position data. Among them, the posture parameters are used to characterize the back posture corresponding to at least one photographed object. In this way, the subsequent smart screen can determine whether the back posture of at least one photographed object is a normal posture or an abnormal posture based on the back posture of at least one photographed object in the first image.

[0164] In some embodiments of the present application, the AI ​​processor in the AI ​​camera may perform target detection on the first image in advance to obtain at least one first area in the first image that includes target features. The first area represents the area where the user is located in the first image. Different first areas correspond to different shooting objects. It should be noted that the target features may include human features or facial features.

[0165] Specifically, the AI ​​processor can perform target detection on the first image to obtain target detection information of at least one photographed object. The target detection information is used to indicate the photographed object in the captured image, i.e., the first image, and the position of the photographed object in the first image. The at least one photographed object involved in the embodiment of the present application may include at least one user viewing the smart screen.

[0166] In an achievable manner, the target detection information may include one or more of a target frame of the photographed object, a size of an area corresponding to the target frame, and coordinates corresponding to the target frame. Next, the area where the target frame corresponding to the photographed object in the target detection information is located is determined as the first area.

[0167] It is understandable that the photographed object may be one or more users. The target detection information is used to characterize the relevant information of the photographed object within the detection range of the image sensor, and can be understood as static information of the photographed object.

[0168] Exemplarily, when the subject of the photograph is a user, the subject of the photograph may include the first user. Figure 7 In (A), the first target detection information may include one or more of a first target frame A corresponding to the first user, an area size or coordinates of the first target frame A.

[0169] For example, when the subject of the photo is multiple users, see Figure 7 In (B), the photographed object may include a first user and a second user. The first target detection information may include a first target frame A corresponding to the first user, a region size or coordinates of the first target frame A, or one or more thereof; the second target detection information may include a first target frame B corresponding to the second user, a region size or coordinates of the first target frame B, or one or more thereof.

[0170] In one feasible manner, the AI ​​processor can input the first image into the target model for detection, and after obtaining the target frame containing human-shaped features, determine the area in the target frame as the first area. In order to take into account that the human body in the first image has different geometric features, the size of the human body in the image is smaller when it is farther away from the smart screen. Therefore, if all first areas are assigned rectangular areas of uniform size, it is easy to cause information loss or redundancy. Therefore, the target frame in the present application can be a rectangle of appropriate size, and the first area can be a rectangular area of ​​different sizes.

[0171] It is understandable that the first area refers to an image area selected from the first image, which is the focus of image analysis. The embodiment of the present application can reduce image processing time and improve image processing accuracy by delineating an area including target features as a prerequisite for further processing of the image. It should be noted that the first area can be a rectangular area, a circular area or an area of ​​any shape. In the embodiment of the present application, it can be a rectangular area. The first area including the target features means that the first area includes a human-shaped area.

[0172] In a feasible manner, for the extraction of the first region, convolutional neural networks (CNN), region proposal network (RPN), regions with cnn features (RCNN), fast RCNN network and residual network, etc., or a combination of multiple networks can be used to extract the ROI area and the corresponding region type. It should be noted that the target model provided in the embodiment of the present application may include one or more of the YOLO algorithm, SSD algorithm and DenseBox algorithm. These algorithms all have the advantages of fast speed and high precision. The embodiment of the present application does not limit the specific form of the target model.

[0173] Of course, the embodiment of the present application will also train the target model. When training the target model, a large number of first images can be used as training samples for training, so that the target model can learn the ability to identify the first area including the target feature in the first image.

[0174] It should be noted that the first image collected may not include the user. In this way, after the AI ​​processor performs target detection on the first image, if it does not obtain at least one first area including target features, it will execute the process of collecting images again.

[0175] In some embodiments of the present application, after obtaining at least one first area in the first image, the AI ​​processor performs posture detection on the at least one first area to determine the posture parameters of the photographed object in the at least one first area.

[0176] Specifically, the AI ​​processor performs posture detection on at least one first area to determine the human joint point position data of the photographed object in the at least one first area. Then, the AI ​​processor determines the posture parameters of the photographed object based on the human joint point position data.

[0177] In one achievable manner, the AI ​​processor may input at least one first region in the first image into a human motion capture model, and output the corresponding human joint point position data in the first region. The human joint point position data includes the joint point position corresponding to at least one joint. That is to say, the embodiment of the present application may extract the human joint point position data in at least one first region through the human motion capture model, and the human joint point position data is used to characterize the position of the joint in the user's body in the first region.

[0178] It should be noted that, in the process of extracting human joint point position data through the human motion capture model, the embodiment of the present application can predetermine the human skeleton position and the joint angle between the bones in at least one first region. Then, based on the human skeleton position, the joint angle between the bones and the topological relationship between the bones in at least one first region, the human joint point position data in at least one first region is output.

[0179] In addition, the AI ​​processor can generate a human 3D skeleton model corresponding to the user in the current posture based on the position data of the human joints. Then, the AI ​​processor can obtain the position data corresponding to the target joints in the human 3D skeleton model. Among them, the target joints include the back joints, the hip joints corresponding to the back, and the head and neck joints. Then, the AI ​​processor can determine the user's posture parameters based on the position data corresponding to the target joints. Among them, the posture parameters include the user's corresponding forward tilt angle, trunk backward bend angle, and lateral tilt angle.

[0180] In other words, the AI ​​processor can fit the human skeleton posture according to the human joint position data, and present the corresponding three-dimensional skeleton model of the human body in three-dimensional space. It can then restore the current orientation and posture of the human body. And the number of joint angles corresponding to the joints is as many as dozens, which can cover the relevant common joints in the human body.

[0181] Then, the AI ​​processor can input the human joint point position data into the back model, and output the posture parameters corresponding to at least one photographed object in the first area. Specifically, the AI ​​processor can select the position data corresponding to the target joint point in the human joint point position data (such as the position data corresponding to the joint points corresponding to the back joints, the upper and lower hip joints near the back, and the head and neck joints), and input it into the back model, thereby obtaining the posture parameters corresponding to the user.

[0182] The following is combined with Figure 8 The posture parameters are elaborated in detail.

[0183] Figure 8 (A) in the figure is a schematic diagram of the interface for collecting images on the smart screen. It is understandable that the user usually faces the smart screen when using the smart screen. Therefore, the images collected by the AI ​​processor in the smart screen are all images of the user's frontal posture. To facilitate understanding of the process of determining posture parameters provided in the embodiment of the present application, a schematic diagram of the side of the human body is used as an example for description. Figure 8 Specifically, the AI ​​processor can perform three-dimensional skeleton modeling based on the acquired first image, and present a three-dimensional skeleton model corresponding to the human body in the three-dimensional space. Figure 8 As shown in (C) in the figure. That is, the AI ​​processor can perform three-dimensional skeleton modeling based on at least one first area in the first image, input at least one first area in the first image into the human motion capture model, and output the corresponding human joint point position data in the first area. Then, the AI ​​processor inputs the position data corresponding to the target joint point into the back model, and outputs the posture parameters corresponding to the user in the first area.

[0184] In one possible implementation, see Figure 8 In (D), during the process of the AI ​​processor outputting posture parameters, the back model receives input human joint position data, and can use the position data corresponding to the back joints, the upper and lower hip joints close to the back, and the head and neck joints in the human joint position data.

[0185] The position data corresponding to the multiple joint points used above may include position points such as point A, point B, point C, point D and point E in the figure. Next, the AI ​​processor selects point A and point E, and calculates the first angle between the first direction and the second direction corresponding to point A and point E in the coordinate system of the side view. The first angle can reflect the current forward tilt of the back of the human body as a whole. In other words, the first angle is the forward tilt angle in the posture parameter. Among them, the clockwise rotation angle of the direction vector is positive, and the counterclockwise rotation angle is negative.

[0186] The AI ​​processor can also fit the position data corresponding to the above-mentioned multiple joint points (such as point A, point B, point C, point D and point E), use the five position points to fit a circle, and calculate the second angle between the center of the circle and the third direction corresponding to point A and the center of the circle and the fourth direction corresponding to point E. The second angle can reflect the curvature of the current human back. In other words, the second angle is the torso back bend angle in the posture parameters. The more curved the human back is, the greater the torso back bend angle. It should be noted that the average error of the arc of the fitted circle to the five key points is the smallest.

[0187] See also Fig. 9 , the AI ​​processor can also calculate the degree of tilt of the entire back of the human body from the frontal perspective. Fig. 9 Then, the AI ​​processor selects point A and point E, and calculates the third angle between the sixth direction and the fifth direction corresponding to point A and point E in the coordinate system of the front view, and the third angle can reflect the current tilt degree of the back of the whole human body. In other words, the third angle is the lateral tilt angle in the posture parameter.

[0188] It should be noted that the embodiments of the present application do not limit the specific forms of the human motion capture model and the back model. Figure 8 and Fig. 9 The position data corresponding to the multiple joints in the figure are used as examples to explain the lateral tilt angle, the forward tilt angle and the backward bending angle of the trunk. It can be understood that when the positions corresponding to the multiple joints are different, the first direction, the second direction, the third direction, the fourth direction, the fifth direction and the sixth direction corresponding to the lateral tilt angle, the forward tilt angle and the backward bending angle of the trunk can also be different.

[0189] Of course, the embodiment of the present application will also train the human motion capture model and the back model. When training the human motion capture model, a large number of first images can be used as training samples for training, so that the human motion capture model can learn the ability to recognize the position data of the human joint points in the first image.

[0190] Similarly, when training the back model, a large amount of human joint point position data can be used as training samples and key points corresponding to human joints for training, so that the back model can learn the ability to recognize the posture parameters in the first image. The above embodiment of the present application only uses the key points corresponding to 5 joints for exemplary description, and does not specifically limit the specific positions and key numbers of the joint key points used.

[0191] In some embodiments of the present application, after determining the posture parameters of the photographed object, the AI ​​processor can obtain the posture detection result of the photographed object according to the posture parameters of the photographed object. Among them, the posture detection result may include normal posture and abnormal posture, and the abnormal posture includes hunchback posture, side-lying posture and leaning back posture. It should be noted that in a scenario where multiple people use the smart screen, the posture detection result may also include user information corresponding to the photographed object. User information may include user number and user name, etc.

[0192] Specifically, after using the back model to determine the posture parameters corresponding to the photographed object in the first area, the AI ​​processor can continue to use the back model to generate a posture detection result of the photographed object based on the posture parameters of the photographed object.

[0193] In one achievable way, continue to see Figure 8 In (B), the AI ​​processor may acquire the first image. Then, the AI ​​processor may perform three-dimensional bone modeling based on the first region in the first image. Figure 8 As shown in (C) in the figure, the AI ​​processor can then determine the posture parameters corresponding to the user in the first area after three-dimensional skeleton modeling, and determine whether the back posture of the photographed subject is abnormal according to the posture parameters.

[0194] Specifically, the AI ​​processor can determine that the subject's back posture is a hunchback posture when the absolute value of the lateral tilt angle in the subject's posture parameters is less than a third preset angle, the forward tilt angle is a positive value and greater than the first preset angle, and the absolute value of the trunk backward bend angle is greater than the second preset angle.

[0195] It can be understood that if the forward tilt angle is a positive value and the angle is larger, it can reflect that the forward tilt angle of the human back is larger. If the absolute value corresponding to the backward bending angle of the torso is larger, it can reflect that the back of the human body is more curved. If the absolute value corresponding to the lateral tilt angle is larger, it can reflect that the back of the human body is more inclined to the side of the human body.

[0196] In this way, when the absolute value of the lateral tilt angle is less than the third preset angle, the forward tilt angle is positive and greater than the first preset angle, and the absolute value of the trunk backward bending angle is greater than the second preset angle, it indicates that the back of the human body is less inclined to the side of the human body, but the back of the human body is more inclined forward and has a greater degree of curvature. Therefore, it can be determined that the back posture of the photographed subject is a hunchback posture.

[0197] In another possible implementation, see Fig.10 In (A), the AI ​​processor may acquire a first image. Then, the AI ​​processor may perform three-dimensional bone modeling based on at least one first region in the first image. Fig.10Then, the AI ​​processor can determine the posture parameters corresponding to the user in the first area after three-dimensional skeleton modeling.

[0198] And when the absolute value of the lateral inclination angle in the posture parameters of the photographed object is greater than or equal to the third preset angle, it is determined that the back posture of the photographed object is a side-lying posture.

[0199] It is understandable that the AI ​​processor can only determine the lateral tilt angle. When the absolute value of the lateral tilt angle is greater than or equal to the third preset angle, it indicates that the back of the human body is tilted to a greater extent toward the side of the human body. Further, it can be determined that the back posture of the subject is lying on its side.

[0200] In another achievable manner, the AI ​​processor may also determine that the back posture of the subject is a side-lying posture when, in the posture parameters of the subject, the forward leaning angle is a positive value and less than or equal to a first preset angle, or the forward leaning angle is a negative value and greater than or equal to a fourth preset angle, and / or the absolute value of the trunk backward bending angle is less than or equal to the second preset angle, and the absolute value of the lateral tilt angle is greater than or equal to the third preset angle.

[0201] It is understandable that the AI ​​processor can not only determine the lateral tilt angle, but also make a comprehensive determination of the lateral tilt angle in combination with the forward tilt angle and / or the backward bending angle of the trunk.

[0202] For example, when the absolute value of the lateral tilt angle is greater than or equal to the third preset angle and the forward tilt angle is a positive value and less than or equal to the first preset angle or the forward tilt angle is a negative value and greater than or equal to the fourth preset angle, it indicates that the back of the human body is tilted to a greater extent to the side of the human body, the back of the human body is tilted forward to a lesser extent, and the back of the human body is tilted backward to a lesser extent. Thus, it can be determined that the back posture of the photographed subject is in a side-lying posture.

[0203] For another example, when the absolute value of the lateral tilt angle is greater than or equal to the third preset angle and the absolute value of the trunk backbend angle is less than or equal to the second preset angle, it indicates that the back of the human body is tilted to a greater extent to the side of the human body and the back of the human body is bent to a lesser extent. Therefore, it can be determined that the back posture of the photographed subject is a side-lying posture.

[0204] For another example, when the forward tilt angle is a positive value and is less than or equal to the first preset angle or the forward tilt angle is a negative value and is greater than or equal to the fourth preset angle, and / or the absolute value of the trunk backward bending angle is less than or equal to the second preset angle, and the absolute value of the lateral tilt angle is greater than or equal to the third preset angle, it indicates that the back of the human body is tilted to a greater extent to the side of the human body, the back of the human body is tilted forward or backward to a lesser extent, and the back of the human body is bent to a lesser extent. Further, it can be determined that the back posture of the photographed subject is a side-lying posture.

[0205] It should be noted that the lateral inclination angle proposed in the embodiment of the present application can also be any angle. When the lateral inclination angle is a positive value, if the absolute value of the lateral inclination angle is greater than or equal to the third preset angle, it can be determined that the back posture of the subject is lying on the left side. When the lateral inclination angle is a negative value, if the absolute value of the lateral inclination angle is greater than or equal to the third preset angle, it can be determined that the back posture of the subject is lying on the right side. The embodiment of the present application can also make a further specific determination of the lying posture, but does not limit the specific determination form of the lying posture on the right side and the lying posture on the left side.

[0206] In another possible implementation, see Fig.10 In (C), the AI ​​processor may acquire the first image. Then, the AI ​​processor may perform three-dimensional bone modeling based on the first region in the first image. Fig.10 Then, the AI ​​processor can determine the posture parameters corresponding to the user in the first area after three-dimensional skeleton modeling.

[0207] And when the forward leaning angle in the posture parameter of the photographed object is a negative value and is smaller than a fourth preset angle, it is determined that the back posture of the photographed object is a backward leaning posture. The fourth preset angle is smaller than the first preset angle.

[0208] It is understandable that the AI ​​processor can only determine the forward tilt angle. For example, the fourth preset angle can be a negative number. When the forward tilt angle is a negative value and is less than the fourth preset angle, it indicates that the back of the human body is leaning back to a greater extent. Therefore, it can be determined that the back posture of the subject is a backward posture.

[0209] In another achievable manner, the AI ​​processor may also determine that the subject's back posture is a backward posture when the absolute value of the trunk backward bending angle in the subject's posture parameters is greater than a second preset angle and / or the absolute value of the lateral tilt angle is less than a third preset angle, and the forward tilt angle is a negative value and less than a fourth preset angle.

[0210] It is understandable that the AI ​​processor can not only determine the forward leaning angle, but also make a comprehensive determination based on the forward leaning angle combined with the trunk backward bending angle and / or the lateral tilt angle.

[0211] For example, when the forward tilt angle is a negative value and is less than the fourth preset angle and the absolute value of the trunk backward bending angle is greater than the second preset angle, it indicates that the back of the human body is leaning back to a greater extent and the back of the human body is bent to a greater extent. Therefore, it can be determined that the back posture of the subject is a backward posture.

[0212] For another example, when the forward tilt angle is a negative value and is less than the fourth preset angle and the absolute value of the lateral tilt angle is less than the third preset angle, it indicates that the back of the human body is leaning back to a greater extent and the back of the human body is tilted to the side of the human body to a lesser extent. Thus, it can be determined that the back posture of the photographed subject is a backward posture.

[0213] For another example, when the forward tilt angle is a negative value and is less than the fourth preset angle, the absolute value of the trunk backward bending angle is greater than the second preset angle, and the absolute value of the lateral tilt angle is less than the third preset angle, it indicates that the back of the human body is leaning back to a greater extent, the back of the human body is bent to a greater extent, and the back of the human body is tilted to a lesser extent to the side of the human body. Therefore, it can be determined that the back posture of the photographed subject is a backward posture.

[0214] In some embodiments of the present application, the posture parameters may also include the direction of the human body. After determining the posture parameters of the subject, the AI ​​processor can obtain the posture detection result of the subject according to the posture parameters when the human body is facing the smart screen in the posture parameters of the subject. In other words, when the user's body is facing the smart screen, it indicates that the user is watching the smart screen. For example, user A sits on the sofa in front of the smart screen, and the body is facing away from the smart screen. This indicates that user A is not watching the smart screen. Therefore, there is no need to perform posture detection on user A.

[0215] It should be noted that when training the back model, a large number of preset angles can be used as training samples for training, so that the back model can learn the ability to determine the target posture. At the same time, a large number of user data corresponding to photographed objects in abnormal postures can be used as training samples for training, so that the back model can learn the ability to determine the target posture.

[0216] The embodiment of the present application only exemplifies the forward tilt angle, the trunk backward bending angle and the lateral tilt angle, and does not specifically limit the specific implementation form of the posture parameter. Also, the specific values ​​of the first preset angle, the second preset angle and the third preset angle used and the number of preset angles are not limited.

[0217] Importantly, when multiple users use the smart screen, the AI ​​processor can perform posture detection on each first area in the first image to obtain the posture parameters corresponding to at least one photographed object in each first area. Then, the AI ​​processor determines the corresponding target posture based on the posture parameters corresponding to at least one photographed object in each first area.

[0218] It can be seen that the embodiment of the present application can perform posture detection on the collected images in the scenario where a single person or multiple people use the smart screen, and determine whether the user's back posture is normal or abnormal. In other words, the embodiment of the present application can perform real-time modeling and supervision of the user's posture in the scene through a contactless solution, and the posture detection process does not require manual participation by the user. It can provide users with a more humane posture detection function and simplify the user's operation. At the same time, when it is determined that the user's back posture is an abnormal posture, the abnormal posture can also be specifically classified, and the user's back posture can be determined as a hunchback posture, a side-lying posture, or a leaning back posture among the abnormal postures. The refinement of the abnormal posture classification is improved.

[0219] S603: When the smart screen determines, based on the posture parameters, that the back posture of the subject is abnormal, it outputs the first information.

[0220] In some embodiments of the present application, when the AI ​​processor determines that the back posture of the photographed subject is abnormal according to the posture parameter, it can output first information, and the first information is used to prompt the photographed subject to adjust the back posture. The first information may include one or more of pop-up prompt information, text prompt information, voice prompt information, or video prompt information.

[0221] In a scenario where a single person uses a smart screen, in one feasible way, see Fig.11 In (A), the AI ​​processor can pause the current playing content and display the first information on the current playing interface. The first information can be implemented as the text "The posture is not standard, please adjust". Of course, the AI ​​processor can also control the speaker to play the first information. The first information can be implemented as a prompt tone such as "The posture is not standard, please adjust" to remind the user to adjust their posture.

[0222] In another possible implementation, see Fig.11 In (B), the AI ​​processor may pause the currently played content and display the captured first image. Next, the first information may be displayed in the first image, and the first information may be implemented as a text “Posture is not standard, please adjust”.

[0223] In another possible implementation, see Fig.12Based on the above process of determining the target posture, if the user's target posture is an abnormal posture, the AI ​​processor can more precisely determine that the user's target posture is a hunchback posture, a side-lying posture, or a backward leaning posture. The AI ​​processor can output the first information, and the first information can be implemented as a text "You can't lean back, sit up straight before you can continue watching." Similarly, the AI ​​processor can also control the speaker to send out the first information, and the first information can be implemented as a prompt sound such as "You can't lean back, sit up straight before you can continue watching", to remind the user to adjust his or her posture.

[0224] In some embodiments of the present application, the AI ​​processor may acquire a first image during the playback of a target media resource. Then, when it is determined according to the posture parameter that the back posture of the photographed subject is abnormal, the AI ​​processor may output first information and pause the playback of the target media resource. In other words, the AI ​​processor may pause the currently playing content and output the first information on the current playback interface.

[0225] It is understandable that when users use the smart screen for a long time, they usually watch some videos, play some games, check some files, etc. The AI ​​processor can detect the user's operation and play the first video. Then, after determining that the back posture of the subject is abnormal, the AI ​​processor controls the first video to pause and outputs the first information in the playback interface corresponding to the first video. On the contrary, if the AI ​​processor determines that the back posture of all subjects is normal, the first video will not be paused.

[0226] That is to say, the AI ​​processor will pause the currently playing video when outputting the first information. On the user side, when the user perceives the first information, the currently watched video will be paused, so as to avoid affecting the viewing of the video while the user is viewing the first information.

[0227] In some embodiments of the present application, the AI ​​processor may also acquire a second image. Then, when the back posture of all the subjects is determined to be normal according to the posture parameters obtained from the second image, the target media resource is resumed to be played. That is to say, after the user adjusts his abnormal posture to a normal posture according to the first information, the smart screen resumes playing the currently played content when it determines that the back posture of all the subjects is normal according to the collected image.

[0228] That is to say, after the user perceives the posture reminder of the smart screen, he can adjust the current abnormal posture to a normal posture, and then the first video can continue to play automatically.

[0229] In some embodiments of the present application, a posture reminder may also be provided to a specific user. The above-mentioned first information may include first prompt information. The AI ​​processor may output first prompt information when it is determined according to preset conditions that there is a target photographing object among N photographing objects; the first prompt information is used to prompt the target photographing object to adjust its back posture, where N is a positive integer.

[0230] For example, in a scenario where it is determined that the back postures of multiple people are abnormal, the administrator user can pre-set the users who need posture reminders. For example, a parent can pre-set that posture reminders are only required for teenagers. In this way, when the AI ​​processor determines that there is a target shooting subject such as a teenager among multiple shooting subjects according to preset conditions, it can output a first prompt information; the first prompt information is used to prompt the teenager to adjust his back posture.

[0231] In an embodiment of the present application, a dynamic image of a normal posture can be generated by using the three-dimensional skeletal model corresponding to the current user modeled in real time, and the user can be guided to complete posture adjustment according to the dynamic image to ensure that the user has a smooth and clear viewing experience.

[0232] Specifically, the AI ​​processor can obtain a target three-dimensional skeleton model in the process of outputting the first prompt information, and the target three-dimensional skeleton model is a human three-dimensional skeleton model corresponding to a normal posture. Then, the AI ​​processor can generate a dynamic image based on the human three-dimensional skeleton model corresponding to the target photographed object and the target three-dimensional skeleton model. The dynamic image presents the process of the human three-dimensional skeleton model corresponding to the target photographed object changing to the target three-dimensional skeleton model. Then, the first prompt information is output, and the first prompt information includes the dynamic image.

[0233] In addition, the first prompt information also includes voice prompt information, and the voice prompt information is used to prompt the target subject to follow the dynamic image to adjust the back posture to a normal posture.

[0234] Exemplarily, the AI ​​processor can pause the currently playing content and output the first prompt information on the current playing interface. The first prompt information includes video prompt information and voice prompt information. The dynamic image in the video prompt information presents the process of the human body three-dimensional skeleton model changing from an abnormal posture to a normal posture. In addition, the first prompt information may also include a prompt tone such as "Please adjust your posture according to the video." In this way, the user can not only adjust the posture through the video, but also perceive the posture reminder through the voice prompt tone.

[0235] In one possible implementation, see Fig.13In (A), the AI ​​processor can obtain the human body 3D skeleton model corresponding to the hunchback posture and the target 3D skeleton model corresponding to the normal posture. Then, the AI ​​processor can generate corresponding video prompt information. It is understandable that the video prompt information can show the user how to adjust from the hunchback posture to the normal posture, so that the user can adjust his own posture through the video prompt information.

[0236] In another possible implementation, see Fig.13 In (B), the AI ​​processor can obtain the human 3D skeleton model corresponding to the side-lying posture and the target 3D skeleton model corresponding to the normal posture. Then, the AI ​​processor can output video prompt information based on the human 3D skeleton model corresponding to the side-lying posture and the target 3D skeleton model. It is understandable that the video prompt information can show the user how to adjust from the side-lying posture to the normal posture, so that the user can adjust the current posture later.

[0237] In another possible implementation, see Fig.13 In (C), the AI ​​processor can obtain the human 3D skeleton model corresponding to the leaning back posture and the target 3D skeleton model corresponding to the normal posture. Then, the AI ​​processor can output video prompt information based on the human 3D skeleton model corresponding to the leaning back posture and the target 3D skeleton model. It is understandable that the video prompt information can show the user how to adjust from the leaning back posture to the normal posture, so that the user can adjust the current posture later.

[0238] In the scenario where multiple people use the smart screen, the postures corresponding to different users may be inconsistent. When each user is in an abnormal posture, the embodiment of the present application can provide a corresponding posture reminder for each user.

[0239] Specifically, the AI ​​processor may output at least one first information, each first information corresponding to a different user.

[0240] For example, when user A and user B are watching the smart screen, the AI ​​processor can output the first information corresponding to user A and the first information corresponding to user B after determining that the back postures of user A and user B are both abnormal. The first information corresponding to user A can be implemented as the text "Don't hunch your back, sit up straight before you can continue watching." The first information corresponding to user B can be implemented as the text "Don't lie on your side, sit up straight before you can continue watching."

[0241] As another example, after determining that the back postures of user A and user B are both abnormal, the AI ​​processor may output the first prompt information corresponding to user A and the first information corresponding to user B. The first information corresponding to user A may show user A how to adjust from a hunchback posture to a normal posture. The first information corresponding to user B may show user B how to adjust from a side-lying posture to a normal posture.

[0242] It should be noted that the above embodiment of the present application only takes the first information as an example, and does not limit the specific implementation form of the user posture reminder.

[0243] For further example, see Fig.14 In (A), user A and user B are watching the smart screen. After determining that the back posture of user A is abnormal, the AI ​​processor can output the first information corresponding to user A. The first information corresponding to user A can be implemented as the text "Don't hunch your back, sit up straight so you can continue watching."

[0244] For further example, see Fig.14 In (B), after determining that the back postures of user A are all abnormal, the AI ​​processor can output the first information corresponding to user A. The first information corresponding to user A can show user A how to adjust from a hunchback posture to a normal posture.

[0245] In some embodiments of the present application, the first image may be a multi-frame image, and the AI ​​processor may perform posture detection on each frame image based on the collected multi-frame images to determine the posture parameters of at least one photographed object in each frame image.

[0246] Then, the AI ​​processor outputs the first information when it is determined that the back posture of the subject in each frame is abnormal according to the posture parameters of at least one subject in each frame. In other words, the embodiment of the present application can determine that the user is watching a movie in front of the smart screen and maintains an abnormal sitting posture for a period of time, and then issue a posture reminder. In addition, it avoids the situation where the user adjusts from an abnormal posture to a correct posture in a short period of time, resulting in frequent posture reminders.

[0247] In some embodiments of the present application, the AI ​​processor may also capture the first image within a first preset duration. The first preset duration may be 6 seconds, 8 seconds, 12 seconds, and so on. It is understandable that the AI ​​processor may perform the posture detection process within a certain period to avoid the situation where the power consumption of the device is increased due to frequent and multiple executions of the posture detection process. The embodiments of the present application do not limit the specific value of the first preset duration.

[0248] In summary, the embodiment of the present application can perform posture detection on users using smart screens, and when it is determined that the user's back posture is abnormal according to the posture detection result, the current playback screen is paused and a posture reminder is given. It can constrain the user to adjust the current abnormal posture to a normal posture, which has a strong constraint effect.

[0249] Moreover, in the process of posture reminder, the embodiment of the present application can remind the user not only through interface pop-up window, voice and other methods, but also through video, which can show the process of the abnormal posture of the current user changing to normal posture. In other words, it can not only make the user aware that he is in an abnormal posture, but also guide the user to complete the correct posture change.

[0250] Then, users can make corresponding standard improvement actions according to the prompt information, so that users can understand and make correct postures more quickly. It improves the interactivity between users and devices, ensures that users maintain a healthy and standard sitting posture, develop good habits, and alleviate vision and spinal problems, thereby improving the user experience.

[0251] In the scenario where multiple people use the smart screen, the embodiment of the present application can support posture detection for all users. At the same time, the embodiment of the present application can also support posture detection only for pre-set users.

[0252] For example, gesture detection is performed on users other than the remote controller holder.

[0253] Another example is posture detection for teenagers or children.

[0254] For another example, posture detection is performed on a user close to the center of the image.

[0255] That is to say, the administrator user can pre-set the users who need posture detection. For example, the administrator user can be a parent, and the parent can set posture detection and posture reminders only for teenagers or children. After the administrator user completes the setting, the smart screen can only traverse the set users who need posture detection in sequence. If there is no user in an abnormal posture, the control display screen continues to play the video selected by the user. If there is a user in an abnormal posture, the first information is output.

[0256] In some implementations of this application, see Figure 6 In (B), the AI ​​processor may also determine J second areas in the at least one first area based on a preset rule, where J is a positive integer, during the process of performing posture detection on the at least one first area. Then, the AI ​​processor may perform posture detection on the J second areas to determine the posture parameters of the photographed object in the second areas.

[0257] The preset rule may include determining areas outside the area where the photographed subject holding the preset item is located in the at least one first area as J second areas.

[0258] Alternatively, the preset rule may further include determining, in at least one first area, areas where photographed subjects whose age values ​​are less than or equal to the age threshold are located as J second areas.

[0259] Alternatively, the preset rule may further include determining, in the at least one first area, an area where the photographed object closest to the image center of the first image is located as the J second areas.

[0260] Specifically, the preset object may be a remote control, and the embodiment of the present application may perform posture detection on other users other than the remote control holder. For example, the AI ​​processor may filter out a target area including the remote control in at least one first area, and then determine the area other than the target area in at least one first area as at least one second area. That is, in this solution, the remote control holder may be set as a parent, and posture detection may not be performed on the remote control holder.

[0261] Alternatively, the embodiment of the present application may perform posture detection only for teenagers or children. The AI ​​processor may input at least one first area into the AI ​​model, and the AI ​​model outputs an age value corresponding to at least one first area. Then, the AI ​​processor may determine the area in at least one first area where the age value is a teenager or a child as at least one second area. In some other ways, the AI ​​processor may also determine at least one second area based on the age value corresponding to the photographed object in the target detection information corresponding to the first area.

[0262] Alternatively, in the embodiment of the present application, in a multi-person scene, posture detection can be performed only on the user closest to the center in the first image. The AI ​​processor can determine the area where the photographed object closest to the image center of the first image is located based on the coordinates corresponding to the target frame in the target detection information corresponding to the first area. At least one second area can then be determined.

[0263] In some embodiments of the present application, the administrator user can use the smart screen to pre-set the users who need to perform gesture detection during the process of pre-setting the users who need to perform gesture detection. Of course, other devices associated with the smart screen can also be used to pre-set the users who need gesture detection. At the same time, the administrator user can set the users who need gesture detection anytime and anywhere. For example, the administrator user can set it after the smart screen is started. For another example, the administrator user can set it after the smart screen is started. The administrator user can also set it while other users are using the smart screen.

[0264] In one possible implementation, continue to refer to Figure 6 In (B), the administrator user can use the smart screen to make pre-settings. After the smart screen detects the input setting operation, it performs the posture detection task according to the posture detection mode corresponding to the setting operation. In other words, after the smart screen detects the input setting operation, it can determine J second areas in at least one first area based on preset rules.

[0265] For example, Fig.15 This is a schematic diagram of a smart screen interface provided by an embodiment of the present application. Fig.15 As shown in (A) of FIG. 1 , it is a front view of the smart screen provided in the embodiment of the present application. The smart screen displays a setting application icon, and the user can click the setting application, such as Fig.15 As shown in (B) in FIG. 1 , in response to a click operation on the setting application, the smart screen displays a user interface of the setting application, wherein the user interface of the setting application displays a list of all functions installed in the smart screen. The user can click on the gesture detection function in the list to trigger the property page corresponding to the gesture detection function.

[0266] like Fig.15 As shown in (C), the smart screen displays the property page corresponding to the posture detection function, which includes the first control corresponding to the first posture detection mode (remote control holder identification), the second control corresponding to the second posture detection mode (teenager or child identification), the third control corresponding to the third posture detection mode (center user identification), the prompt control corresponding to other posture reminder modes, the confirmation control and the cancel control. The user can click on any of the first control, the second control and the third control to trigger the subsequent smart screen to perform posture detection for a specific user.

[0267] In this way, the smart screen detects the user's click operation on the first control, and performs the posture detection task according to the first posture detection mode corresponding to the first control. Alternatively, the smart screen detects the user's click operation on the second control, and performs the posture detection task according to the second posture detection mode corresponding to the second control. Alternatively, the smart screen detects the user's click operation on the third control, and performs the posture detection task according to the third posture detection mode corresponding to the third control. For example, the smart screen can perform posture detection and posture reminders for users close to the center of the image.

[0268] In another possible implementation, the administrator user can use the smart life application in the mobile phone to make pre-settings. After the mobile phone detects the input setting operation, it sends the setting instruction corresponding to the setting operation to the smart screen to instruct the smart screen to perform subsequent posture detection and posture reminder tasks according to the setting instruction corresponding to the setting operation. Among them, the administrator user can be the account holder who logs in to the corresponding account of the smart screen.

[0269] For example, Fig.16 This is a schematic diagram of an interface of a smart life application provided in an embodiment of the present application. Fig.16 As shown in (A) of FIG. 1 , it is a front view of a mobile phone provided in an embodiment of the present application. The display screen of the mobile phone displays a smart life application icon, and the user can click on the smart life application icon, such as Fig.16 As shown in (B), in response to a click operation on the smart life application icon, the mobile phone displays the user interface of the smart life application, and the user interface may include pre-configured scene names, electronic devices at home, and corresponding status information. For example, the electronic devices at home and the corresponding status information include the TV in the living room (online), the smart screen in the master bedroom (online), the audio in the master bedroom (offline), and the smart door lock (online). The pre-configured scene names include home scene, sleep scene, and going out scene. The user interface may also include icons of configured electronic devices.

[0270] It is understandable that the configured electronic device can be an electronic device associated with a user account. The user needs to log in to the corresponding user account in the smart life application. And the electronic device pre-configured by the user needs to be bound to the user account to complete the configuration. If the user has not logged in to the smart life application, he needs to log in to the corresponding user account and enter the user name and password.

[0271] Continue to see Fig.16 In (B), the user can also click the device control corresponding to the "Smart Screen" on the user interface to trigger the device details page corresponding to the Smart Screen. Fig.16 As shown in (C), after the user clicks on the device control corresponding to the "Smart Screen", the mobile phone responds to the user's click operation and controls the display screen to display the device details page corresponding to the Smart Screen. The device details page includes the device icon, device status, function area and recording area corresponding to the Smart Screen. Among them, the function area includes user management controls, posture detection management controls, power management controls and more function controls. The user can click on the posture detection management control to trigger the posture detection management interface.

[0272] For example, Fig.16 As shown in (D), the mobile phone detects the user's click operation on the gesture detection management control and displays the gesture detection management interface. The gesture detection management interface includes a first control corresponding to the first gesture detection mode (remote control holder identification), a second control corresponding to the second gesture detection mode (youth or child identification), a third control corresponding to the third gesture detection mode (center user identification), other gesture reminder mode controls, a confirmation control, and a cancel control.

[0273] The user can click on any of the first, second and third controls to trigger the subsequent smart screen to perform posture detection for a specific user. Then, the mobile phone detects the user's click operation on the first control and sends a first setting instruction to the smart screen. The first setting instruction is used to instruct the smart screen to perform a posture detection task according to the first posture detection mode corresponding to the first control. In this way, the smart screen can perform posture detection and posture reminders for users other than the remote control holder.

[0274] Alternatively, the mobile phone detects a user's click operation on the second control and sends a second setting instruction to the smart screen, and the second setting instruction is used to instruct the smart screen to perform a posture detection task according to the second posture detection mode corresponding to the second control. In this way, the smart screen can perform posture detection and posture reminders for teenagers or children.

[0275] Alternatively, the mobile phone detects a user's click operation on the third control and sends a third setting instruction to the smart screen, and the third setting instruction is used to instruct the smart screen to perform a posture detection task according to the third posture detection mode corresponding to the third control. In this way, the smart screen can perform posture detection and posture reminders for users close to the center of the image.

[0276] It can be understood that after receiving the setting instruction sent by the mobile phone, the smart screen can determine J second areas in at least one first area based on preset rules.

[0277] It can be seen that when multiple people use the smart screen, the embodiment of the present application can not only perform posture detection for each user, but also perform posture detection only for pre-set users. Users can set users who need to perform posture detection anytime and anywhere, and can also switch users who need to perform posture detection anytime and anywhere, such as switching from the first posture detection mode to the second posture detection mode. It can provide users with more humanized functions and enhance the user experience.

[0278] In some embodiments of the present application, the smart screen can also interact with other associated devices.

[0279] Exemplarily, the smart screen can be linked with associated devices such as mobile phones, smart watches, and headphones, and the posture detection results can be fed back to the user on the associated device side. Among them, the user on the associated device side can be the account holder who logs in to the corresponding account of the smart screen, that is, the administrator user. In this way, the administrator user can decide to remind one or more users using the smart screen. At the same time, the smart screen can also track and count postures for each user, and output the posture statistics corresponding to each user.

[0280] In some embodiments of the present application, when the smart screen determines that the back postures of N photographed subjects are abnormal according to the posture parameters, it sends the posture detection result to the second device such as a mobile phone. The posture detection result includes the abnormal postures corresponding to the N photographed subjects.

[0281] The mobile phone receives the posture detection result from the smart screen, and outputs the second information according to the posture detection result; the second information is used to prompt that the back posture of N photographed subjects is abnormal.

[0282] Then, the mobile phone detects a trigger operation on the target subject among N subjects, and sends a trigger instruction to the smart screen. The trigger instruction is used to instruct a posture reminder for the target subject.

[0283] The smart screen can output a first prompt message according to the trigger instruction; the first prompt message is used to prompt the target subject to adjust his back posture.

[0284] Specifically, in the process of determining that the back postures of multiple photographed subjects are abnormal according to the posture parameters, the AI ​​processor Fig.17 , the AI ​​processor can send the posture detection result to the mobile phone. The mobile phone is associated with the smart screen. For example, the account logged in to the mobile phone and the smart screen is the same account. The posture detection result includes abnormal postures corresponding to multiple photographed objects.

[0285] Next, after the mobile phone receives the posture detection result sent by the AI ​​processor in the smart screen, it can output the second information according to the posture detection result. Among them, the second information is used to prompt that the back posture of multiple subjects is an abnormal posture. Then, the mobile phone can detect the trigger operation for the target subject among multiple subjects, and send a trigger instruction to the smart screen, so that the smart screen can remind the target subject of the posture. The smart screen receives the trigger instruction sent by the mobile phone, and outputs the first prompt information according to the trigger instruction.

[0286] Exemplarily, the posture detection result includes a hunchback posture corresponding to the first user "user A" and a side-lying posture corresponding to the second user "user B". The mobile phone outputs the second information according to the posture detection result. The second information may also include one or more of pop-up prompt information, text prompt information, voice prompt information or video prompt information.

[0287] In one possible implementation, see Fig.18In (A), the mobile phone can display the second information, which can be implemented as a pop-up window, including a fourth control "User A is in a hunchback posture, user B is in a side-lying posture, please confirm whether to remind them". Of course, the mobile phone can also control the speaker to play the second information, which can be implemented as a prompt sound such as "User A is in a hunchback posture, user B is in a side-lying posture, please confirm whether to remind them", to prompt users A and B of abnormal postures.

[0288] In another possible implementation, see Fig.18 In (B), the mobile phone can display the second information, and the second information can be implemented in the form of a pop-up window, which includes the fifth control, the user control corresponding to the user, the confirmation control, and the cancel control. The fifth control is implemented as the text "User A is in a hunchback posture, user B is in a side-lying posture, please select the user who needs posture reminder". In this way, the user on the mobile phone side can trigger the smart screen to provide posture reminders for specific users. The mobile phone detects a trigger operation for the target shooting subject such as user A (a click operation on the user control and the confirmation control), and sends a trigger instruction to the smart screen, so that the smart screen outputs the first prompt information. Of course, the mobile phone can also control the speaker to play the second information to prompt users A and B of abnormal postures.

[0289] In some embodiments of the present application, when the user selects a specific user for gesture reminder through the mobile phone, the mobile phone can also detect the user's trigger operation on the target shooting object and send a trigger instruction to the smart screen. The smart screen can output the first prompt information according to the trigger instruction.

[0290] It should be noted that the associated devices corresponding to the smart screen may include the above-mentioned mobile phones, smart watches, wireless headphones, smart glasses, etc. The embodiment of the present application does not specifically limit the device type and number of associated devices. At the same time, the implementation method of the above-mentioned second information is only illustrative, and the embodiment of the present application also does not limit the specific implementation method of the second information.

[0291] In some embodiments of the present application, the smart screen can also output the posture statistics corresponding to each photographed subject after determining that the back posture of the photographed subject is an abnormal posture according to the posture parameters; the posture statistics include the abnormal posture and the posture holding time corresponding to the abnormal posture. Of course, the posture statistics can also include the normal posture and the posture holding time corresponding to the normal posture.

[0292] In some embodiments of the present application, the user can also select a specific associated device to control the smart screen for posture reminders. For example, the user can select a mobile phone to control the smart screen for posture reminders, so that the smart screen can send the posture detection results to the mobile phone. For another example, the user can select a smart watch to control the smart screen for posture reminders, so that the smart screen can send the posture detection results to the smart watch. For another example, the user can select smart glasses / wireless headphones to control the smart screen for posture reminders, so that the smart screen can send the posture detection results to the smart glasses / wireless headphones.

[0293] In some embodiments of the present application, the user can select a specific associated device on a mobile phone or smart screen.

[0294] For example, Fig.19 As shown in (A) in FIG, the mobile phone detects the user's click operation on the gesture detection management control and displays the gesture detection management interface. Fig.19 As shown in (B), the posture detection management interface includes a first reminder control corresponding to the first posture reminder mode (mobile phone posture reminder), a second reminder control corresponding to the second posture reminder mode (smart watch posture reminder), a third reminder control corresponding to the third posture reminder mode (smart glasses / wireless headset posture reminder), a confirmation control, and a cancel control.

[0295] For example, Fig. 20 As shown in , the smart screen can display the property page corresponding to the posture detection function. The property page corresponding to the posture detection function includes the first reminder control corresponding to the first posture reminder mode (mobile phone posture reminder), the second reminder control corresponding to the second posture reminder mode (smart watch posture reminder), the third reminder control corresponding to the third posture reminder mode (smart glasses / wireless headphones posture reminder), the confirmation control and the cancel control.

[0296] In this way, the user can click on any of the first reminder control, the second reminder control, and the third reminder control to trigger the smart screen to subsequently send the gesture detection result to the associated device. Figure 6 In (C), after detecting the user's selection operation for the posture reminder mode, the smart screen determines the associated device corresponding to the posture reminder mode and sends the posture detection result to the associated device, so that the associated device can display the second prompt information later.

[0297] It can be seen that the embodiment of the present application can not only support posture detection for pre-set users, but also support the selection of specific associated devices for posture reminders. Users can set the associated devices for posture reminders anytime and anywhere, and use the associated devices to perform precise posture reminders. Specifically, when multiple people use the smart screen, the smart screen can also be controlled through associated devices to remind specific users of postures. At the same time, the embodiment of the present application can also output posture statistics, and can support real-time supervision and recording of the corresponding posture of each user. It is convenient for users to view their own posture status, provide users with more humane functions, and enhance the user experience.

[0298] An embodiment of the present application further provides an interactive system, which includes a first device and a second device.

[0299] The first device is configured to acquire a first image, where the first image includes at least one photographed object.

[0300] The first device is also configured to perform posture detection on the first image, determine the human body joint point position data of the photographed object, and determine the posture parameters according to the human body joint point position data.

[0301] The first device is also configured to send a posture detection result to the second device when it is determined according to the posture parameters that the back postures of N photographed subjects are abnormal postures; wherein the posture detection result includes the abnormal postures corresponding to the N photographed subjects.

[0302] The second device is configured to receive the posture detection result and output second information according to the posture detection result; the second information is used to prompt that the back posture of the N photographed subjects is an abnormal posture.

[0303] The second device is also configured to detect a trigger operation on a target photographing object among the N photographing objects, and send a trigger instruction to the first device, where the trigger instruction is used to instruct a posture reminder for the target photographing object.

[0304] The first device is also configured to output first prompt information according to the trigger instruction; the first prompt information is used to prompt the target subject to adjust his back posture.

[0305] An embodiment of the present application also provides a display device, which includes: a smart camera and a display screen, and the smart camera includes a processor.

[0306] The processor is configured to acquire a first image, where the first image includes at least one photographed object.

[0307] The processor is also configured to perform posture detection on the first image, determine the human joint point position data of the photographed object, and determine the posture parameters based on the human joint point position data; the posture parameters include the joint angles corresponding to the human joint points; the posture parameters are used to characterize the back posture corresponding to at least one photographed object.

[0308] The processor is also configured to control the display screen to output first information when it is determined according to the posture parameters that the back posture of the subject is abnormal.

[0309] The display screen is configured to output first information, where the first information is used to prompt the subject to adjust his back posture.

[0310] In some schemes, multiple embodiments of the present application may be combined, and the combined scheme may be implemented. Optionally, some operations in the process of each method embodiment may be optionally combined, and / or the order of some operations may be optionally changed. Furthermore, the execution order between the steps of each process is only exemplary and does not constitute a limitation on the execution order between the steps, and other execution orders may be used between the steps. It is not intended to indicate that the execution order is the only order in which these operations can be performed.

[0311] A person skilled in the art will be able to think of many ways to reorder the operations described in the embodiments of the present application. In addition, it should be noted that the process details involved in a certain embodiment of the present application are also applicable to other embodiments in a similar manner, or different embodiments can be used in combination.

[0312] In addition, some steps in the method embodiment may be equivalently replaced by other possible steps. Alternatively, some steps in the method embodiment may be optional and may be deleted in certain usage scenarios. Alternatively, other possible steps may be added to the method embodiment.

[0313] Furthermore, the various method embodiments may be implemented separately or in combination.

[0314] The present application also provides an electronic device, such as the above-mentioned smart screen. Fig.21 As shown, the electronic device may include one or more processors 1910 , a memory 1920 , and a communication interface 1930 .

[0315] The memory 1920 , the communication interface 1930 , and the processor 1910 are coupled together. For example, the memory 1920 , the communication interface 1930 , and the processor 1910 may be coupled together via a bus 1940 .

[0316] The communication interface 1930 is used for data transmission with other devices. The memory 1920 stores computer program codes. The computer program codes include computer instructions. When the computer instructions are executed by the processor 1910, the electronic device executes the posture detection method in the embodiment of the present application.

[0317] Among them, the processor 1910 can be a processor or a controller, for example, a central processing unit (CPU), a general processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the present disclosure. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of DSP and microprocessors, and the like.

[0318] The bus 1940 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus 1940 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.21 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0319] An embodiment of the present application also provides an electronic device, which includes: a memory, a camera and a display screen, wherein the camera includes a processor, the memory and the display screen are coupled to the processor, the display screen is used to output first information or second information, and the memory is used to store computer-readable instructions. When the processor reads the computer-readable instructions from the memory, the electronic device executes the above-mentioned posture detection method.

[0320] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program code is stored. When the processor executes the computer program code, the electronic device executes the relevant method steps in the method embodiment.

[0321] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, it enables the computer to execute the relevant method steps in the above method embodiment.

[0322] An embodiment of the present application also provides a chip system, including: a processor, the processor is coupled to a memory, the memory is used to store programs or instructions, when the program or instructions are executed by the processor, the chip system implements the method in any of the above method embodiments.

[0323] Optionally, the processor in the chip system may be one or more. The processor may be implemented by hardware or by software. When implemented by hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented by software, the processor may be a general-purpose processor implemented by reading software code stored in a memory.

[0324] Optionally, the memory in the chip system may also be one or more. The memory may be integrated with the processor or may be separately arranged with the processor, which is not limited in the embodiments of the present application. Exemplarily, the memory may be a non-transient processor, such as a read-only memory ROM, which may be integrated with the processor on the same chip or may be arranged on different chips respectively. The embodiments of the present application do not specifically limit the type of memory and the arrangement of the memory and the processor.

[0325] Exemplarily, the chip system can be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD) or other integrated chips.

[0326] Among them, the electronic device, computer storage medium or computer program product provided in this application is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0327] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0328] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0329] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, which may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0330] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0331] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0332] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A posture detection method, characterized in that: Applied to a first device, the method includes: Acquire a first image, where the first image includes at least one photographed object; Performing posture detection on the first image to determine human joint point position data corresponding to the at least one photographed object, and determining posture parameters according to the human joint point position data; the posture parameters include joint angles corresponding to the human joint points; the posture parameters are used to characterize the back posture corresponding to the at least one photographed object; When it is determined according to the posture parameters that the back posture of the photographed subject is abnormal, first information is output; the first information is used to prompt the photographed subject to adjust the back posture.

2. The method according to claim 1, characterized in that Acquire the first image, including: Acquiring the first image during the process of the first device playing the target media resource; The method further comprises: When it is determined according to the posture parameter that the back posture of the photographed subject is abnormal, pausing the playing of the target media resource; acquiring a second image; When it is determined that the back posture of the at least one photographed object is a normal posture according to the posture parameters obtained from the second image, the target media resource is resumed to be played.

3. The method according to claim 1 or 2, characterized in that: The first information includes first prompt information; when it is determined according to the posture parameter that the back posture of the photographed subject is abnormal, outputting the first information includes: When it is determined according to a preset condition that there is a target photographing object among the N photographing objects, the first prompt information is output; the first prompt information is used to prompt the target photographing object to adjust its back posture, and N is a positive integer.

4. The method according to claim 1 or 2, characterized in that: The first information includes first prompt information; when it is determined according to the posture parameter that the back posture of the photographed subject is abnormal, outputting the first information includes: When it is determined according to the posture parameter that the back postures of N photographed subjects are abnormal, sending the posture detection result to the second device; wherein N is a positive integer, the posture detection result includes the abnormal postures corresponding to the N photographed subjects, and the second device is associated with the first device; receiving a trigger instruction sent by the second device; the trigger instruction is used to instruct to perform a posture reminder on a target photographing object among the N photographing objects; According to the trigger instruction, the first prompt information is output; the first prompt information is used to prompt the user to adjust the back posture of the target subject.

5. The method according to claim 3 or 4, characterized in that: The outputting the first prompt information includes: Acquire a target three-dimensional skeleton model, where the target three-dimensional skeleton model is a three-dimensional skeleton model of a human body corresponding to a normal posture; Generate a dynamic image according to the three-dimensional human skeleton model corresponding to the target photographed object and the target three-dimensional skeleton model; the dynamic image presents a process of changing from the three-dimensional human skeleton model corresponding to the target photographed object to the target three-dimensional skeleton model; The first prompt information is output, where the first prompt information includes the dynamic effect image.

6. The method according to claim 5, characterized in that The first prompt information also includes voice prompt information, and the voice prompt information is used to prompt the target shooting object to follow the dynamic image to adjust the back posture to the normal posture.

7. The method according to any one of claims 1 to 6, characterized in that: The first information includes one or more of pop-up prompt information, text prompt information, voice prompt information or video prompt information.

8. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: After determining that the back posture of the subject is abnormal according to the posture parameters, the posture statistics corresponding to each subject are output; the posture statistics include the abnormal posture and the posture holding time corresponding to the abnormal posture.

9. The method according to any one of claims 1 to 8, characterized in that: The performing posture detection on the first image to determine the human body joint point position data corresponding to the at least one photographed object includes: Performing object detection on the first image to obtain at least one first area; Perform posture detection on the at least one first area to determine human body joint point position data corresponding to the at least one photographed object.

10. The method according to claim 9, characterized in that The performing posture detection on the at least one first area comprises: Based on a preset rule, determining J second areas in the at least one first area, where J is a positive integer; Performing posture detection on the J second areas; The preset rule includes determining, in the at least one first area, areas other than the area where the photographed subject holding the preset item is located as the J second areas; Alternatively, the preset rule includes determining, in the at least one first area, areas where photographed subjects whose age values ​​are less than or equal to the age threshold are located as the J second areas; Alternatively, the preset rule includes determining, in the at least one first area, an area where the photographed object closest to the image center of the first image is located as the J second areas.

11. The method according to any one of claims 1 to 10, characterized in that: The abnormal posture includes a hunchback posture, a side-lying posture or a backward leaning posture; the posture parameters include a forward leaning angle, a trunk backward bending angle and a lateral tilting angle corresponding to the photographed object; and determining that the back posture of the photographed object is an abnormal posture according to the posture parameters includes: When the forward tilt angle is positive and greater than the first preset angle, the absolute value of the trunk backward bending angle is greater than the second preset angle, and the absolute value of the lateral tilt angle is less than the third preset angle, it is determined that the back posture of the photographed subject is the hunchback posture; When the absolute value of the lateral tilt angle is greater than or equal to the third preset angle, determining that the back posture of the photographed subject is the side-lying posture; When the forward leaning angle is a negative value and is smaller than a fourth preset angle, it is determined that the back posture of the photographed subject is the backward leaning posture, and the fourth preset angle is smaller than the first preset angle.

12. The method according to claim 11, characterized in that When the absolute value of the lateral tilt angle is greater than or equal to the third preset angle, determining that the back posture of the photographed subject is a side-lying posture includes: When the forward tilt angle is a positive value and is less than or equal to the first preset angle or the forward tilt angle is a negative value and is greater than or equal to the fourth preset angle, and / or the absolute value of the trunk backward bending angle is less than or equal to the second preset angle, and the absolute value of the lateral tilt angle is greater than or equal to the third preset angle, it is determined that the back posture of the photographed subject is the side-lying posture; When the forward tilt angle is a negative value and is smaller than the fourth preset angle, determining that the back posture of the photographed subject is a backward tilt posture includes: When the absolute value of the trunk backward bending angle is greater than the second preset angle, and / or the absolute value of the lateral tilt angle is less than the third preset angle, and the forward tilt angle is a negative value and less than the fourth preset angle, it is determined that the back posture of the subject is the backward leaning posture.

13. The method according to any one of claims 1 to 12, characterized in that: Determining the posture parameters according to the human body joint position data includes: Generate a three-dimensional human skeleton model corresponding to the photographed object in the current posture according to the human joint point position data; Acquire position data corresponding to target joint points in the three-dimensional human skeleton model; the target joint points include back joint points, hip joint points corresponding to the back, and head and neck joint points; The posture parameter is determined according to the position data corresponding to the target joint point.

14. A posture detection method, characterized in that: Applied to the second device, the method includes: receiving a gesture detection result from a first device, the first device being associated with the second device; The posture detection result includes abnormal postures of N photographed subjects corresponding to back postures, where N is a positive integer; Outputting second information according to the posture detection result; the second information is used to prompt that the back postures of the N photographed objects are abnormal; A trigger operation for a target photographing object among the N photographing objects is detected, and a trigger instruction is sent to the first device, where the trigger instruction is used to instruct a posture reminder for the target photographing object.

15. An interactive system, characterized in that: The interactive system includes a first device and a second device; The first device is configured to acquire a first image, wherein the first image includes at least one photographed object; The first device is further configured to perform posture detection on the first image, determine human joint point position data of the photographed object, and determine posture parameters according to the human joint point position data; The first device is further configured to send a posture detection result to the second device when it is determined according to the posture parameter that the back postures of the N photographed subjects are abnormal postures; wherein the posture detection result includes the abnormal postures corresponding to the N photographed subjects; The second device is configured to receive the posture detection result and output second information according to the posture detection result; the second information is used to prompt that the back posture of the N photographed objects is an abnormal posture; The second device is further configured to detect a trigger operation on a target photographing object among the N photographing objects, and send a trigger instruction to the first device, wherein the trigger instruction is used to instruct to perform a gesture reminder on the target photographing object; The first device is also configured to output first prompt information according to the trigger instruction; the first prompt information is used to prompt the user to adjust the back posture of the target subject.

16. A display device, characterized in that: The display device comprises: a smart camera and a display screen, wherein the smart camera comprises a processor; The processor is configured to acquire a first image, wherein the first image includes at least one photographed object; The processor is further configured to perform posture detection on the first image, determine the position data of the human joint points of the photographed object, and determine the posture parameters according to the position data of the human joint points; the posture parameters include the joint angles corresponding to the human joint points; the posture parameters are used to characterize the back posture corresponding to the at least one photographed object; The processor is further configured to control the display screen to output first information when it is determined according to the posture parameter that the back posture of the photographed subject is an abnormal posture; The display screen is configured to output first information, where the first information is used to prompt the subject to adjust his back posture.

17. An electronic device, characterized in that: The electronic device includes: a memory, a camera and a display screen, the camera includes a processor, the memory and the display screen are coupled to the processor, the display screen is used to output the first information or the second information, and the memory is used to store computer-readable instructions. When the processor reads the computer-readable instructions from the memory, the electronic device executes the posture detection method as described in any one of claims 1 to 13, or the posture detection method as described in claim 14.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a computer program, and when the computer program is executed on an electronic device, the electronic device executes the posture detection method according to any one of claims 1 to 13, or the posture detection method according to claim 14.

Images