Posture monitoring method, device, system, electronic device and storage medium

Abstract

The application provides a posture monitoring method, device and system, electronic equipment and a storage medium. The skin exposed area of a target object can be identified from an image of the target object, wherein the skin exposed area of the target object includes a hand area and / or a face area of the target object. Then, the posture of the target object is determined based on the position of the skin exposed area of the target object or the position of the skin exposed area of the target object relative to a specific reference target, thereby achieving the purpose of monitoring the posture of people. Moreover, through monitoring the posture, people can discover and adjust the non-standard posture in time, thereby avoiding the influence of the non-standard posture on the health of people.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of posture monitoring, and particularly relates to a posture monitoring method, device, system, electronic device and storage medium. BACKGROUND

[0002] When people are in a certain posture for a long time, if the posture is not standard, it may affect the physical state of people. For example, an unstandard sitting posture may make people develop bad habits such as humpback and body distortion, and even cause diseases such as cervical spondylosis, lumbar disc herniation and myopia. Therefore, how to monitor the posture of people in order to help people adjust the unstandard posture in time is a technical problem to be solved by those skilled in the art. SUMMARY

[0003] Therefore, the present application provides a posture monitoring method, device, system, electronic device and storage medium, which can monitor the posture of people.

[0004] The technical scheme provided by the present application is as follows:

[0005] In a first aspect, the present application provides a posture monitoring method, comprising:

[0006] identifying a skin exposed area of a target object from an image of the target object; the skin exposed area of the target object includes a hand area and / or a face area of the target object;

[0007] determining a posture of the target object based on a position of the skin exposed area of the target object or a position of the skin exposed area of the target object relative to a specific reference target.

[0008] Further, in the above method, the skin exposed area of the target object is identified from the image of the target object, comprising:

[0009] identifying a skin color area of the target object from the image of the target object;

[0010] determining the skin exposed area of the target object based on the skin color area of the target object.

[0011] Further, in the above method, the skin color area of the target object is identified from the image of the target object, comprising:

[0012] detecting a target area with a color within a set skin color interval from the image of the target object as the skin color area of the target object.

[0013] Further, in the above method, the skin exposed area of the target object is determined based on the skin color area of the target object, comprising:

[0014] inputting the skin color region of the target object into a pre-trained skin bare region extraction network to obtain a skin bare region of the target object output by the skin bare region extraction network.

[0015] Further, in the above method, the skin bare region extraction network is obtained by joint training with a skin color region segmentation network; the joint training process includes:

[0016] training the skin color region segmentation network by taking an image of a sample object as a first training sample and a position of a skin color region of the sample object in the image of the sample object as a first training label;

[0017] training the skin bare region extraction network by taking an output result obtained by inputting an image of a sample object into the skin color region segmentation network as a second training sample and a position of a skin bare region of the sample object in the image of the sample object as a second training label.

[0018] Further, in the above method, determining the pose of the target object based on the position of the skin bare region of the target object includes:

[0019] determining a pose type of the target object according to the position of the skin bare region of the target object and positions of skin bare regions corresponding to various pose types set in advance.

[0020] Further, in the above method, determining a pose type feature of the target object according to the position of the skin bare region of the target object and positions of skin bare regions corresponding to various pose types set in advance includes:

[0021] determining a retrieval feature of the skin bare region of the target object according to the position of the skin bare region of the target object and image features;

[0022] retrieving a target pose type corresponding to a skin bare region closest to the retrieval feature of the skin bare region of the target object from a pre-set pose type library according to the retrieval feature of the skin bare region of the target object; the pose type library stores various pose types and retrieval features of skin bare regions corresponding to the various pose types;

[0023] determining the target pose type as the pose type of the target object.

[0024] Further, in the above method, determining a retrieval feature of the skin bare region of the target object according to the position of the skin bare region of the target object and image features includes:

[0025] inputting the position and image features of the skin exposed area into a pre-trained retrieval feature mapping network to obtain retrieval features of the skin exposed area of the target object output by the retrieval feature mapping network; wherein the retrieval feature mapping network is obtained by training a shallow neural network.

[0026] Further, in the above method, the pose of the target object is determined based on the position of the skin exposed area of the target object relative to a specific reference target, including:

[0027] determining a relative position relationship between the skin exposed area of the target object and a set reference object according to the position of the skin exposed area of the target object; the relative position relationship between the skin exposed area of the target object and the set reference object includes at least one of a position relationship between a hand region of the target object and the set reference object, and a position relationship between a face region of the target object and the set reference object;

[0028] determining a pose type of the target object according to the relative position relationship and a relative position relationship corresponding to each set pose type.

[0029] Further, in the above method, further comprising:

[0030] outputting the pose of the target object; or

[0031] if the pose of the target object belongs to a set non-standard pose, outputting the pose of the target object and / or outputting a pose correction reminder.

[0032] In a second aspect, the present application provides a pose monitoring device, comprising:

[0033] a recognition module configured to recognize a skin exposed area of a target object from an image of the target object; the skin exposed area of the target object including a hand region and / or a face region of the target object;

[0034] a determination module configured to determine a pose of the target object based on a position of the skin exposed area of the target object or a position of the skin exposed area of the target object relative to a specific reference target.

[0035] In a third aspect, the present application provides an electronic device, comprising:

[0036] a memory and a processor;

[0037] wherein the memory is configured to store a program;

[0038] the processor is configured to realize the method of any one of the above by running the program in the memory.

[0039] In a fourth aspect, the present application provides a posture monitoring system, comprising a control device and a camera device connected in communication;

[0040] The camera device is configured to acquire an image of a target object;

[0041] The control device is configured to identify a skin exposed area of the target object from the image of the target object, wherein the skin exposed area of the target object comprises a hand area and / or a face area of the target object; and determine a posture of the target object based on a position of the skin exposed area of the target object or a position of the skin exposed area of the target object relative to a specific reference target.

[0042] In a fifth aspect, the present application provides a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method of any one of the above aspects.

[0043] The posture monitoring method provided by the present application can identify a skin exposed area of a target object from an image of the target object, wherein the skin exposed area of the target object comprises a hand area and / or a face area of the target object. Then, a posture of the target object is determined based on a position of the skin exposed area of the target object or a position of the skin exposed area of the target object relative to a specific reference target, thereby achieving the purpose of monitoring the posture of people. Moreover, by monitoring the posture, people can find and adjust the non-standard posture in time, thereby avoiding the influence of the non-standard posture on the health of people. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only embodiments of the present application, and those skilled in the art can obtain other drawings according to the provided drawings without creative labor.

[0045] Figure 1 is a flowchart of a posture monitoring method provided by an embodiment of the present application;

[0046] Figure 2 is a structural diagram of a convolutional neural network provided by an embodiment of the present application;

[0047] Figure 3 is a flowchart of determining a posture type of a target object provided by an embodiment of the present application;

[0048] Figure 4 is a mapping diagram of a feature mapping network provided by an embodiment of the present application;

[0049] Figure 5 is a schematic diagram of a tree structure provided by an embodiment of the present application;

[0050] Figure 6 is a structural schematic diagram of a posture monitoring device provided by an embodiment of the present application;

[0051] Figure 7 is a structural schematic diagram of an electronic device provided by an embodiment of the present application;

[0052] Figure 8 is a structural schematic diagram of a posture monitoring system provided by an embodiment of the present application. DETAILED DESCRIPTION

[0053] SUMMARY

[0054] The technical solution of the embodiment of the present application is applicable to the application scenario of monitoring posture. By using the technical solution of the embodiment of the present application, the purpose of monitoring posture can be achieved. People can discover and adjust non-standard posture in time, so as to avoid the influence of non-standard posture on people's health.

[0055] When people are in a certain posture for a long time, if the posture is non-standard, it may affect people's health. For example, if the long-time sitting posture is non-standard, people may develop bad habits such as humpback and body distortion. In severe cases, it may also cause diseases such as cervical spondylosis and lumbar disc herniation. Moreover, if people study or work in a non-standard sitting posture, it may also induce eye diseases such as myopia.

[0056] However, in the prior art, there is a lack of effective monitoring of people's posture, and non-standard posture cannot be reminded and corrected. Based on this, the embodiment of the present application provides a posture monitoring method, device, system, electronic device and storage medium. Through monitoring posture, people can discover and adjust non-standard posture in time, so as to avoid the influence of non-standard posture on people's health.

[0057] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0058] Exemplary Method

[0059] The embodiment of the present application provides a posture monitoring method, which can be executed by an electronic device. The electronic device can be any device having a data and instruction processing function, for example, a computer, a smart terminal, a server, etc. Referring to FIG. 1, Figure 1 The method comprises the following steps.

[0060] In S101, a skin exposed area of a target object is recognized from an image of the target object.

[0061] The target object refers to a subject of posture monitoring, and the target object is generally a living body. In some embodiments, the target object is a human body. The image of the target object refers to an image obtained by photographing the target object. The image of the target object can be a video image or a picture image, which is not limited in the embodiment.

[0062] The skin exposed area of the target object refers to a skin area without clothes or other articles attached and exposed. In the embodiment, the skin exposed area of the target object includes a hand area and / or a face area of the target object. In some embodiments, in addition to the hand area and / or the face area of the target object, if an arm area, a neck area, etc. of the target user are also without clothes or other articles attached, the skin exposed area can also include the arm area, the neck area, etc. of the target user.

[0063] In the embodiment of the present application, the image of the target object is processed to recognize the skin exposed area of the target object from the image of the target object. For example, if the skin exposed area of the target object includes a hand area and a face area of the target object, the embodiment can process the image of the target object to recognize the position of the hand area and the position of the face area of the target object from the image of the target object.

[0064] In some embodiments, a skin exposed area recognition model can be pre-trained, the image of the target object is input into the skin exposed area recognition model, and the skin exposed area of the target object output by the skin exposed area recognition model is obtained.

[0065] If the skin exposed area of the target object includes a hand area and a face area, during training of the skin exposed area recognition model, a large number of sample objects can be set, an image of each sample object is obtained as a training sample, and a specific position of the hand area and the face area of the sample object in the image is marked as a training label. A convolutional neural network model can be selected as a basic model of the skin exposed area recognition model, and the training process is as follows: the training sample is input into the skin exposed area recognition model to obtain an output result of the skin exposed area recognition model, the loss value of the skin exposed area recognition model is determined by comparing the training label and the output result of the skin exposed area recognition model, the parameters of the skin exposed area recognition model are adjusted to reduce the loss value of the skin exposed area recognition model, and then the above training process is repeated until the loss value of the skin exposed area recognition model is less than a set value, and the training is completed. The set value can be set according to actual conditions, which is not limited in the embodiment.

[0066] In some embodiments, a quadratic polynomial mixed skin color model can also be used to identify the skin exposed area of the target object from the image of the target object, which is not limited in the embodiment.

[0067] In S102, the pose of the target object is determined based on the position of the skin exposed area of the target object or the position of the skin exposed area of the target object relative to a specific reference target.

[0068] In some embodiments, after the skin exposed area of the target object is obtained, the pose of the target object can be determined according to the position of the skin exposed area of the target object. That is, in the embodiment, the pose of the target object can be determined at least according to the position of the hand area and / or the face area in the skin exposed area.

[0069] Specifically, a first pose detection model can be set in advance, and the skin exposed area of the target object is input into the first pose detection model to obtain the pose of the target object output by the first pose detection model.

[0070] During training of the first pose detection model, a large number of sample objects can be set, and the skin exposed area of each sample object is obtained as a training sample, and the corresponding pose of each skin exposed area is marked as a training label. The first pose detection model can also use a convolutional neural network model as a basic model, and the specific training process is the same as that of the skin exposed area recognition model in the above embodiment. Therefore, the training process of the first pose detection model can be referred to the training process of the skin exposed area recognition model in the above embodiment, which is not described herein.

[0071] In some embodiments, the pose of the target object can be determined according to the position of the skin exposed area of the target object relative to the specific reference target. That is, in the present embodiment, the pose of the target object can be determined at least according to the position of the hand region and / or the face region in the skin exposed area relative to the specific reference target.

[0072] Specifically, the second pose detection model can be pre-set, and the skin exposed area of the target object and the position of the specific reference target can be input into the second pose detection model to obtain the pose of the target object output by the second pose detection model.

[0073] During training of the second pose detection model, a large number of sample objects can be set, and the skin exposed area of the sample object and the position of the specific reference target can be obtained as training samples, and the pose corresponding to each skin exposed area can be marked as a training label. The second pose detection model can also use a convolutional neural network model as a basic model, and the specific training process is the same as that of the skin exposed area recognition model in the above embodiment. Therefore, the training process of the second pose detection model can be referred to the training process of the skin exposed area recognition model in the above embodiment, which will not be described here.

[0074] In the above embodiments, the skin exposed area of the target object can be recognized from the image of the target object, wherein the skin exposed area of the target object includes the hand region and / or the face region of the target object. Then, the pose of the target object can be determined based on the position of the skin exposed area of the target object or the position of the skin exposed area of the target object relative to the specific reference target, thereby achieving the purpose of monitoring the pose of people. Moreover, through monitoring the pose, people can timely find and adjust the non-standard pose, thereby avoiding the influence of the non-standard pose on the health of people.

[0075] As an optional implementation, in another embodiment of the present application, the step of recognizing the skin exposed area of the target object from the image of the target object in the above embodiment can specifically include the following steps:

[0076] The skin color region of the target object is recognized from the image of the target object; and the skin exposed area of the target object is determined based on the skin color region of the target object.

[0077] In the embodiment of the present application, when the skin exposed area of the target object is recognized from the image of the target object, in addition to the description in the above embodiment, the skin color region of the target object can be first recognized from the image of the target object, and then the skin color region of the target object is processed to obtain the skin exposed area of the target object.

[0078] Exemplarily, the skin color segmentation model can be pre-trained, and an image of the target object is input into the skin color segmentation model to obtain a skin color region of the target object output by the skin color segmentation model.

[0079] During training of the skin color segmentation model, a large number of sample objects can be set, and images of the sample objects are obtained as training samples, and the skin exposed regions in the images of the sample objects are marked as training samples. The skin color segmentation model can also use a convolutional neural network model as a basic model, and the specific training process is the same as that of the skin exposed region identification model in the above embodiment. Therefore, the training process of the skin color segmentation model can be understood by referring to the training process of the skin exposed region identification model in the above embodiment, which will not be described herein.

[0080] After obtaining the skin color region of the target object, the skin exposed region of the target object can be determined from the skin color region of the target object. Exemplarily, if the skin exposed region of the target object includes a hand region and a face region, after obtaining the skin color region of the target object, the specific positions of the hand region and the face region can be determined from the skin color region of the target object.

[0081] In some embodiments, a region determination model can be pre-trained, and the skin color region of the target object is input into the region determination model to obtain at least a hand region and / or a face region of the target object output by the region determination model, and the skin exposed region of the target object is obtained.

[0082] During training of the region determination model, a large number of sample objects can be set, and the skin color regions of the sample objects are obtained as training samples, and the positions of the skin exposed regions such as the head region and the hand region in the images of the sample objects are marked as training labels. The region determination model can also use a convolutional neural network model as a basic model, and the specific training process is the same as that of the skin exposed region identification model in the above embodiment. Therefore, the training process of the region determination model can be understood by referring to the training process of the skin exposed region identification model in the above embodiment, which will not be described herein.

[0083] In the above embodiments, the skin color region of the target object is first identified, and then the skin exposed region of the target object is extracted from the skin color region of the target object, so that the specific positions of the skin exposed regions such as the face region and the hand region can be quickly determined.

[0084] As an optional implementation, in another embodiment of the present application, the step of identifying the skin color region of the target object from the image of the target object in the above embodiment can specifically include the following steps:

[0085] From the image of the target object, a target region with color within a set skin color interval is detected as the skin color region of the target object.

[0086] In the embodiments of the present application, when the skin color region of the target object is identified from the image of the target object, in addition to training the skin color segmentation model as described in the above embodiments, a skin color interval can be set in advance, and the image of the target object is color detected, and the region in the image of the target object with color within the set skin color interval is determined as the skin color region of the target object.

[0087] In some embodiments, color detection can be performed by the RGB value of the image of the target object. Wherein RGB is the integer value of three different channels of the image. The RGB value interval of the skin color region can be set in advance, and then the RGB value of each pixel of the image of the target object is detected, and the region composed of pixels with RGB value within the above RGB value interval is determined as the skin color region of the target object.

[0088] In some embodiments, a quadratic polynomial mixed skin color model can also be used to identify the skin color region of the target object from the image of the target object. Specifically, the constraint equation set of the quadratic polynomial mixed skin color model is:

[0089] l u (R)=-1.3767R 2 +1.0743R+0.1452;

[0090] l l (R)=-0.776R 2 +0.5601R+0.1766;

[0091] W(R,G)=(R-0.33) 2 +(G-0.33) 2 .

[0092] On the basis of the above three equations, the skin color region can be realized by the following rules:

[0093] R1:G>l l (R)AND G<l u (R);

[0094] R2:W(R,G)≥0.0004;

[0095] R3:R>G>B;

[0096] R4:R-G≥45.

[0097] In the above embodiments, a region with a color within a set skin color interval in an image of a target object is detected as a skin color region of the target object. In this way, a complex model does not need to be introduced, the calculation is simple, the calculation amount is small, the storage space occupied is small, the application range is wide, and a low-power chip can also run without pressure.

[0098] As an optional implementation, in another embodiment of the present application, the step of determining the skin bare area of the target object based on the skin color region of the target object in the above embodiment can specifically include the following steps:

[0099] The skin color region of the target object is input into a pre-trained skin bare area extraction network to obtain the skin bare area of the target object output by the skin bare area extraction network.

[0100] In this embodiment, when extracting the skin bare area of the target object based on the skin color region of the target object, the skin bare area extraction network can also be pre-trained, the skin color region of the target object is input into the skin bare area extraction network, and the skin bare area of the target object output by the skin bare area extraction network is obtained.

[0101] In some embodiments, when training the skin bare area extraction network, a large number of sample objects can be set, the skin color region of the sample object is obtained as a training sample, and the position of the corresponding head region, hand region and other skin bare areas of each skin color region is marked as a training label. The skin bare area extraction network can also use a convolutional neural network model as a basic model, and its specific training process is the same as that of the skin bare area recognition model in the above embodiment. Therefore, the training process of the skin bare area extraction network can be referred to the training process of the skin bare area recognition model in the above embodiment, which will not be described here.

[0102] However, the skin bare area extraction network trained in the above manner is not only complex, but also has a large calculation amount and occupies a large space. If it cannot be run locally, it also needs certain networking conditions for cloud-terminal hybrid operation, which is not suitable for low-power chips. In order to solve the above problems, other ways can be used to train the skin bare area extraction network.

[0103] As an optional implementation, the skin bare area extraction network is obtained by joint training with the skin color region segmentation network. The joint training process includes:

[0104] The image of the sample object is taken as the first training sample, the position of the skin color region of the sample object in the image of the sample object is taken as the first training label, and the skin color region segmentation network is trained;

[0105] The output result obtained by inputting the image of the sample object into the skin color region segmentation network is used as a second training sample, and the position of the skin exposed region of the sample object in the image of the sample object is used as a second training label. The skin exposed region extraction network is trained.

[0106] Specifically, in the embodiments of the present application, the skin color region segmentation network and the skin exposed region extraction network can be jointly trained. After the training of the skin exposed region extraction network is completed, the skin color region segmentation network can be discarded.

[0107] In the joint training of the skin exposed region extraction network and the skin color region segmentation network, the skin color region segmentation network is first trained using the first training sample and the first training label. A large number of sample objects can be set, and the images of the sample objects are obtained as the first training sample, and the skin color regions in the images of the sample objects are marked as the first training label. In some embodiments, the skin color regions in the images of the sample objects include skin color regions and other regions that are not skin color. The specific training process is as follows:

[0108] The first training sample is input into the skin color region segmentation network to obtain the output result of the skin color region segmentation network. By comparing the first training label with the output result of the skin color region segmentation network, the loss value of the skin color region segmentation network is determined. The parameters of the skin color region segmentation network are adjusted to reduce the loss value of the skin color region segmentation network. Then the above training process is repeated until the loss value of the skin color region segmentation network is less than a set value, and the training is completed. The set value can be set according to actual conditions, and the present embodiment is not limited.

[0109] After the training of the skin color region segmentation network is completed, the output result of the skin color region segmentation network can be used to train the skin exposed region extraction network. Specifically, a large number of sample objects can be set, and the output result obtained by inputting the image of the sample object into the skin color region segmentation network is used as a second training sample, and the position of the skin exposed region such as the head region and the hand region in each second training sample is marked as a second training label.

[0110] The specific training process of the skin exposed region extraction network is the same as the training process of the skin color region segmentation network in the above embodiments. Therefore, the training process of the skin exposed region extraction network can be referred to the training process of the skin color region segmentation network in the above embodiments, which will not be described here.

[0111] It should be noted that in addition to the method of training the skin color region segmentation network first and then training the skin exposed region extraction network after the training of the skin color region segmentation network is completed as described in the above embodiments, the skin color region segmentation network and the skin exposed region extraction network can also be trained simultaneously, and the present embodiment is not limited.

[0112] In some embodiments, the skin color region segmentation network and the bare skin region extraction network are two parts of a convolutional neural network. For example... Figure 2 As shown, the convolutional neural network includes an encoder, a first decoder, a hidden layer, and a second decoder. The skin color region segmentation network includes the encoder, the first decoder, and the hidden layer of the convolutional neural network, while the skin exposed region extraction network includes the second decoder.

[0113] During training, the encoder, decoder, and hidden layer are first trained using the first training samples and first training labels, as described in the above embodiments. The output of the hidden layer is the output of the skin color region segmentation network. After the encoder, decoder, and hidden layer are trained, the sample object is input into the encoder, and the output of the hidden layer is used as the second training sample through processing by the encoder, decoder, and hidden layer. The positions of the face region, hand region, and other regions of the sample object in the image are used as the second training labels to train the second decoder.

[0114] Furthermore, the constraint equation L of this convolutional neural network all for:

[0115] L all =L pix +L skin

[0116] Among them, L pix To extract constraints for the network portion of the exposed skin area, L skin Constraints for segmenting the network portion for skin-colored regions. L pix L skin They are all in the same form, each representing a Euclidean distance minimization equation between its corresponding input and label graphs. For example, x represents the output result, and y represents the training label. It should be noted that both the output result and the training label can be pixel values.

[0117] After training, the hidden layer and the encoder and first decoder before it can be discarded, retaining only the second encoder. When segmenting exposed skin regions, the image containing the skin-colored region of the target object can be directly input into the second encoder, which then determines each exposed skin region of the target object based on the image containing that region.

[0118] In the above embodiments, a more lightweight skin exposed area extraction network can be obtained through joint training, which can not only improve the processing speed, but also enable the process to run on low-power chips such as embedded chips due to the generally simple structure of the skin exposed area extraction network, and the terminal can complete the entire operation independently due to the small amount of calculation without the assistance of the cloud.

[0119] In some embodiments, the skin color region of the target object can be identified from the image of the target object using a quadratic polynomial mixed skin color model, and then the image including the skin color region of the target object is processed using the second encoder trained in the above embodiments to obtain the specific position of the skin exposed area such as the face and hands of the target object. In this way, the advantages of low resource consumption and high efficiency of traditional skin color segmentation methods can be combined, and on the basis of skin color segmentation, pixel-level result optimization can be performed using deep learning, and a class label can be given for each pixel to facilitate effective differentiation between the hand region and / or face region in the skin exposed area.

[0120] It should be further noted that the training of the skin color region segmentation network in the present embodiment is easily confused with the training mode of directly inputting the skin color region of the sample object and outputting the skin exposed area of the sample object. In the embodiments of the present application, the hidden layer is only an approximation of skin color segmentation, which still implies the original information in the given original input image, so that multi-scale feature fusion is performed based on the original information. The original information has a more obvious effect on the final pixel-level classification in the training, which can ensure the effective convergence of the skin exposed area extraction network. At the same time, due to the approximation characteristics of the hidden layer, after obtaining the skin exposed area extraction network, the skin color segmentation result obtained by traditional methods such as the quadratic polynomial mixed skin color model can be directly replaced by the hidden layer to achieve the purpose of less effect loss.

[0121] After the skin exposed area of the target object is identified, some skin exposed areas may have holes, burrs, or other phenomena, which affect subsequent pose analysis. The skin exposed area of the target object can be processed using dilation, erosion, or other operations in image morphology, or using a template filling method to obtain a skin exposed area with clear and uniform edges.

[0122] In the embodiments of the present application, the skin color region segmentation network is provided with a class label of the skin exposed area, i.e., at least the hand region and / or face region in the skin exposed area can be labeled, therefore, based on the above-mentioned embodiments, it can be quickly determined whether the image of the target object includes a hand and a face, and if it is determined that the image of the target object includes a hand and a face, the specific position of the hand and the face can also be quickly determined.

[0123] In some embodiments, the skin exposed area includes a hand area and a face area, and the segmented face area is denoted as F, and the hand area is denoted as H. Then the overall feature G(x) = {F p ,H p ,f F (x),f H (x)} can be obtained, where F p and H p are position information of the face and the hand, expressed in the form of (x, y, w, h), and f F (x) and f H (x) are image features of the segmented face and hand area. In some embodiments, the image features can use Scale-invariant feature transform (SIFT) or Local binary patterns (LBP) features, and the present embodiment is not limited in this regard.

[0124] As an optional implementation, in another embodiment of the present application, the steps of the above embodiment determine the pose of the target object based on the position of the skin exposed area of the target object, and can specifically include the following steps:

[0125] According to the position of the skin exposed area of the target object, and the position of the skin exposed area corresponding to the set various pose types, the pose type of the target object is determined.

[0126] In the embodiment of the present application, the set various pose types and the position of the skin exposed area corresponding to the set various pose types are pre-stored. Therefore, by comparing the position of the skin exposed area of the target object with the position of the skin exposed area corresponding to the set various pose types, the pose type corresponding to the target position in the position of the skin exposed area corresponding to the set various pose types, which is closest to the position of the skin exposed area of the target object, can be determined as the pose type of the target object.

[0127] In the above embodiment, by constructing a database, the pose type of the target object is determined by comparing the position of the skin exposed area of the target object with the position of the skin exposed area corresponding to the set various pose types, which does not need to use a complex model for processing, has small calculation amount and high calculation efficiency, can run on a local low-power chip, meets the requirement of real-time, does not depend on cloud computing, and does not infringe on user privacy.

[0128] As an optional implementation, as Figure 3As shown, in another embodiment of the present application, the steps of the above embodiment are used to determine the posture type feature of the target object according to the position of the skin exposed area of the target object and the position of the skin exposed area corresponding to the set posture type, and can specifically include the following steps:

[0129] S301, determining the search feature of the skin exposed area of the target object according to the position of the skin exposed area of the target object and the image feature.

[0130] The image feature of the position of the skin exposed area of the target object can use SIFT or LBP features, and the present embodiment is not limited. The position of the skin exposed area of the target object and the image feature can be mapped to a search feature that can be searched.

[0131] In some embodiments, the position of the skin exposed area and the image feature are input into a pre-trained search feature mapping network to obtain the search feature of the skin exposed area of the target object output by the search feature mapping network, wherein the search feature mapping network is obtained by training a shallow neural network.

[0132] Specifically, a shallow neural network can be trained as a search feature mapping network based on metric learning, such as Figure 4 As shown. The constraint equation L of the loss function of the search feature mapping network is:

[0133] L = L cls + L simi + L center

[0134] Where L cls is a common cross-entropy loss; L simi is a similarity loss, commonly used contrastive loss or triplet loss, a typical definition is:

[0135]

[0136] Where a is an anchor sample, p is a different sample of the same class as a, and n is a sample of a different class from a; L center is a loss function for measuring the distance between the sample and the sample center cluster, and in the training process, the class center feature and the feature of the current sample are updated, but the addition of this loss can make the class center feature converge faster, and also provides a trainable and updateable basis for the subsequent use of the class center, wherein:

[0137]

[0138] As shown Figure 4As shown, through the shallow neural network, the original features G(x) of the position and image features of the skin exposed area of the target object can be mapped into features H(x) that can be used for fast retrieval.

[0139] Since the shallow neural network is a shallow neural network, its calculation cost is small, and its application range is more extensive. Even if it is run on a local low-power chip, it can meet the real-time requirements and does not rely on cloud computing, and does not infringe on user privacy.

[0140] In some embodiments, in addition to training the shallow neural network, other neural networks can also be selected to train as a retrieval feature mapping network, which is not limited in the present embodiment.

[0141] S302, according to the retrieval feature of the skin exposed area of the target object, retrieve the target posture type corresponding to the skin exposed area with the closest retrieval feature distance from the pre-set posture type library.

[0142] In the present embodiment, the posture type library is pre-set, and the posture type library stores a plurality of posture types and the retrieval features corresponding to the skin exposed areas of the plurality of posture types. Specifically, the present embodiment can set a large number of sample objects, and collect the retrieval features corresponding to the sample objects in different postures according to the above-mentioned embodiments.

[0143] In the retrieval, according to the retrieval feature of the skin exposed area of the target object, retrieve the skin exposed area with the closest retrieval feature distance from the pre-set posture type library, and determine the posture type corresponding to the skin exposed area with the closest distance as the posture type of the target object.

[0144] In some embodiments, the exposed area of the target object includes a face area and a hand area, a large number of sample objects can be set, a large amount of image data containing human faces and hands can be collected, and the specific positions of the face area and the hand area can be identified from the image of the target object according to the above-mentioned embodiments. Based on the specific positions of the face area and the hand area, posture labeling is performed to distinguish normal postures from abnormal postures and subdivide the corresponding abnormalities.

[0145] A tree structure is established to map and divide the abnormal postures into the tree. For example, first, a top node is established, such as a hand abnormality top node and a face abnormality top node; second, based on the top node, a second layer of subdivision nodes is obtained by subdivision, such as hand under the chin in a sitting position, which is established in the second layer of hand abnormalities, and head tilt is established in the second layer of face abnormalities, and similarly, the pre-defined abnormalities can be included in the tree, wherein the tree structure is preferably two layers, such as shown in the figure. Figure 5

[0146] ​For each pose type corresponding to each bottom node, the images of each pose type in the image of the sample object can be analyzed according to the above embodiment to obtain the retrieval features of each pose type. At the same time, the category center point generated in the retrieval feature mapping process of each pose type is determined as the representative feature of the corresponding bottom node.

[0147] Statistical analysis can then be performed on the tree structure. The distances between all samples and representative features within each bottom-level node are calculated, and the maximum tolerance distance is determined. The representative feature of the top-level node shared by multiple bottom-level nodes is the center point of those nodes. During subsequent retrieval, if the distance between the target object's search feature and each node exceeds the maximum tolerance distance, the target object's search feature can be ignored as interference.

[0148] During retrieval, for an image of a target object, the retrieval features of the image can be determined first. These features are then traversed and matched in a tree to determine the individual pose category. During the traversal and matching, a one-to-one match can be performed with upper-level nodes. If the distance between the target object's retrieval features and a representative feature of a certain upper-level node is less than the maximum tolerance distance of that node, then a one-to-one match can be performed with the lower-level nodes of that upper-level node. If the distance between the target object's retrieval features and a representative feature of a certain lower-level node of that upper-level node is less than the maximum tolerance distance of that lower-level node, and that lower-level node does not have any further lower-level nodes, then the pose type corresponding to that lower-level node can be determined as the target pose type. If the distance between the target object's retrieval features and a representative feature of a certain lower-level node of that upper-level node is less than the maximum tolerance distance of that lower-level node, and that lower-level node has further lower-level nodes, then the above steps can be followed to further traverse and compare the nodes one by one down to the next lower level, until the bottom-level nodes are reached to determine the target pose type.

[0149] For example, if the tree structure is Figure 5 The two-layer structure shown above, after obtaining the retrieval features of the target object, determines that the distance between the retrieval features of the target object and the representative node of the hand abnormal node is less than the maximum fault tolerance distance of the hand abnormal node. Then, the lower-level nodes of the hand abnormal node can be traversed. If it is detected that the distance between the retrieval features of the target object and the representative node of the hand chin rest node is less than the maximum fault tolerance distance of the hand chin rest node, then the target posture type can be determined to be hand chin rest.

[0150] S303. Determine the target posture type as the posture type of the target object.

[0151] In this embodiment, the target pose type is determined as the pose type of the target object.

[0152] In the above embodiments, the proposed discrimination method can greatly reduce the time consumption compared with the classical deep learning method, and is more suitable for the use scenarios of low-power embedded chips.

[0153] As an optional implementation, in another embodiment of the present application, the step of the above embodiment determines the posture of the target object based on the position of the skin exposed area of the target object relative to the specific reference target, and can specifically include the following steps:

[0154] According to the position of the skin exposed area of the target object, the relative position relationship between the skin exposed area of the target object and the set reference object is determined; according to the relative position relationship and the relative position relationship corresponding to each posture type set, the posture type of the target object is determined.

[0155] Specifically, the specific reference target refers to a pre-set reference target. Any position-fixed object in the image of the target object can be set as the specific reference target. For example, when monitoring the posture of the target object, the image of the target object often includes the image of a table and / or a chair, and the image of the table and / or the chair in the image of the target object can be set as the specific reference target, and the position of the image of the table and / or the chair is the position of the specific reference target. A coordinate system can also be established in the image of the target object, and the position corresponding to the set coordinate can be selected as the position of the specific reference target.

[0156] The position relationship of the skin exposed area of the target object relative to the specific reference target includes at least the distance relationship between the skin exposed area of the target object and the specific reference target, and the spatial position relationship between the skin exposed area of the target object and the specific reference target. Further, the relative position relationship between the skin exposed area of the target object and the set reference object includes at least one of the position relationship between the hand region of the target object and the set reference object, and the position relationship between the face region of the target object and the set reference object.

[0157] Among them, the distance interval between the skin exposed area of the target object and the specific reference target under different postures can be pre-set, the actual distance between the skin exposed area of the target object and the specific reference target is detected, the distance interval where the actual distance is located is determined, and the posture corresponding to the distance interval where the actual distance is located is determined as the current posture of the target object.

[0158] For example, if the posture of the sitting posture of the target object is monitored, the skin exposed area includes the face area of the target object, and the specific reference target includes a table, the distance between the face area of the target object and the tabletop can be set to be less than X1, the posture of the target object includes "bending", the distance between the face area of the target object and the tabletop is set to be between X1 and X2, the posture of the target object includes "normal distance from the tabletop", and the distance between the face area of the target object and the tabletop is greater than X2, the posture of the target object includes "face up". If the actual distance between the face area of the target object and the tabletop is X3, and X3 is between X1 and X2, it can be determined that the posture of the target object is "normal distance from the tabletop".

[0159] For example, if the posture of the sitting posture of the target object is monitored, the skin exposed area includes the face area of the target object, and the specific reference target includes a table, the distance between the face area of the target object and the tabletop can be set to be less than X1, the posture of the target object includes "bending", the distance between the face area of the target object and the tabletop is set to be between X1 and X2, the posture of the target object includes "normal distance from the tabletop", and the distance between the face area of the target object and the tabletop is greater than X2, the posture of the target object includes "face up". If the actual distance between the face area of the target object and the tabletop is X3, and X3 is between X1 and X2, it can be determined that the posture of the target object is "normal distance from the tabletop".

[0160] The reference spatial position relationship between the skin exposed area of the target object and the specific reference target in different postures can be set in advance, the actual spatial position relationship between the skin exposed area of the target object and the specific reference target is detected, the target reference spatial position relationship identical to the actual spatial position relationship is selected from the reference spatial position relationship, and the posture corresponding to the target reference spatial position relationship is determined as the current posture of the target object.

[0161] For example, if the posture of the sitting posture of the target object is monitored, the skin exposed area includes the face area of the target object, and the specific reference target includes a table, the distance between the face area of the target object and the tabletop can be set to be less than X1, the posture of the target object includes "bending", the distance between the face area of the target object and the tabletop is set to be between X1 and X2, the posture of the target object includes "normal distance from the tabletop", and the distance between the face area of the target object and the tabletop is greater than X2, the posture of the target object includes "face up". If the actual distance between the face area of the target object and the tabletop is X3, and X3 is between X1 and X2, it can be determined that the posture of the target object is "normal distance from the tabletop".

[0162] It should be noted that if the posture of the target object is monitored by determining the posture of the target object according to the position of the skin exposed area of the target object relative to the specific reference object, the position of the specific reference object needs to be determined before starting the monitoring. For example, if a table is set as the specific reference object, the position of the table needs to be determined before starting the posture monitoring of the target object. The process can be performed by using motion foreground segmentation, Gaussian mixture model or VIBE, and those skilled in the art can refer to the prior art, which will not be repeated here.

[0163] Moreover, if the position of the specific reference object is fixed and does not change, the position of the specific reference object can be determined when the posture monitoring is first performed and stored in the local or cloud storage. When the posture monitoring is performed subsequently, the pre-stored position of the specific reference object can be called when the monitoring is started.

[0164] In the above embodiments, the posture of the target object can be quickly determined by the relative position relationship between the skin exposed area of the target object and the set reference object, without using complex models for processing, with small calculation amount and high calculation efficiency. Moreover, the method can be run on a local low-power chip to meet the real-time requirement and does not depend on cloud computing, without infringing the privacy of the user.

[0165] As an optional implementation, the method of the above embodiments can specifically include the following steps in another embodiment of the present application.

[0166] outputting the posture of the target object; or

[0167] If the posture of the target object belongs to the set non-standard posture, the posture of the target object is outputted and / or a posture correction reminder is outputted.

[0168] Specifically, in the embodiments of the present application, the posture type of the target object can be directly outputted, so that the target object can be corrected in time when the target object knows that the posture is abnormal. Alternatively, when the posture of the target object is abnormal, i.e., in the set non-standard posture, for example, the target object is in a sitting posture, and the set “hand under the cheek” and “bending over a desk” postures are non-standard postures, the posture of the target object and / or a posture correction reminder can be outputted, so that the target object can be corrected in time.

[0169] The non-standard posture can be set according to the actual situation, and the embodiments are not limited. In the above embodiments, only the position of the skin exposed area corresponding to the non-standard posture and the relative position relationship corresponding to the non-standard posture can be stored in the database, so that the non-standard posture can be detected in time and the reminder information can be outputted during the posture monitoring, while the information in the database is reduced and the running efficiency is improved.

[0170] In the above embodiments, the posture of the target object can be output, and / or a posture correction reminder can be output, so as to timely remind the target object to adjust the sitting posture.

[0171] In some embodiments, long-time posture abnormalities of the target object, such as fatigue and looking around, can also be monitored. Specifically, each frame of image of the target object can be detected, and each frame of image can be scored. If the scores of multiple frames of image fall within a threshold interval of a long-time posture abnormality, it can be determined that the posture of the target object is the long-time posture abnormality. In addition, the long-time posture abnormality can also be monitored by using a deep learning network through time series modeling, which is not limited in the embodiment. It should be noted that, since the long-time abnormality itself needs a certain observation time, even if a low-power chip is used for low-efficiency processing, the user tolerance can still be obtained.

[0172] In some embodiments, the posture monitoring method of the embodiment can be applied to the scene of student sitting posture monitoring. Based on the method, student sitting posture monitoring and reminding can be realized without networking and without infringing on user privacy.

[0173] Exemplary device

[0174] Corresponding to the above posture monitoring method, the embodiment of the application further discloses a posture monitoring device, as shown in Figure 6 The device comprises:

[0175] The recognition module 100 is configured to recognize a skin exposed area of the target object from the image of the target object. The skin exposed area of the target object comprises a hand area and / or a face area of the target object.

[0176] The determination module 110 is configured to determine the posture of the target object based on the position of the skin exposed area of the target object or the position of the skin exposed area of the target object relative to a specific reference target.

[0177] As an optional implementation, in another embodiment of the application, the recognition module 100 of the above embodiment can specifically comprise:

[0178] The recognition unit is configured to recognize a skin color area of the target object from the image of the target object.

[0179] The first determination unit is configured to determine the skin exposed area of the target object based on the skin color area of the target object.

[0180] As an optional implementation, in another embodiment of the present application, the identification unit of the above embodiment, when identifying the skin color region of the target object from the image of the target object, is specifically used for:

[0181] Detecting a target region with a color within a set skin color interval from the image of the target object as the skin color region of the target object.

[0182] As an optional implementation, in another embodiment of the present application, the identification unit of the above embodiment, when determining the skin exposed region of the target object based on the skin color region of the target object, is specifically used for:

[0183] Inputting the skin color region of the target object into a pre-trained skin exposed region extraction network to obtain the skin exposed region of the target object output by the skin exposed region extraction network.

[0184] As an optional implementation, in another embodiment of the present application, the skin exposed region extraction network is obtained by joint training with the skin color region segmentation network; the apparatus of the above embodiment further comprises:

[0185] A joint training module configured to train the skin color region segmentation network by taking an image of a sample object as a first training sample, and taking a position of a skin color region of the sample object in the image of the sample object as a first training label;

[0186] Taking an output result obtained by inputting the image of the sample object into the skin color region segmentation network as a second training sample, and taking a position of a skin exposed region of the sample object in the image of the sample object as a second training label, the skin exposed region extraction network is trained.

[0187] As an optional implementation, in another embodiment of the present application, the determination module 110 of the above embodiment, when determining the pose of the target object based on the position of the skin exposed region of the target object or the position of the skin exposed region of the target object relative to a specific reference target, is specifically used for:

[0188] Determining the pose type of the target object according to the position of the skin exposed region of the target object and the positions of the skin exposed regions corresponding to various set pose types.

[0189] As an optional implementation, in another embodiment of the present application, the determination module 110 of the above embodiment comprises:

[0190] A second determination unit configured to determine a search feature of the skin exposed region of the target object according to the position of the skin exposed region of the target object and the image feature.

[0191] The searching unit is configured to search, according to the search feature of the skin exposed area of the target object, a target posture type corresponding to a skin exposed area closest to the search feature of the skin exposed area of the target object from a preset posture type library, wherein the posture type library stores a plurality of posture types and search features corresponding to skin exposed areas of the plurality of posture types.

[0192] The third determining unit is configured to determine that the target posture type is the posture type of the target object.

[0193] As an optional implementation, in another embodiment of the present application, the second determining unit of the above embodiment, when determining the search feature of the skin exposed area of the target object according to the position and image feature of the skin exposed area of the target object, is specifically configured to:

[0194] input the position and image feature of the skin exposed area into a pre-trained search feature mapping network to obtain the search feature of the skin exposed area of the target object output by the search feature mapping network; wherein the search feature mapping network is obtained by training a shallow neural network.

[0195] As an optional implementation, in another embodiment of the present application, the determining module 110 of the above embodiment comprises:

[0196] The fourth determining unit is configured to determine, according to the position of the skin exposed area of the target object, a relative position relationship between the skin exposed area of the target object and a set reference object, wherein the relative position relationship between the skin exposed area of the target object and the set reference object comprises at least one of a position relationship between a hand region of the target object and the set reference object and a position relationship between a face region of the target object and the set reference object.

[0197] The fifth determining unit is configured to determine, according to the relative position relationship and a relative position relationship corresponding to each set posture type, the posture type of the target object.

[0198] As an optional implementation, in another embodiment of the present application, the device of the above embodiment further comprises:

[0199] The output module is configured to output the posture of the target object; or

[0200] If the posture of the target object belongs to a set non-standard posture, the posture of the target object is output and / or a posture correction reminder is output.

[0201] Specifically, the specific working content of each unit of the posture monitoring device described above can be seen from the content of the method embodiments described above, and will not be repeated here.

[0202] Exemplary electronic device, posture monitoring system, storage medium and computing program product

[0203] Corresponding to the posture monitoring method described above, the embodiments of the present application also disclose an electronic device, which can be seen from Figure 7 The electronic device includes:

[0204] a memory 200 and a processor 210;

[0205] The memory 200 is connected with the processor 210, and is configured to store programs.

[0206] The processor 210 is configured to realize the posture monitoring method disclosed in any of the embodiments above by running the programs stored in the memory 200.

[0207] Specifically, the electronic device described above can further include a bus, a communication interface 220, an input device 230 and an output device 240.

[0208] The processor 210, the memory 200, the communication interface 220, the input device 230 and the output device 240 are connected with each other through the bus. Among them:

[0209] The bus can include a path for transmitting information between various components of the computer system.

[0210] The processor 210 can be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or can be an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of programs of the present application. It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a ready-to-use programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0211] The processor 210 can include a main processor, and can also include a baseband chip, a modem, etc.

[0212] The memory 200 stores programs for implementing the technical solutions of the present application, and can also store operating systems and other key services. Specifically, the programs can include program codes, and the program codes include computer operation instructions. More specifically, the memory 200 can include a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM), other types of dynamic storage devices that can store information and instructions, a disk memory, a flash, and the like.

[0213] The input device 230 can include devices that receive data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer, a gravity sensor, and the like.

[0214] The output device 240 can include devices that allow information to be output to a user, such as a display screen, a printer, a speaker, and the like.

[0215] The communication interface 220 can include devices of the transceiver type or the like for communicating with other devices or communication networks, such as an Ethernet, a radio access network (RAN), a wireless local area network (WLAN), and the like.

[0216] The processor 210 executes programs stored in the memory 200 and calls other devices, which can be used to implement the steps of the posture monitoring method provided by the above-mentioned embodiments of the present application.

[0217] Another embodiment of the present application also provides a posture monitoring system. Referring to FIG. 3, Figure 7 The posture monitoring system includes a control device 300 and a camera device 310 in communication connection;

[0218] The camera device 310 is configured to acquire an image of a target object;

[0219] The control device 300 is configured to identify a skin exposed area of the target object from the image of the target object; the skin exposed area of the target object includes a hand region and / or a face region of the target object; and determine a posture of the target object based on a position of the skin exposed area of the target object or a position of the skin exposed area of the target object relative to a specific reference target.

[0220] The posture monitoring system provided by the embodiment is of the same application concept as the posture monitoring method provided by the embodiments of the application, can execute the posture monitoring method provided by any of the embodiments of the application, and has the function modules and technical details not described in detail in the embodiment that are corresponding to the posture monitoring method. For the specific processing content of the posture monitoring method provided by the embodiments of the application, refer to the posture monitoring method provided by the embodiments of the application, which will not be described here.

[0221] In some embodiments, the posture monitoring system can be a smart phone, a tablet computer, a learning tablet, or the like, and can also be a learning clock, a learning desk, or the like provided with a control chip. The embodiment is not limited in this regard.

[0222] In addition to the above method and device, the embodiments of the application can also be a computer program product, which includes computer program instructions. When the computer program instructions are run by the processor 210, the processor 210 executes the steps of the posture monitoring method provided by the embodiments of the application.

[0223] The computer program product can be written in any combination of one or more programming languages to perform the operations of the embodiments of the application, including object-oriented programming languages such as Java, C++, and conventional procedural programming languages such as "C" language or similar programming languages. The program code can be executed entirely on a user computing device, partially on a user device, as a separate software package, partially on a user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0224] In addition, the embodiments of the application can also be a computer readable storage medium, which stores computer program instructions. When the computer program instructions are run by the processor, the processor 210 executes the steps of the posture monitoring method provided by the embodiments of the application.

[0225] The computer readable storage medium can use any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium may, for example, include but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0226] For each method embodiment described above, for the sake of simplicity, the method embodiments are described as a series of acts. But those skilled in the art will understand that the method embodiments are not limited by the order of acts, as some steps can occur in other orders or concurrently with each other. Moreover, those skilled in the art will appreciate that the embodiments described herein are exemplary only and not limiting, as the scope of the application is not to be determined by the embodiments disclosed herein, but instead by the claims below.

[0227] It should be noted that each of the above-described embodiments can be implemented in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between embodiments can be mutually referred to. For the device embodiments, since they are basically similar to the method embodiments, they are described more simply, and the relevant parts are referred to the part of the method embodiments.

[0228] The steps in the method embodiments of the application can be adjusted in order, combined, and reduced according to actual needs. The technical features recorded in the embodiments can be replaced or combined.

[0229] The modules and sub-modules in the devices and terminals in the embodiments of the application can be combined, divided, and reduced according to actual needs.

[0230] In several embodiments provided by the application, it should be understood that the disclosed terminal, device, and method can be implemented by other ways. For example, the terminal embodiments described above are only schematic, and the division of modules or sub-modules is only a logical function division. In actual implementation, another division mode can be adopted, for example, a plurality of sub-modules or modules can be combined or integrated into another module, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed mutual elements can be indirect coupling or communication connection through some interfaces, devices, or modules, which can be electrical, mechanical, or other forms.

[0231] The modules or sub-modules described as separate components can or can not be physically separated, and the components of the modules or sub-modules can or can not be physical modules or sub-modules, that is, they can be located in one place, or can be distributed on a plurality of network modules or sub-modules. According to actual needs, some or all of the modules or sub-modules can be selected to achieve the purpose of the embodiment.

[0232] In addition, each functional module or sub-module in each embodiment of the present application can be integrated in one processing module, or each module or sub-module can exist physically alone, or two or more modules or sub-modules can be integrated in one module. The integrated module or sub-module can be realized in the form of hardware or in the form of a software functional module or sub-module.

[0233] Those skilled in the art will further appreciate that the units and algorithm steps of the examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or any combination thereof. To clearly illustrate the interchangeability of hardware and software, various examples have been described generally in terms of their functionality, without limitation to the corresponding description in any particular combination of hardware and software. Those skilled in the art will recognize many modifications and variations of this example that are apparent in light of the above teachings. It is therefore intended that such modifications and variations not depart from the scope of the application.

[0234] The steps of a method or algorithm described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0235] Finally, it should be noted that the terms "first" and "second" and the like are used merely to distinguish one element from another, and do not necessarily indicate a physical or chronological order. Furthermore, the terms "comprise", "include", or their variants, mean "consisting of" or "consisting essentially of", so that when these terms are used, the processes, methods, articles, or apparatuses include not only those elements expressly listed, but also other elements inherent in such processes, methods, articles, or apparatuses. Without more limitations, an element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0236] The foregoing description of the disclosed embodiments enables a person skilled in the art to make or use the application. Modifications of these embodiments will occur to persons of skill in the art, and that the generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Therefore, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method of posture monitoring, characterized by, The method comprises: identifying a skin exposed area of a target object from an image of the target object; the skin exposed area of the target object comprises a hand area and / or a face area of the target object; based on a position of the skin exposed area of the target object relative to a specific reference target, determining a posture monitoring result of the target object, the posture monitoring result comprising that a posture of the target object belongs to a set non-standard posture; wherein, identifying the skin exposed area of the target object from the image of the target object comprises: identifying a skin color area of the target object from the image of the target object; based on the skin color area of the target object, determining the skin exposed area of the target object by a pre-trained skin exposed area extraction network; wherein, identifying the skin color area of the target object from the image of the target object comprises: detecting a target area with a color within a set skin color interval from the image of the target object as the skin color area of the target object; the skin exposed area extraction network is obtained by joint training with a skin color area segmentation network; the skin color area segmentation network comprises an encoder, a first decoder and a hidden layer, and the skin exposed area extraction network comprises a second decoder; the joint training process comprises: taking an image of a sample object as a first training sample, a position of a skin color area of the sample object in the image of the sample object as a first training label, and training the encoder, the first decoder and the hidden layer in the skin color area segmentation network; after the training of the skin color area segmentation network is completed, taking an output result obtained by processing the image of the sample object through the encoder, the first decoder and the hidden layer as a second training sample, a position of a skin exposed area of the sample object in the image of the sample object as a second training label, and training the second decoder in the skin exposed area extraction network; after the training of the second decoder is completed, discarding the encoder, the first decoder and the hidden layer, and retaining the second decoder to constitute the skin exposed area extraction network.

2. The method of claim 1, wherein, based on the skin color area of the target object, determining the skin exposed area of the target object by a pre-trained skin exposed area extraction network comprises: inputting the skin color area of the target object into the pre-trained skin exposed area extraction network to obtain the skin exposed area of the target object output by the skin exposed area extraction network.

3. The method of claim 1, wherein, The method further comprises: based on the position of the skin exposed area of the target object, determining the posture monitoring result of the target object; wherein, based on the position of the skin exposed area of the target object, determining the posture monitoring result of the target object comprises: determining a posture type of the target object according to the position of the skin exposed area of the target object and positions of skin exposed areas corresponding to various posture types set in advance.

4. The method of claim 3, wherein, determining a posture type of the target object according to the position of the skin exposed area of the target object and positions of skin exposed areas corresponding to various posture types set in advance comprises: According to the position and image features of the skin exposed area of the target object, a search feature of the skin exposed area of the target object is determined; According to the search feature of the skin exposed area of the target object, a target posture type corresponding to the search feature of the skin exposed area of the target object that is closest to the search feature of the skin exposed area of the target object is searched from a pre-set posture type library; the posture type library stores a plurality of posture types and search features of skin exposed areas corresponding to the plurality of posture types; The target posture type is determined as the posture type of the target object.

5. The method of claim 4, wherein, According to the position and image features of the skin exposed area of the target object, a search feature of the skin exposed area of the target object is determined, comprising: The position and image features of the skin exposed area are input into a pre-trained search feature mapping network to obtain the search feature of the skin exposed area of the target object output by the search feature mapping network; wherein the search feature mapping network is obtained by training a shallow neural network.

6. The method of claim 1, wherein, Based on the position of the skin exposed area of the target object relative to a specific reference target, the posture of the target object is determined, comprising: According to the position of the skin exposed area of the target object, the relative position relationship between the skin exposed area of the target object and a set reference object is determined; the relative position relationship between the skin exposed area of the target object and the set reference object includes at least one of the position relationship between the hand region of the target object and the set reference object and the position relationship between the face region of the target object and the set reference object; According to the relative position relationship and the relative position relationship corresponding to various posture types set, the posture type of the target object is determined.

7. The method of claim 1, wherein, Further comprising: outputting the posture monitoring result of the target object; or if the posture monitoring result of the target object belongs to a set non-standard posture, outputting the posture monitoring result of the target object and / or outputting a posture correction reminder.

8. A posture monitoring apparatus characterized by comprising: Comprising: an identification module configured to identify a skin exposed area of a target object from an image of the target object; The skin exposed area of the target object includes a hand region and / or a face region of the target object; a determination module configured to determine a posture monitoring result of the target object based on the position of the skin exposed area of the target object relative to a specific reference target, the posture monitoring result including whether the posture of the target object belongs to a set non-standard posture; wherein the identification module identifies the skin exposed area of the target object from the image of the target object, comprising: identifying a skin color region of the target object from the image of the target object; determining the skin exposed area of the target object based on the skin color region of the target object by a pre-trained skin exposed area extraction network; wherein the identification module identifies the skin color region of the target object from the image of the target object, comprising: detecting a target region with a color within a set skin color interval from the image of the target object as the skin color region of the target object; The skin bare area extraction network is obtained through joint training with a skin color area segmentation network; the skin color area segmentation network comprises an encoder, a first decoder and a hidden layer, and the skin bare area extraction network comprises a second decoder; the joint training process comprises: taking an image of a sample object as a first training sample, a position of a skin color area of the sample object in the image of the sample object as a first training label, and training the encoder, the first decoder and the hidden layer in the skin color area segmentation network; after the training of the skin color area segmentation network is completed, taking an output result obtained by processing the image of the sample object through the encoder, the first decoder and the hidden layer as a second training sample, a position of a skin bare area of the sample object in the image of the sample object as a second training label, and training the second decoder in the skin bare area extraction network; after the training of the second decoder is completed, discarding the encoder, the first decoder and the hidden layer, and retaining the second decoder to constitute the skin bare area extraction network.

9. An electronic device, comprising: comprise: a memory and a processor; wherein the memory is configured to store a program; the processor is configured to implement the method according to any one of claims 1 to 7 by running the program in the memory.

10. A posture monitoring system, characterized by comprise a control device and a camera device connected in communication; the camera device is configured to acquire an image of a target object; the control device is configured to identify a skin bare area of the target object from the image of the target object; the skin bare area of the target object comprises a hand region and / or a face region of the target object; based on a position of the skin bare area of the target object relative to a specific reference target, a posture monitoring result of the target object is determined, the posture monitoring result comprising that a posture of the target object belongs to a set non-standard posture; wherein identifying the skin bare area of the target object from the image of the target object comprises: identifying a skin color area of the target object from the image of the target object; determining the skin bare area of the target object based on the skin color area of the target object through a pre-trained skin bare area extraction network; wherein identifying the skin color area of the target object from the image of the target object comprises: detecting a target region with a color within a set skin color interval from the image of the target object as the skin color area of the target object; The skin bare area extraction network is obtained through joint training with a skin color area segmentation network; the skin color area segmentation network comprises an encoder, a first decoder and a hidden layer, and the skin bare area extraction network comprises a second decoder; the joint training process comprises: taking an image of a sample object as a first training sample, a position of a skin color area of the sample object in the image of the sample object as a first training label, and training the encoder, the first decoder and the hidden layer in the skin color area segmentation network; After the skin color region segmentation network is trained, an output result obtained by processing the image of the sample object through the encoder, the first decoder and the hidden layer is taken as a second training sample, a position of a skin bare region of the sample object in the image of the sample object is taken as a second training label, and the second decoder in the skin bare region extraction network is trained. After the second decoder is trained, the encoder, the first decoder and the hidden layer are discarded, and the second decoder is reserved to constitute the skin bare region extraction network.

11. A storage medium, characterized by The storage medium has a computer program stored thereon, and the computer program is executed by the processor to implement the method in any one of claims 1 to 7.

Images