Posture assessment device, method and system

Abstract

Embodiments of the present application provide a posture evaluation device, method and system. An image capturing device is used to capture an image. Key points in the image are identified. The key points correspond to a plurality of positions of a subject in the image and include a first group of key points and a second group of key points. The positions of the first group of key points are obtained and a first geometric relationship is defined via the association of the first group of key points. The relative positions between the subject and the image capturing device are determined based on the first geometric relationship or the number of key points. Each of the relative positions has a corresponding preset geometric relationship. A second geometric relationship is formed via the association of the second group of key points and the preset geometric relationship, and a comparison result is generated. In this way, the identification accuracy and efficiency can be improved.

Description

Technical Field

[0001] This invention relates to an image recognition technology, and more particularly, to an image-based posture assessment device, method, and system. Background Technology

[0002] Poor posture can lead to muscle or bone pain or other negative effects. For example, prolonged sitting with a hunched back can cause stiff back muscles, weakness in the abdomen or buttocks, and tightness in the hip joints. Although current technology can use ultrasound ranging to detect the depth of body parts and estimate posture, the results are rather rough and inaccurate. Therefore, the assessment of human posture still needs improvement. Summary of the Invention

[0003] The present invention relates to a posture assessment device, method and system, which can determine posture based on the positional relationship between key points in an image, thereby improving the accuracy and efficiency of estimation.

[0004] According to embodiments of the present invention, the posture evaluation method includes (but is not limited to) the following steps: capturing a test image using an image capture device; identifying several key points in the test image; these key points corresponding to several positions of a subject in the test image; these key points including a first group of key points and a second group of key points; obtaining the positions of the first group of key points and defining a first geometric relationship through the association of the first group of key points; determining the relative position between the subject and the image capture device based on the first geometric relationship; each relative position having a corresponding preset geometric relationship; comparing the preset geometric relationship with a second geometric relationship formed by the second group of key points, and generating a comparison result.

[0005] According to an embodiment of the present invention, the posture evaluation device includes (but is not limited to) a memory and a processor. The memory stores program code. The processor is coupled to the memory. The processor loads and executes the program code to be configured to identify several key points in an image under test, obtain the positions of a first group of key points, define a first geometric relationship through the association of the first group of key points, determine the relative position between the subject and the image capture device based on the first geometric relationship, compare a preset geometric relationship with a second geometric relationship formed by the association of a second group of key points, and generate a comparison result. The key points correspond to several positions of the subject in the image under test, including the first group of key points and the second group of key points. Each relative position has a corresponding preset geometric relationship.

[0006] According to an embodiment of the present invention, a posture evaluation system includes (but is not limited to) an image capture device, a memory, and a processor. The image capture device acquires an image to be tested. The memory stores program code. The processor is coupled to the memory and the image capture device. The processor loads and executes the program code, configured to identify several key points in the image to be tested, obtain the positions of a first group of key points, define a first geometric relationship through the association of the first group of key points, determine the relative position between the subject and the image capture device based on the first geometric relationship, compare a preset geometric relationship with a second geometric relationship formed by the association of a second group of key points, and generate a comparison result. The key points correspond to several positions of the subject in the image to be tested, including the first group of key points and the second group of key points. Each relative position has a corresponding preset geometric relationship.

[0007] Based on the above, the posture assessment apparatus, method, and system according to embodiments of the present invention determine the relative position between the subject and the image capture device based on the positional relationship of the subject's key points in the image to be tested, and determine whether the subject is in a normal posture based on the corresponding relative position. This improves the accuracy of posture assessment and enhances recognition efficiency. Attached Figure Description

[0008] The accompanying drawings are included to further illustrate the invention, and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.

[0009] Figure 1 This is a block diagram of components of a posture evaluation system according to an embodiment of the present invention;

[0010] Figure 2 This is a flowchart of a posture evaluation method according to an embodiment of the present invention;

[0011] Figures 3A to 3B This is a schematic diagram of a subject being photographed according to an embodiment of the present invention;

[0012] Figure 3C This is a schematic diagram showing the orientation of the main body according to an embodiment of the present invention;

[0013] Figure 4 This is a schematic diagram of key point markings on the front according to an embodiment of the present invention;

[0014] Figure 5 This is a schematic diagram of key point markings on a slanted side according to an embodiment of the present invention;

[0015] Figure 6 This is a schematic diagram of key point markings on the side according to an embodiment of the present invention;

[0016] Figure 7This is a schematic diagram illustrating key point markings in various reading scenarios according to an embodiment of the present invention;

[0017] Figure 8A and Figure 8B This is a schematic diagram of frontal recognition according to an embodiment of the present invention;

[0018] Figure 9A and Figure 9B This is a schematic diagram of oblique side recognition according to an embodiment of the present invention;

[0019] Figure 10 This is a schematic diagram of frontal recognition according to another embodiment of the present invention;

[0020] Figure 11 This is a schematic diagram of side recognition according to another embodiment of the present invention;

[0021] Figure 12 This is a schematic diagram of normal posture recognition from the front according to an embodiment of the present invention;

[0022] Figure 13 This is a schematic diagram of normal posture recognition from the front according to another embodiment of the present invention;

[0023] Figure 14A and Figure 14B This is a schematic diagram of normal posture recognition from the front according to another embodiment of the present invention;

[0024] Figure 15 This is a schematic diagram of normal posture recognition from the front according to another embodiment of the present invention;

[0025] Figure 16A and Figure 16B This is a schematic diagram of normal posture recognition from a slanted side according to an embodiment of the present invention;

[0026] Figure 17A and Figure 17B This is a schematic diagram of normal posture recognition on an oblique side according to another embodiment of the present invention;

[0027] Figure 18 This is a schematic diagram of normal posture recognition on the side according to an embodiment of the present invention;

[0028] Figure 19A and Figure 19B This is a schematic diagram of normal posture recognition on the side according to another embodiment of the present invention;

[0029] Figure 20 This is a flowchart illustrating an alarm based on an embodiment of the present invention.

[0030] Explanation of icon numbers

[0031] 1: Posture assessment system;

[0032] 100: Posture assessment device;

[0033] 110: Memory;

[0034] 130: Processor;

[0035] 150: Image capture device;

[0036] S210~S290, S201~S205: Steps;

[0037] θ1~θ3, θ7, θ8, θ10, θ11, θ13: angle;

[0038] B1~B4: Main body;

[0039] HP: Horizontal plane;

[0040] VT: Vertical plane;

[0041] P1: Neck;

[0042] P2-1, P2-2: Shoulders;

[0043] P3-1, P3-2: Ears;

[0044] P4-1, P4-2: Eyes;

[0045] P5: Nose;

[0046] IM1~IM3: Images to be tested;

[0047] S1~S15: Samples;

[0048] x, y: axes;

[0049] RH1, RH2, RH3, RH4: Second line;

[0050] h1, h2, h3, h4: Spacing;

[0051] w1, w2, h5, w3, h6: Distance;

[0052] Connect LE, LE2, and LE3.

[0053] θ5, θ6, θ9, θ12, θ16: included angle;

[0054] RH5: Fourth line;

[0055] b: Angular tolerance. Detailed Implementation

[0056] Reference will now be made in detail to exemplary embodiments of the invention, examples of which are illustrated in the accompanying drawings. Wherever possible, the same component symbols are used in the drawings and description to denote the same or similar parts.

[0057] Figure 1 This is a block diagram of the components of a posture evaluation system 1 according to an embodiment of the present invention. Please refer to... Figure 1 The posture assessment system 1 includes (but is not limited to) a posture assessment device 100 and an image capture device 150.

[0058] The posture assessment device 100 includes a memory 110 and a processor 130. The posture assessment device 100 can be a desktop computer, laptop computer, smartphone, tablet computer, server, medical testing instrument, smart desk lamp, smart office / reading device or other computing device.

[0059] The memory 110 can be any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk drive (HDD), solid-state drive (SSD), or similar component. In one embodiment, the memory 110 is used to store program code, software modules, configuration settings, data, or files.

[0060] Processor 130 is coupled to memory 110. Processor 130 may be a central processing unit (CPU), a graphics processing unit (GPU), or other programmable general-purpose or special-purpose microprocessor, digital signal processor (DSP), programmable controller, field-programmable gate array (FPGA), application-specific integrated circuit (ASIC), neural network accelerator, or other similar components or combinations thereof. In one embodiment, processor 130 is used to perform all or part of the operations of posture evaluation device 100, and may load and execute various program codes, software modules, files, and data stored in memory 110.

[0061] The image capture device 150 may be a camera, video camera, monitor, or a similar device. In one embodiment, the image capture device 150 may be built into or external to the posture evaluation device 100 body.

[0062] In one embodiment, the image capture device 150 may include components such as an image sensor (e.g., a charge-coupled device (CCD), complementary metal-oxide-semiconductor (CMOS), etc.), an optical lens, an image control circuit, and an image processor. In some embodiments, the lens specifications (e.g., aperture, magnification, focal length, viewing angle, image sensor size, etc.) and their quantity of the image capture device 150 may be adjusted according to actual needs. For example, the image capture device 150 includes a fisheye lens, and its image processor or processor 130 can unfold the fisheye image captured by the fisheye lens into a panoramic image. As another example, the image capture device 150 includes a wide-angle lens, and its image processor or processor 130 can correct distortion in the image captured by the image sensor.

[0063] The methods described in this embodiment of the invention will be explained below in conjunction with the various devices, components, and modules in the posture assessment system 1. The various processes of this method may be adjusted according to the implementation situation, and are not limited thereto.

[0064] Figure 2 This is a flowchart of a posture evaluation method according to an embodiment of the present invention. Please refer to... Figure 2 The processor 130 may use the image capture device 150 to capture the image to be tested (step S210). Specifically, the image to be tested is an image of the subject captured by the image capture device 150 of the pose evaluation device 100 or other external image capture device. In one embodiment, the subject is a human body. In one embodiment, the image to be tested is of the upper body of the subject (e.g., the waist, shoulders, or above the chest).

[0065] For example, Figures 3A to 3B This is a schematic diagram of the main body according to an embodiment of the present invention. Please refer to... Figure 3A The image capture device 150 is placed on a horizontal plane HP (e.g., a desktop), and the height of the horizontal plane HP is approximately between the chest and waist of the subject B1. The image capture device 150 may be configured with a fisheye lens to capture the upper body of the subject B1.

[0066] Please refer to Figure 3BThe image capture device 150 is placed on a vertical plane VT (e.g., a wall), and the height of the image capture device 150 is approximately at the head of the subject B1. At this time, the image capture device 150 can be configured with a wide-angle lens or even a standard lens to capture the upper body of the subject B1.

[0067] Figure 3C This is a schematic diagram showing the orientation of the main body according to an embodiment of the present invention. Please refer to... Figure 3C Assuming the image capture device 150 is located on the desktop (e.g., Figure 3A As shown), the image capture device 150 has a field of view covering at least 180 degrees. Subjects B2, B3, and B4 are all within the field of view of the image capture device 150 at angles θ1 (e.g., 90 degrees), θ2 (e.g., 45 degrees), and θ3 (e.g., 180 degrees), respectively. In some situations, the field of view of the image capture device 150 may be affected by the table surface where the subjects are located, and only the upper body of subjects B4, B3, and B4 may be captured. In other situations, the image capture device 150 may also be configured to face a specific angle so that its field of view only covers the shoulders or other specific parts of subjects B4, B3, and B4.

[0068] It should be noted that, Figure 3C The relative positions (e.g., angles θ1, θ2, θ3) of the main bodies B2, B3, B4 and the image capture device 150 shown are merely illustrative examples. In reality, there may be other relative positions, and this example is not intended to limit the embodiments of the present invention.

[0069] In some embodiments, the processor 130 can segment the image to be tested and obtain specific parts of the subject. These parts include, for example, the head, neck, and shoulders. That is, when the original image to be tested is of the full body, three-quarter body, or other body proportions of the subject, the processor 130 can obtain the upper body or other body parts of the subject B1 to B4 by segmenting the original image to be tested.

[0070] It should be noted that the location of the image capture device 150 may vary depending on the actual situation. In some embodiments, the content of the image to be captured or a specific part of the subject may be changed according to actual needs. In other embodiments, the posture evaluation device 100 may also acquire images captured by other image capture devices via a network or self-storage media.

[0071] Processor 130 can identify key points in the image under test (step S230). In one embodiment, key points include the location of one or two eyes, one or two ears, nose, neck, and / or one or two shoulders. In other embodiments, key points can be any organ, joint, edge, or extension of any of the foregoing of a subject. These key points are used for subsequent posture evaluation.

[0072] For keypoint recognition, in one embodiment, processor 130 may label keypoints in the image to be tested based on a neural network (NN). Neural networks (or artificial neural networks) can optimize their internal structure and / or parameters based on mathematical and statistical learning methods to address problems in human perception.

[0073] In one embodiment, the neural network is a deep learning neural network. The architecture of a deep learning neural network includes an input layer, hidden layers, and an output layer. In the input layer, numerous neurons receive a large amount of non-linear input information. In the hidden layers, numerous neurons and connections may form one or more layers, and each layer includes linear combinations and non-linear activation functions. In some embodiments, such as a recurrent neural network, the output of one layer in the hidden layers is used as the input of another layer. Information is transmitted, analyzed, and / or weighed in the neuronal connections to form a prediction result in the output layer. The training procedure for the neural network involves finding the parameters (e.g., weights, biases, etc.) and connections in the hidden layers.

[0074] In one embodiment, the neural network is a convolutional neural network (CNN). A CNN typically includes one or more convolutional layers and a fully connected layer at the top (i.e., the aforementioned deep learning neural network), and may also include associated weights and pooling layers. Notably, in the convolutional layers, each layer of neurons is arranged in a two-dimensional matrix, and a specified convolutional kernel performs a convolution operation on each layer's input matrix to obtain a feature map.

[0075] In other embodiments, the neural network may also be OpenPose, Inception, GoogleNet, AlexNet, or other network structures.

[0076] In one embodiment, the neural network is trained on training samples that have already been labeled with keypoints. These training samples have been labeled with specific categories in one or more specific regions (e.g., regions of interest (ROIs) or bounding boxes). These categories are the defined keypoints. The neural network can analyze the training samples to derive patterns from them, and then use these patterns to predict unknown data. In other words, the trained neural network can infer from the image under test, determine the ROIs, bounding boxes, or selected regions in the image based on the inference results, and label keypoints and their types in these specific regions accordingly.

[0077] In other embodiments, processor 130 uses techniques such as Har features, Speeded UpRobust Features (SURF), scale-invariant feature transform (SIFT), Adaboost, or other image recognition and / or feature matching techniques to identify key points and then mark key points from the image to be tested.

[0078] For example, Figure 4 This is a schematic diagram of key point markings on the front according to an embodiment of the present invention. Please refer to... Figure 3C and Figure 4 Assuming there are 150 pairs of image capture devices Figure 3C The main body B2, located directly in front of the image capture device 150 (facing the image capture device 150), is photographed to obtain the image to be tested, IM1. The processor 130 can identify the neck P1, shoulders P2-1, P2-2, ears P3-1, P3-2, eyes P4-1, P4-2, and nose P5 from the image to be tested.

[0079] Figure 5 This is a schematic diagram of key point markings on an oblique side according to an embodiment of the present invention. Please refer to... Figure 3C and Figure 5 Assuming there are 150 pairs of image capture devices Figure 3C The main body B3, located at the right front of the image capture device 150 (facing the image capture device 150 at an oblique angle), captures an image IM2 to be tested. The processor 150 can identify the neck P1, shoulders P2-1, P2-2, ears P3-2, eyes P4-1, P4-2, and nose P5 from the image to be tested.

[0080] Figure 6 This is a schematic diagram of key point markings on the side according to an embodiment of the present invention. Please refer to... Figure 3C and Figure 6 Assuming there are 150 pairs of image capture devices Figure 3CThe main body B4, located to the left of the image capture device 150 (facing sideways to the image capture device 150), captures an image IM3 to be tested. The processor 150 can identify the neck P1, shoulders P2-2, ears P3-2, eyes P4-2, and nose P5 from the image to be tested.

[0081] The processor 130 acquires the positions of a first group of key points among those key points and defines a first geometric relationship through the association of the first group of key points (step S250). Specifically, the key points include a first group of key points and a second group of key points. Some or all of the key points can be used as the first group of key points and / or the second group of key points. The first group of key points is related to the relative position between the subject and the image capture device 150, and the second group of key points is related to the correctness of the pose. The detailed evaluation method will be described in subsequent embodiments.

[0082] Taking a reading context as an example (but not limited to this context), Figure 7 This is a schematic diagram illustrating key point markings in various reading scenarios according to an embodiment of the present invention. Please refer to... Figure 7 Due to different user habits or environmental influences, the geometric relationships formed by the association of key points may differ under different relative positions (e.g., samples S1-S5 have the subject facing the image capture device 150 from the front, samples S6-S10 have the subject facing the image capture device 150 from an oblique angle, and samples S11-S15 have the subject facing the image capture device 150 from the side). Geometric relationships may relate to the angles, distances, proportions, relative positions, or relative directions formed by the lines connecting the key points and / or the geometric shapes they form due to their location.

[0083] The processor 150 can determine the relative position between the subject and the image capture device 150 based on the first geometric relationship (step S270). In one embodiment, the processor 130 can analyze the first reference geometric relationship corresponding to the association of the first group of key points from a known sample (an image in which the relative position between the subject and the image capture device 150 has been confirmed), compare this first reference geometric relationship with the first geometric relationship, and confirm the relative position between the subject and the image capture device 150 in the image to be tested based on the comparison result.

[0084] In one embodiment, the relative positions between the subject and the image capture device 150 include a frontal and oblique side view, and the first group of key points includes the positions of both shoulders and the neck. The processor 130 may define a first line and two second lines. The first line extends horizontally from the neck position or towards one of the shoulders. In one embodiment, when the link between the two shoulder positions is a horizontal line (e.g., relative to any reference horizontal plane), the first line extends horizontally from the neck position. In one embodiment, when the link between the two shoulder positions is an oblique line, the first line extends from the neck position towards one of the shoulders.

[0085] On the other hand, these two second lines are located parallel to each other on both sides of the first line and are spaced apart from the first line. For example, when the first line extends horizontally from the neck, the two second lines are also horizontal. As another example, when the first line extends from the neck towards a shoulder, the two second lines are parallel to this extension but are not necessarily horizontal (depending on the inclination of the connection between the neck and shoulder).

[0086] The processor 150 can determine whether the shoulder position is within a first range between the two second lines. In this embodiment, the first geometric relationship is the positional relationship between the shoulder position and the first range. For example, the shoulder being within the first range, or the distance between the shoulder and the first range. That is, whether the left and right shoulders of the subject fall within the first range.

[0087] The processor 130 can determine whether the relative position is frontal or oblique based on a first geometric relationship. For example, Figure 8A and Figure 8B This is a schematic diagram of frontal recognition according to an embodiment of the present invention. Please refer to... Figure 8A The connection between the positions of the two shoulders P2-1 and P2-2 is roughly a horizontal line. The first line (taking the x-axis (or a horizontal line, and the y-axis a vertical line) as an example) starts at the neck P1 position and extends horizontally outwards (e.g., to the left and right sides of the drawing). Two second lines RH1 and RH2 are located above and below the x-axis (i.e., the first line). The second lines RH1 and RH2 are parallel to the first line and have distances h1 and h2 (these values ​​can be changed according to actual needs) between them. The processor 130 can set the range between the second lines RH1 and RH2 as the first range. Please refer to... Figure 8B The processor 130 can determine whether shoulders P2-1 and P2-2 are located within a first range between the second lines RH1 and RH2. When both shoulders P2-1 and P2-2 are within the first range, the processor 130 can determine that the relative position between the subject and the image capture device 150 is frontal. That is, when the first line extends horizontally from the position of the neck P1, and the positions of shoulders P2-1 and P2-2 are located within the first range between the two second lines RH1 and RH2, the processor 130 determines that the relative position is frontal. At this time, the subject is facing the image capture device 150, and its two shoulders are approximately horizontal (e.g., ...). Figure 4 (As shown). When shoulders P2-1 and P2-2 are not within the first range, the processor 130 can determine that the subject is not facing the image capture device 150 from the front.

[0088] Another example of the first geometric relationship between the shoulder and the first range, Figure 9A and Figure 9B This is a schematic diagram of oblique side recognition according to an embodiment of the present invention. Please refer to... Figure 9A and Figure 9B The connection between the positions of shoulders P2-1 and P2-2 is roughly a diagonal line, with the first line starting from the position of neck P1 and extending towards shoulder P2-1. Two second lines, RH3 and RH4, are located diagonally above and below the first line, respectively. The second lines RH3 and RH4 are parallel to the first line and have distances h3 and h4 (these values ​​can be changed according to actual needs) between them. The processor 130 can set the range between the second lines RH3 and RH4 as a first range. The line connecting shoulder P2-1 to neck P1 (taking the x-axis as an example) forms an angle θ5 with the first line. The processor 130 can determine whether shoulder P2-2 is located within the first range between the second lines RH3 and RH4. When shoulder P2-2 is located within the first range (or the angle θ5 is less than an angle threshold (e.g., 15, 30, or 45 degrees)), the processor 130 can determine that the relative position between the subject and the image capture device 150 is a diagonal side. In other words, when the first line extends from the neck P1 towards the shoulder P2-1, and the shoulder P2-1 is located within the first range RH3, RH4 between the two second lines, the processor 130 determines that the relative position is a sloping side. At this time, the main body is sloping, and its two shoulders are approximately tilted relative to the horizontal line (e.g., Figure 5 (As shown). When shoulder P2-2 is not within the first range (or the included angle θ5 is greater than the angle threshold value), processor 130 can determine that the subject is not facing the image capture device 150 at an oblique angle.

[0089] In another embodiment, the first group of key points includes the positions of the two shoulders and the neck, and the processor 130 can define a first ratio. The first ratio is the ratio of the distances from the two shoulders P2-1, P2-2 to the neck P1, respectively. That is, the ratio of the distances from the left shoulder and the right shoulder to the neck. In this embodiment, the first geometric relationship is the first ratio.

[0090] The processor 130 can determine whether the first proportion is within the first proportion range, and determine whether the relative position is a front or oblique side based on the first geometric relationship. For example, Figure 10 This is a schematic diagram of frontal identification according to another embodiment of the present invention. Please refer to... Figure 10, the distance from the right shoulder P2-1 to the neck P1 is w1, and the distance from the left shoulder P2-2 to the neck P1 is w2. The processor 130 can set a reference ratio r1 and an error value b1 to form a first ratio interval (i.e., the numerical interval from r1 - b1 (i.e., the difference between the reference ratio r1 and the error value b1) to r1 + b1 (i.e., the sum of the reference ratio r1 and the error value b1); in other words, the first ratio interval is greater than or equal to the difference between the reference ratio r1 and the error value b1, and the first ratio interval is less than or equal to the sum of the reference ratio r1 and the error value b1). The processor 130 can further determine whether the first ratio (e.g., the distance w1 divided by the distance w2) is within the first ratio interval. When the first ratio is within the first ratio interval (i.e., r1 - b1 ≤ w1 / w2 ≤ r1 + b1), the processor 130 can determine that the relative position between the subject and the image capture device 150 is frontal.

[0091] In a preferred embodiment, when max{w1, w2} / min{w1, w2} ≤ 1.1, the processor 130 can also determine that the relative position between the subject and the image capture device 150 is frontal. Here, max{w1, w2} refers to taking the maximum value of w1 and w2, and min{w1, w2} refers to taking the minimum value of w1 and w2.

[0092] For another example of the first ratio of the shoulder and the neck, please refer to Figure 10 , similarly, the processor 130 can still further determine whether the first ratio (e.g., the distance w1 divided by the distance w2) is within the first ratio interval. When the first ratio is not within the first ratio interval (i.e., r1 - b1 > w1 / w2 or w1 / w2 > r1 + b1), the processor 130 can determine that the relative position between the subject and the image capture device 150 is obliquely frontal.

[0093] For another example of the first ratio of the shoulder and the neck, please refer to again Figure 10 , similarly, when 1.1 < max{w1, w2} / min{w1, w2} ≤ 1 + b, the processor 130 can also determine that the relative position between the subject and the image capture device 150 is obliquely frontal. Here, b > 0.1, and the value of b may vary according to the placement angle of the image capture device 150.

[0094] It should be noted that the reference ratio r1 and the error value b1 for the obliquely frontal side are different from the corresponding values for the frontal side. For example, the reference ratio r1 for the obliquely frontal side (e.g., 0.7, or 0.8) can be less than the reference ratio r1 for the frontal side (e.g., 1, or 1.2). In addition, the error value is not limited to b1, or there may be two error values respectively for the upper and lower limits of the first ratio interval.

[0095] In another embodiment, the processor 130 can determine the relative position between the subject and the image capture device 150 based on the number of a first group of key points. The relative position between the subject and the image capture device 150 is lateral, and the first group of key points includes the positions of one or two shoulders, one or two eyes, and one or two ears. The processor 130 can determine the number of shoulders, eyes, and ears. For example, the number may be a specific value or exceed a range.

[0096] The processor 130 can determine the relative position between the subject and the image capture device 150 based on the number of shoulders, eyes, and ears. For example, Figure 11 This is a schematic diagram of side recognition according to another embodiment of the present invention. Please refer to... Figure 11 When the subject is in profile, there is only one point for each of the following: eyes P4-2, ears P3-2, and shoulders P2-2. That is, the number of each of the following is one: eyes P4-2, ears P3-2, and shoulders P2-2. In other words, when the number of shoulders, eyes, and ears is one each, the processor 130 determines that the relative position is profile.

[0097] It should be noted that the aforementioned range, proportion, and quantity may vary depending on the context, and the embodiments of the present invention are not limited thereto. For example, factors such as the size of the subject and the distance between it and the image capture device 150 may change the aforementioned range, proportion, and quantity.

[0098] The processor 130 can compare the preset geometric relationship with the second geometric relationship formed by the association of the second group of key points in the key points, and generate a comparison result (step S290). Specifically, please refer again to... Figure 7 Different orientations result in different positional relationships at key points, therefore, specific relative positions between the subject and the image capture device 150 have corresponding normal pose determination rules. In one embodiment, each relative position has a corresponding preset geometric relationship. The processor 130 can analyze the preset geometric relationships between key points based on known samples (confirmed to be in normal poses). That is, the preset geometric relationships are predefined based on the association between those key points (i.e., the second group of key points) where the subject is in a normal pose.

[0099] The processor 130 can define a second geometric relationship based on the type of relative position. The second geometric relationship includes the positions of two keypoints in the second group, one or more lines formed by the positions of any two keypoints in the second group, the ratio of the two lines, or the angles, ratios, or geometric relationships between those lines and a third line. The third line is a reference line extended from one of the keypoints in the second group. The processor 130 can compare this preset geometric relationship with the second geometric relationship (i.e., determine whether the second geometric relationship satisfies the preset geometric relationship), and based on the comparison result, confirm whether the subject in the image under test is in a normal pose.

[0100] In one embodiment, the preset geometric relationships of these normal postures derive from frontal, oblique, and / or lateral angle analyses.

[0101] For frontal angle analysis (i.e., relative position is frontal), in one embodiment, processor 130 can determine whether the line formed by two of those key points (i.e., a second group of key points) is parallel to a horizontal line, and generate a comparison result accordingly. In this embodiment, the second group of key points includes the positions of the two eyes, the second geometric relationship is the line formed by the positions of the two eyes, and processor 130 can define the horizontal line as a preset geometric relationship. The horizontal line can be a horizontal line extending from the nose, neck, or other key points, or a reference line at another angle.

[0102] For example, Figure 12 This is a schematic diagram illustrating normal posture recognition from the front, according to an embodiment of the present invention. Please refer to... Figure 12 The two eyes, P4-1 and P4-2, form a line LE. In a normal posture, line LE should be approximately parallel to the horizontal line (taking the x-axis as an example). That is, the angle between line LE and the horizontal line is approximately zero degrees. When the comparison result shows that line LE is parallel to the horizontal line, the processor 130 determines that the subject in the image under test is facing forward in a normal posture. When the comparison result shows that line LE is not parallel to the horizontal line, the processor 130 determines that the subject in the image under test is not facing forward in a normal posture (hereinafter collectively referred to as an undesirable posture).

[0103] Figure 13 This is a schematic diagram illustrating normal posture recognition from the front, according to another embodiment of the present invention. Please refer to... Figure 13 When the subject's head is tilted, the line connecting their two eyes, P4-1 and P4-2, is not parallel to the horizontal line. That is, the angle θ6 between the line LE2 (forming the lines P4-1 and P4-2) and the horizontal line (taking the x-axis as an example) is greater than zero degrees or even greater than other angles. Therefore, the processor 130 can set the angle range to 0–10 degrees, 0–15 degrees, or other ranges. For example, the processor 130 can use the highest, second-highest, lowest, or second-lowest value in a known sample, or the average, median, or other representative value of any one or more of these values, as the upper and / or lower limits of the angle range. When the comparison result shows that the angle θ6 between the line LE2 and the horizontal line is within the angle range, the processor 130 determines that the subject in the image being front-facing is in a normal posture. Conversely, when the comparison result shows that the angle θ6 between the line LE2 and the horizontal line is not within the angle range, the processor 130 determines that the subject in the image being front-facing is in an undesirable posture.

[0104] For frontal angle analysis, in another embodiment, the second group of key points includes the positions of the two eyes and two ears, and the processor 130 can determine whether the positions of the two ears are within a second range formed by the two eyes, and generate a comparison result accordingly. In this embodiment, the second geometric relationship is the positional relationship between the two ears and the second range. For example, whether any one or two ears are located within the second range or their relative distance or direction from the second range. The processor 130 can define the second range as the line connecting the positions of the two eyes and a reference line parallel to this line. That is, the second range is located between the line connecting the two eyes and the reference line. It is worth noting that the second range is a preset geometric relationship. The processor 130 can determine the shape and size of the second range based on key points in known samples that are determined to be frontal and in a normal pose.

[0105] For example, Figure 14A and Figure 14B This is a schematic diagram illustrating normal posture recognition from the front, according to another embodiment of the present invention. Please refer to... Figure 14A A fourth line RH5 (parallel to line LE3) is formed at a distance h5 above the line LE3 connecting the two eyes P4-1 and P4-2 (the value of which can be changed according to actual needs), and a second range is formed between line LE3 and the fourth line RH5. Please refer to... Figure 14B The processor 130 can determine whether the two ears P3-1 and P3-2 are located within the second range formed by the line LE3 and the fourth line RH5. When the comparison result shows that the two ears P3-1 and P3-2 are within the second range, the processor 130 determines that the subject in the image under test is in a normal posture with its head facing forward. However, when the comparison result shows that the two ears P3-1 and P3-2 are not within the second range (possibly because the subject's head is tilted down at too large an angle), the processor 130 determines that the subject in the image under test is in an undesirable posture with its head facing forward.

[0106] For frontal angle analysis, in another embodiment, the second group of key points includes the positions of the neck, nose, and both shoulders. The processor 130 can define a second ratio. The second ratio is the ratio of a first distance to a second distance. The first distance is the distance from the nose to the neck, and the second distance is the shortest distance (or vertical distance) between the two shoulders. The processor 130 can determine whether the second ratio is within a second ratio interval and generate a comparison result accordingly. In this embodiment, the second geometric relationship is this second ratio, and the second ratio of a normal posture is located within the second ratio interval (which may differ from the first ratio interval corresponding to the first ratio). That is, the second ratio interval is a preset geometric relationship. The processor 130 can set a reference ratio r2 and an error value b2 based on key points determined to be in a normal posture from known samples to form this second ratio interval (i.e., the range from r2-b2 (i.e., the difference between the reference ratio r2 and the error value b2) to r2+b2 (i.e., the sum of the reference ratio r2 and the error value b2)).

[0107] For example, Figure 15 This is a schematic diagram illustrating normal posture recognition from the front, according to another embodiment of the present invention. Please refer to... Figure 15 The distance from nose P5 to neck P1 is h6, and the distance between shoulders P2-1 and P2-2 is w3. The processor 130 can further determine whether the second ratio (e.g., distance h6 divided by distance w3) is within the second ratio interval (r2-b2 to r2+b2, i.e., the second ratio interval is greater than or equal to the difference between the reference ratio r2 and the error value b2, and less than or equal to the sum of the reference ratio r2 and the error value b2). When the comparison result is that the second ratio is within the second ratio interval (i.e., r2-b2≦h6 / w3≦r2+b2), the processor 130 can determine that the subject in the image under test is in a normal posture with a frontal orientation. When the comparison result is that the second ratio is not within the second ratio interval (i.e., r2-b2>h6 / w3 or h6 / w3>r2+b2), the processor 130 can determine that the subject in the image under test is in an undesirable posture with a frontal orientation (e.g., the subject is hunched over or tilted back). It should be noted that the error value is not limited to b2, and there may be two error values, one for the upper limit and one for the lower limit of the second proportional interval.

[0108] For oblique profile angle analysis (i.e., relative position as oblique profile), in one embodiment, processor 130 may define the angle between the line connecting two of the key points (i.e., a second group of key points) and a third line. In this embodiment, the second group of key points includes the positions of the neck and nose, and processor 130 may define the line connecting the positions of the neck and nose. The third line is a horizontal line, and the second geometric relationship is this angle. Furthermore, the preset geometric relationship is a preset angle range. Processor 130 may determine the preset angle range based on the angles corresponding to oblique profiles and normal postures in a known sample. For example, processor 130 may use the highest, second highest, lowest, second lowest, or any one or more of the above averages, medians, or other representative values ​​in the known sample as the upper and / or lower limits of the angle range.

[0109] Processor 130 can determine whether the included angle satisfies a preset geometric relationship. For example, Figure 16A and Figure 16B This is a schematic diagram illustrating normal posture recognition from an oblique side according to an embodiment of the present invention. Please refer to... Figure 16A The processor 130 can set an angle range (e.g., angle θ7 to θ8) based on a horizontal line (for example, the third line on the x-axis, with neck P1 as the center point). This angle range is related to the permissible downward tilt angle of neck P1. Please refer to... Figure 16BThe processor 130 can determine whether the angle θ9 between the line connecting the neck P1 and the nose P5 and the horizontal line is within an angle range. When the comparison result is that the angle θ9 is within the angle range, the processor 130 determines that the subject in the image under test is in a normal posture with its oblique side facing the horizontal direction. When the comparison result is that the angle θ9 is not within the angle range, the processor 130 determines that the subject in the image under test is in an undesirable posture with its oblique side facing the horizontal direction.

[0110] For oblique side angle analysis, in another embodiment, the second group of key points are the positions of the eyes and ears. The processor 130 can define the line formed by the positions of the eyes and ears, and define the angle between the line and the third line. In this embodiment, the third line is a horizontal line, the second geometric relationship is the angle between the line formed by the positions of the eyes and ears and the third line, and the preset geometric relationship is a preset angle range.

[0111] Processor 130 can determine whether the included angle satisfies a preset geometric relationship and generate a comparison result accordingly. For example, Figure 17A and Figure 17B This is a schematic diagram illustrating normal posture recognition from an oblique side according to another embodiment of the present invention. Please refer to... Figure 17A The processor 130 can set an angle range (e.g., angle θ10 to θ11) based on a horizontal line (for example, the third line on the x-axis, centered at the eye P4-2). This angle range is related to the allowable downward tilt angle of the neck P1. Please refer to... Figure 17B The processor 130 can determine whether the angle θ12 between the line connecting the eyes P4-2 and the ears P3-2 and the horizontal line is within an angle range. When the comparison result is that the angle θ12 is within the angle range, the processor 130 determines that the subject in the image under test is in a normal posture with its oblique side facing the camera. When the comparison result is that the angle θ12 is not within the angle range, the processor 130 determines that the subject in the image under test is in an undesirable posture with its oblique side facing the camera.

[0112] It should be noted that, Figure 17A and Figure 17B It could also be the line connecting the eye (P4-1) and the ear (P3-1), and it is a horizontal line based on the eye (P4-1).

[0113] For side angle analysis (i.e., relative position as side), in one embodiment, the processor 130 can also define the angle between the line connecting two of the key points (i.e., the second group of key points) and the third line. In this embodiment, the second group of key points includes the positions of the neck and nose, and the processor 130 can define the line connecting the positions of the eyes and ears. The third line is a horizontal line, the second geometric relationship is the angle between the connecting line and the third line, and the preset geometric relationship is a preset angle range.

[0114] Processor 130 can determine whether the included angle satisfies a preset geometric relationship and generate a comparison result accordingly. For example, for example, Figure 18 This is a schematic diagram illustrating normal posture recognition from the side according to an embodiment of the present invention. Please refer to... Figure 18 The processor 130 can set the angle tolerance b and the angle θ13, and determine whether the angle between the line connecting the neck P1 and the nose P5 and the horizontal line (taking the x-axis as an example, i.e., the third line, with the neck P1 as the center point) is within the angle range (angle θ13-b to angle θ13+b). When the comparison result is that the angle between the line connecting the neck P1 and the nose P5 and the horizontal line is within the angle range, the processor 130 determines that the subject in the image under test is in a normal posture with a side view. When the comparison result is that the angle between the line connecting the neck P1 and the nose P5 and the horizontal line is not within the angle range, the processor 130 determines that the subject in the image under test is in an undesirable posture with a side view.

[0115] Regarding the side angle analysis, in another embodiment, the second group of key points are the positions of the eyes and ears. The processor 130 can define the line formed by the positions of the eyes and ears, and define the angle between the line and the third line. In this embodiment, the third line is a horizontal line, the second geometric relationship is the angle between the line formed by the positions of the eyes and ears and the third line, and the preset geometric relationship is a preset angle range.

[0116] Processor 130 can determine whether the included angle satisfies a preset geometric relationship and generate a comparison result accordingly. For example, Figure 19A and Figure 19B This is a schematic diagram illustrating normal posture recognition on the side according to another embodiment of the present invention. Please refer to... Figure 19A The processor 130 can set an angle range (e.g., angle θ14 to θ15) based on a horizontal line (for example, the third line on the x-axis, centered at the eye P4-2). This angle range is related to the allowable downward tilt angle of the neck P1. Please refer to... Figure 17B The processor 130 can determine whether the angle θ16 between the line connecting the eyes P4-2 and the ears P3-2 and the horizontal line is within an angle range. When the comparison result is that the angle θ16 is within the angle range, the processor 130 determines that the subject in the image under test is in a normal posture with its oblique side facing the camera. When the comparison result is that the angle θ16 is not within the angle range, the processor 130 determines that the subject in the image under test is in an undesirable posture with its oblique side facing the camera.

[0117] It should be noted that, Figure 19A and Figure 19B It could also be the line connecting the eye (P4-1) and the ear (P3-1), and it is a horizontal line based on the eye (P4-1).

[0118] Furthermore, the aforementioned range, proportion, and angle range may vary depending on the context, and the embodiments of the present invention are not limited thereto. For example, factors such as the size of the subject and the distance between it and the image capture device 150 may alter the aforementioned range, proportion, and angle range.

[0119] Whether it's a normal or poor reading or viewing posture, maintaining a fixed posture for a long time will cause fatigue. Therefore, statistical analysis can be performed on individual postures to issue different warnings. In one embodiment, the processor 130 can issue a warning based on a comparison result and the cumulative number of times a second comparison result is obtained. The comparison result and the second comparison result are a normal posture or a poor posture. The cumulative number is the statistical number of times the comparison result and the second comparison result indicate a normal posture or an abnormal posture (i.e., a poor posture). The second comparison result is related to the comparison result obtained by comparing a normal posture with one or more second test images, and the acquisition time of the test image and the second test image may be different. For example, the image capture device 150 captures one image every 10 seconds, every 30 seconds, or every minute as the test image or the second test image. In addition, the cumulative number of times a warning needs to be issued may be related to the image acquisition time, statistical time, health education information, or other needs.

[0120] Figure 20 This is a flowchart illustrating an alarm based on an embodiment of the present invention. Please refer to... Figure 20 The processor 130 performs sample matching on the image to be tested or the second image to be tested (step S201). For example, the processor 130 uses images judged to be normal postures as positive samples and images judged to be poor postures as negative samples, and counts the cumulative number of positive and negative samples respectively. The processor 130 can determine whether the cumulative number meets the alarm requirements (e.g., the cumulative number is greater than a certain number, or the cumulative number within a certain time period reaches a certain number) (step S203). When the cumulative number meets the alarm requirements, the processor 130 can issue an alarm (step S205). The alarm is, for example, a reminder related to sound, light, or a combination thereof. For example, by flashing a light bulb of a specific color or by emitting a prompt sound. When the cumulative number does not yet meet the alarm requirements, the processor 130 continues to count the cumulative number.

[0121] In another embodiment, the processor 130 may also issue an alert upon detecting a negative sample.

[0122] In summary, in the posture evaluation device and method of this invention, the orientation of the subject relative to the image capture device is determined by analyzing the positional relationship formed by several key points of the subject in the image to be tested, and it is determined whether the subject with a specific orientation is in a normal posture. The positional relationship may be the angle, distance, or proportion formed by the key points and / or the lines connecting them. This improves the accuracy and efficiency of posture estimation.

[0123] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method of posture assessment, characterized in that, The method comprises: capturing an image to be tested by an image capturing device; identifying a plurality of key points in the image to be tested, wherein the key points are located at a plurality of positions of a subject in the image to be tested, and the key points comprise a first group of key points and a second group of key points; obtaining positions of the first group of key points, and defining a first geometric relationship via association of the first group of key points; judging a relative position between the subject and the image capturing device according to the first geometric relationship, wherein each of the relative positions has a corresponding preset geometric relationship, wherein the preset geometric relationship is defined in advance via association of the second group of key points when the subject is in a normal posture, the first group of key points of the subject comprises positions of two shoulders and a neck, and the step of judging the relative position between the subject and the image capturing device comprises: defining a first line and two second lines, wherein the first line is horizontally extended from the position of the neck or is extended in a direction of the shoulders, and the two second lines are respectively located on two sides of the first line in parallel and have a distance from the first line; judging whether the positions of the shoulders are located in a first range between the two second lines, and the first geometric relationship is a positional relationship between the positions of the shoulders and the first range; and judging the relative position as a front view or an oblique side view according to the first geometric relationship; wherein, when it is judged that the first line is horizontally extended from the position of the neck, the connection of the positions of the two shoulders is a horizontal line; wherein, when it is judged that the first line is extended in a direction of the shoulders from the position of the neck, the connection of the positions of the two shoulders is an oblique line; wherein, the step of judging the relative position as the front view or the oblique side view according to the first geometric relationship comprises: judging that the positions of the shoulders are located in the first range between the two second lines, and further judging the relative position as the front view, wherein the first line is horizontally extended from the position of the neck; and judging that the positions of the shoulders are located in the first range between the two second lines, and further judging the relative position as the oblique side view, wherein the first line is extended in a direction of the shoulders from the position of the neck; and comparing the preset geometric relationship with a second geometric relationship formed by association of the second group of key points, and generating a comparison result.

2. The posture evaluation method according to claim 1, characterized by, The step of identifying the key points in the image to be tested comprises: labeling the key points in the image to be tested based on a neural network, wherein the neural network is trained by labeled learning samples of the key points.

3. The posture evaluation method according to claim 1, characterized by, The method further comprises: judging whether the subject is in the normal posture according to the comparison result, wherein the preset geometric relationship is defined in advance via association of the key points when the subject is in the normal posture; wherein, after the step of judging whether the subject is in the normal posture according to the comparison result, the method further comprises: issuing a warning according to the comparison result and a cumulative number of second comparison results, wherein the cumulative number is a statistical number of the comparison result and the second comparison result being the normal posture or not, the second comparison result is related to at least one second image to be measured, and the image to be measured and the at least one second image to be measured are captured at different times.

4. The posture evaluating method according to claim 1, characterized by, The first group of key points located on the subject includes positions of two shoulders and a neck, and the step of determining the relative position between the subject and the image capturing device includes: defining a first ratio, wherein the first ratio is a ratio of two distances from the two shoulders to the neck, respectively; determining whether the first ratio is within a first ratio interval, wherein the first geometric relationship is the first ratio; and determining the relative position as a front view or an oblique side view according to the first geometric relationship; wherein the first ratio interval is greater than or equal to a difference between a reference ratio and a first error value, the reference ratio is set for key points determined as the normal posture, a second error value and the first error value are error values for upper and lower limits of the first ratio interval, and the first ratio interval is less than or equal to a sum of the reference ratio and the second error value, determining the relative position as the front view, wherein the first ratio is within the first ratio interval; and determining the relative position as the oblique side view, wherein the first ratio is not within the first ratio interval.

5. The posture evaluation method according to claim 1, characterized by, Further comprising: determining the relative position between the subject and the image capturing device according to a number of the first group of key points; wherein the first group of key points located on the subject includes positions of at least one shoulder, at least one eye, and at least one ear, and the step of determining the relative position between the subject and the image capturing device includes: determining a number of the at least one shoulder, the at least one eye, and the at least one ear; determining the relative position between the subject and the image capturing device according to the number of the at least one shoulder, the at least one eye, and the at least one ear; and determining the relative position as a side view, wherein the number of the at least one shoulder, the at least one eye, and the at least one ear is one.

6. The posture evaluation method according to claim 1, characterized by, The relative positions include a front view, an oblique side view, and a side view, and the step of comparing the second geometric relationship formed by the association of the preset geometric relationship corresponding to each of the relative positions and the second group of key points includes: defining the second geometric relationship according to a type of the relative position, wherein the second geometric relationship includes positions of two of the second group of key points, at least one line formed by positions of any two of the second group of key points, a ratio of two lines, or an included angle, a ratio, or a geometric relationship between the at least one line and a third line, the third line being a reference line extended from one of the second group of key points; and determining whether the second geometric relationship satisfies the preset geometric relationship.

7. The posture evaluation method according to claim 6, wherein When the relative position is the front face, and the second group of key points comprises positions of two eyes, the step of comparing the preset geometric relationship with the second geometric relationship formed by the association of the second group of key points comprises: defining a horizontal line as the preset geometric relationship; determining whether the line formed by the positions of the two eyes is parallel to the horizontal line, wherein the line formed by the positions of the two eyes is the second geometric relationship; and generating the comparison result; When the relative position is the front face, and the second group of key points comprises positions of two eyes and two ears, the step of comparing the preset geometric relationship with the second geometric relationship formed by the association of the key points comprises: defining a second range between the line formed by the positions of the two eyes and a reference line parallel to the line, wherein the second range is the preset geometric relationship; determining whether the positions of the two ears are located in the second range, wherein the positions of the two ears are the second geometric relationship; and generating the comparison result.

8. The posture evaluation method according to claim 6, characterized by, When the relative position is the front face, and the second group of key points comprises positions of a neck, a nose, and two shoulders, the step of comparing the preset geometric relationship with the second geometric relationship formed by the association of the second group of key points comprises: defining a second ratio, wherein the second ratio is a ratio of a first distance and a second distance, the first distance is a distance from the nose to the neck, and the second distance is a distance between the two shoulders, and the second geometric relationship is the second ratio; determining whether the second ratio is within a second ratio interval, wherein the second ratio interval is the preset geometric relationship; and generating the comparison result. The second ratio interval is greater than or equal to a difference between a reference ratio and a third error value, the reference ratio is set for key points determined as the normal posture, fourth and third error values are error values for upper and lower limits of the second ratio interval, and the second ratio interval is less than or equal to a sum of the reference ratio and the fourth error value.

9. The posture evaluation method of claim 6, wherein When the relative position is the oblique side face or the side face, and the second group of key points comprises positions of a neck and a nose, the step of comparing the preset geometric relationship with the second geometric relationship formed by the association of the key points comprises: defining a line formed by the position of the neck and the position of the nose; defining an included angle between the line and a third line, wherein the third line is a horizontal line, and the included angle is the second geometric relationship; determining whether the included angle satisfies the preset geometric relationship, wherein the preset geometric relationship is a preset angle interval; and generating the comparison result. When the relative position is the oblique side face or the side face, and the second group of key points comprises positions of an eye and an ear, the step of comparing the preset geometric relationship with the second geometric relationship formed by the association of the key points comprises: defining a line formed by the position of the eye and the position of the ear; defining an angle between the line and the third line, wherein the third line is a horizontal line and the angle is the second geometric relationship; determining whether the angle satisfies the preset geometric relationship, the preset geometric relationship being a preset angle interval; and generating the comparison result.

10. The posture evaluation method according to claim 1, characterized by, Before the step of identifying the key points in the image to be tested, the method further comprises: unwrapping the fisheye image into a panorama image, and taking the panorama image as the image to be tested.

11. A posture evaluation device, comprising: a memory storing program code; and a processor coupled to the memory, configured to: identify a plurality of key points in an image to be tested, wherein the image to be tested is captured by an image capturing device, the key points correspond to a plurality of positions of a subject in the image to be tested, and the key points include a first group of key points and a second group of key points; obtain positions of the first group of key points, and define a first geometric relationship via association of the first group of key points; determine relative positions between the subject and the image capturing device according to the first geometric relationship, wherein each of the relative positions has a corresponding preset geometric relationship, and the preset geometric relationship is defined in advance via association of the second group of key points when the subject is in a normal posture; and compare the preset geometric relationship with a second geometric relationship formed by association of the second group of key points, and generate a comparison result, wherein the first group of key points of the subject includes positions of two shoulders and a neck, and the processor is further configured to: define a first line and two second lines, wherein the first line is horizontally extended from the position of the neck or is extended in a direction of the shoulders, and the two second lines are respectively parallel to and spaced apart from the first line; determine whether the positions of the shoulders are located in a first range between the two second lines, and the first geometric relationship is a positional relationship between the positions of the shoulders and the first range; and determine the relative position as a front view or an oblique side view according to the first geometric relationship; wherein the processor is further configured to determine that the first line is horizontally extended from the position of the neck, and a line connecting the positions of the two shoulders is a horizontal line; wherein the processor is further configured to determine that the first line is extended in a direction of the shoulders from the position of the neck, and a line connecting the positions of the two shoulders is an oblique line; wherein the processor is further configured to: determine that the positions of the shoulders are located in the first range between the two second lines, and further determine that the relative position is the front view, wherein the first line is horizontally extended from the position of the neck; and determine that the positions of the shoulders are located in the first range between the two second lines, and further determine that the relative position is the oblique side view, wherein the first line is extended in a direction of the shoulders from the position of the neck.

12. The posture evaluating apparatus according to claim 11, characterized by the processor is further configured to: labeling the key points in the to-be-tested image based on a neural network, wherein the neural network is trained by labeled learning samples of the key points.

13. The posture evaluating apparatus according to claim 11, wherein The processor is further configured to: determine whether the subject is in a normal posture according to the comparison result, wherein the preset geometric relationship is predefined by the correlation of the key points when the subject is in the normal posture; wherein the processor is further configured to: issue a warning according to the comparison result and a cumulative number of a second comparison result, wherein the cumulative number is a number of times that the comparison result and the second comparison result are the normal posture or are not the normal posture, the second comparison result is related to at least one second to-be-tested image, and the to-be-tested image and the at least one second to-be-tested image are captured at different times.

14. The posture evaluating apparatus according to claim 11, wherein The first group of key points on the subject includes the positions of two shoulders and a neck, and the processor is further configured to: define a first ratio, wherein the first ratio is a ratio of two distances from the two shoulders to the neck, respectively; determine whether the first ratio is within a first ratio interval, wherein the first geometric relationship is the first ratio; and determine the relative position as a front view or an oblique side view according to the first geometric relationship; wherein the first ratio interval is greater than or equal to a difference between a reference ratio and a first error value, and the first ratio interval is less than or equal to a sum of the reference ratio and a second error value, the reference ratio is set for the key points determined as the normal posture, the second error value and the first error value are error values for upper and lower limits of the first ratio interval, and the processor is further configured to: determine the relative position as the front view, wherein the first ratio is within the first ratio interval; and determine the relative position as the oblique side view, wherein the first ratio is not within the first ratio interval.

15. The posture evaluating apparatus according to claim 11, wherein The processor is further configured to: determine the relative position between the subject and the image capture device according to the number of the first group of key points; wherein the first group of key points on the subject includes the positions of at least one shoulder, at least one eye, and at least one ear, and the processor is further configured to: determine the number of the at least one shoulder, the at least one eye, and the at least one ear; determine the relative position between the subject and the image capture device according to the number of the at least one shoulder, the at least one eye, and the at least one ear; and determine the relative position as a side view, wherein the number of the at least one shoulder, the at least one eye, and the at least one ear is one, respectively.

16. The posture evaluating apparatus according to claim 11, wherein The relative position includes a front view, an oblique side view, and a side view, and the processor is further configured to: defining the second geometric relationship according to a type of the relative position, wherein the second geometric relationship comprises positions of two of the second group of key points, at least one line formed by positions of any two of the second group of key points, a ratio of the at least one line, or an angle, a ratio or a geometric relationship between the at least one line and a third line, the third line being a reference line extended from one of the second group of key points; and determining whether the second geometric relationship satisfies the preset geometric relationship. 17.The gesture evaluation device of claim 16, wherein, when the relative position is the front face and the second group of key points comprises positions of two eyes, the processor is further configured to: define a horizontal line as the preset geometric relationship; determine whether the line formed by the positions of the two eyes is parallel to the horizontal line, wherein the line formed by the positions of the two eyes is the second geometric relationship; and generate the comparison result. 19.The gesture evaluation device of claim 16, wherein, when the relative position is the oblique side face or the side face and the second group of key points comprises positions of a neck and a nose, the processor is further configured to: define the line formed by the position of the neck and the position of the nose; define the angle between the line and a third line, wherein the third line is a horizontal line, and the angle is the second geometric relationship; determine whether the angle satisfies a preset geometric relationship, wherein the preset geometric relationship is a preset angle interval; and generate the comparison result. ​ ​ ​ ​ ​ ​ ​ ​ ​ 18. The posture evaluating apparatus according to claim 16, characterized by ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ when the relative position is the oblique side or the side, and the second group of key points includes positions of eyes and ears, the processor is further configured to: define a line connecting the position of the eyes and the position of the ears; define an included angle between the line and a third line, wherein the third line is a horizontal line, and the included angle is the second geometric relationship; determine whether the included angle satisfies the preset geometric relationship, the preset geometric relationship being a preset angle interval; and generate the comparison result.

20. A posture evaluation system, comprising: an image capturing device configured to capture a test image; a memory configured to store program codes; a processor coupled to the memory and the image processing device, and configured to load and execute the program codes to: identify a plurality of key points in the test image, wherein the test image is captured by the image capturing device, the key points correspond to a plurality of positions of a subject in the test image, and the key points include a first group of key points and a second group of key points; obtain positions of the first group of key points, and define a first geometric relationship via association of the first group of key points; determine relative positions between the subject and the image capturing device according to the first geometric relationship, wherein each of the relative positions has a corresponding preset geometric relationship, and the preset geometric relationship is defined in advance via association of the second group of key points when the subject is in a normal posture; and compare a second geometric relationship formed via association of the second group of key points with the preset geometric relationship, and generate a comparison result, wherein the first group of key points of the subject includes positions of two shoulders and a neck, and the processor is further configured to: define a first line and two second lines, wherein the first line is horizontally extended from the position of the neck or is extended from the position of the neck towards the positions of the shoulders, and the two second lines are respectively and parallelly located on two sides of the first line with a distance from the first line; determine whether the positions of the shoulders are located in a first range between the two second lines, and the first geometric relationship is a positional relationship between the positions of the shoulders and the first range; and determine the relative position as a front or an oblique side according to the first geometric relationship; wherein the processor is further configured to determine that the first line is horizontally extended from the position of the neck, and a line connecting the positions of the two shoulders is a horizontal line; wherein the processor is further configured to determine that the first line is extended from the position of the neck towards the positions of the shoulders, and a line connecting the positions of the two shoulders is an oblique line; wherein the processor is further configured to: determine that the positions of the shoulders are located in the first range between the two second lines, and further determine that the relative position is the front, when the first line is horizontally extended from the position of the neck; and determine that the positions of the shoulders are located in the first range between the two second lines, and further determine that the relative position is the oblique side, when the first line is extended from the position of the neck towards the positions of the shoulders. ​ ​

Images