Determination of the body posture
Patent Information
- Application Number
- EP2023707029
- Authority / Receiving Office
- EP · EP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2022-03-21
- Filing Date
- 2023-02-20
- Publication Date
- 2024-11-20
AI Technical Summary
The increasing trend of sedentary work due to digital content processing on electronic devices, particularly PCs, leads to back problems caused by poor posture and lack of exercise, which existing technologies fail to effectively monitor and address.
A method and system using a camera-connected electronic device to read images, determine the relative position of body reference points, and issue advisory signals for poor posture, employing image evaluation techniques including neural networks and 3D-capable cameras to provide feedback on posture.
Effectively monitors and improves posture by providing timely warnings, reducing the risk of back problems associated with sedentary work, leveraging on-board resources like laptop cameras and advanced image analysis methods for accurate posture assessment.
Smart Images

Figure 1.1
Abstract
Description
[0001] Description
[0002] Determination of posture
[0003] The invention relates to a system and a method for determining body posture when working on a PC.
[0004] A general trend in the workplace is the shift of all content to the digital world. Content processing is no longer done on paper or other hardware, but on electronic devices such as tablets and PCs.
[0005] This trend has received enormous impetus from the pandemic, as safety regulations such as social distancing rules have been implemented virtually worldwide by shifting to home office. This means that even more work is being done on the PC. Meetings and other personal communication, which before the pandemic tended to take place away from the PC, are now also being conducted from the PC.
[0006] The proportion of sedentary work has therefore increased further. Sedentary work with little variety is considered stressful for the back and is blamed for some of the back problems experienced by the population. Reasons cited for this include a lack of exercise and poor posture. This lack of exercise can be monitored via smartphones and smartwatches, for example, by counting steps. However, this is more difficult when it comes to posture.
[0007] The object of the invention is to provide a method and an arrangement with which a person can be given feedback on his or her posture when working on an electronic device, in particular a PC.
[0008] This object is achieved by a method having the features specified in claim 1. A further solution consists in an arrangement having the features of claim 10. In the method according to the invention for determining the posture of a person when working on an electronic device, in particular a PC, in a first step at least one image is read out from a camera of the electronic device. Then in a second step a value representing the posture recognizable in the image is determined. Finally an indication signal is output if the value lies within a warning range.
[0009] The arrangement according to the invention for determining the posture of a person when working on an electronic device, in particular a PC, comprises a camera connected to the electronic device for image transmission and a program running on the electronic device for evaluating at least one image received by the camera, the program carrying out the method described above.
[0010] This ensures that posture can be determined and monitored, especially during screen work, and that the worker can be warned if their posture is not recommended. This can lead to an improvement in posture, provided the person responds to the warning signals. The advantage of this is that on-board features are only used for many work PCs, namely the camera that is usually already integrated in laptops. Due to the working conditions during the pandemic, desktop PCs often have a camera, although not integrated.
[0011] Advantageous embodiments of the method and arrangement according to the invention emerge from the dependent claims. The embodiment of the independent claims can be combined with the features of one of the subclaims or, preferably, with those of several subclaims. Accordingly, the following additional features can be provided:
[0012] The value can be determined from the relative apparent position of body reference points in the image. The body reference points are suitable parts of the body which are selected so that they are clearly visible in the image and their relative apparent position is a good indicator of the body posture, particularly the posture of the upper body. Apparent refers to the position as it appears in the two-dimensional image of the camera. Two points in three-dimensional space, the connecting line of which is parallel to the camera's line of sight, have no distance in the camera's image, even if their three-dimensional distance is large. In contrast, two points, the connecting line of which is essentially perpendicular to the camera's line of sight, have a distance in the image which is proportional to the three-dimensional distance (and the distance from the camera). Relative means that the distance between two body reference points is considered, not the absolute position of a body reference point.
[0013] The invention takes advantage of the fact that a precise posture analysis is not necessary; it is sufficient to identify the essential element of a "bad" posture. This is usually seen as the case when a type of "hunched posture" is adopted, i.e. the back is bent towards the screen. Frequently, the shoulders are also raised due to use of the mouse and keyboard. Both of these ensure that the body elements around the shoulder are more parallel to the camera than in an upright and straight posture, which is assumed to be "good". In the latter case, the body elements around the shoulder are arranged much more perpendicular to the camera's line of sight.
[0014] In the apparent position in the image, certain body reference points are therefore closer together when a posture is assumed that is poorly defined and further apart when a posture that is assumed that is well defined.
[0015] Since absolute values for the apparent relative position depend heavily on the person, it can be advantageous to store at least some of these values over a longer period of time and to determine the current posture from the stored values. For example, a maximum value for the distance between the body reference points can be determined from the stored values and this can be interpreted as a measure of good posture. In order to ensure that such a stored value is available, it can be advantageous to guide the person through a one-time settings program in which various body postures are assumed under guidance and the results are saved in the form of the camera image or values derived from it.
[0016] The body reference points can be a selection of the following body parts:
[0017] - ears,
[0018] - Jaw,
[0019] - shoulder,
[0020] - sternum .
[0021] In particular, the distance between ears and shoulders, ears and sternum, or jaw and sternum can be considered. These distances are a direct measure of the person's posture and are advantageously relatively easy to identify in the camera image.
[0022] The shoulder and ears in particular can be found if only the silhouette of the person has been determined from the image. The silhouette, in turn, is relatively easy to find. It stands out particularly well in differences in lighting and contrast compared to the much more distant background. In addition to simple graphical methods, more complex methods of image analysis can also be used. For example, a neural network can be used to analyze an image. Neural networks are known to deliver good results in image analysis when the images are so complex or variable that classic image recognition methods no longer work. To achieve this, neural networks require training that is suitable and sufficiently variable for the application.This is best achieved using a sufficient number of images of different people, dressed differently, taken with a range of possible image sections and camera positions (from above / from below). The range of image sections is small, as at least the head and shoulders, and depending on the design, the upper chest area as well, must be recognizable. Pure head shots, on the other hand, are not sufficient for recognizing posture and therefore do not need to be taken into account or trained. The image elements to be recognized, i.e. the body reference points, are easy to determine manually as input values for training.
[0023] A vertical distance between the body reference points can be determined as a relative apparent position. In other words, the horizontal position of the body reference points is ignored, since for the part of the body posture to be observed, only the inclination of the head, shoulders, and upper chest area is important, and these can be determined purely from the vertical distance.
[0024] The relative apparent position can be determined for both sides of the body, and a comparison can be used to determine a posture deviating from vertical. Since the sides of the body are essentially symmetrical, a slanted posture can be determined by comparing the body reference points between the left and right sides. For example, a connecting vector between the two positions of the ears and a connecting vector between the positions of the shoulders can be determined. Several conclusions can be drawn from these.
[0025] If the direction of the vectors remains different from the horizontal for an extended period, the camera's position may be tilted. This can be checked by evaluating other image elements. For example, straight lines in the image can be detected. These often run horizontally or vertically and therefore allow position control. Another possibility is for the camera to include a position sensor, as is the case with smartphones, for example.
[0026] If the camera position is essentially horizontal, a prolonged tilt of the vectors may indicate a general tilt. This can also be detected and stored for further analysis.
[0027] In the case of a short-term tilt, however, the determined posture values can be discarded because the person is only changing their posture.
[0028] From the image, an altitude angle can be determined at which the camera records the person, and the position of the reference points can be corrected using the altitude angle. The altitude angle describes the upward or downward tilt of the camera at which the image of the person is recorded. When working on a PC, the screen is typically in the direct line of sight of the person; therefore, cameras on PCs typically record the person from above, as the camera is attached to the top edge of the screen. On laptops, the camera is typically positioned below the direct line of sight.
[0029] To determine the altitude angle, the camera can be equipped with a position sensor. For example, if the camera is a smartphone camera, such a sensor is typically present, as smartphones have generally been equipped with a position sensor for years that can determine the device's position in space (relative to Earth's gravity) in order to, for example, rotate the screen display appropriately.
[0030] If no such position sensor is available, the altitude angle can be determined from the image itself. For this purpose, for example, patterns of body contours captured from different altitude angles can be stored. These patterns are compared with the contours present in the image to determine the current altitude angle.
[0031] Additionally or alternatively, straight lines present in the image can be identified. These often originate from the edges of furniture or the room itself in which the person is located and are highly likely to be vertical or horizontal. Their course can also be used to estimate the angle at which the image was taken.
[0032] Another way to determine the altitude angle is to use a neural network. This is conveniently trained with images of people taken from different altitude angles and with different image sections. For simplification, the neural network can also be applied only to the contour of the person. The contour can be determined, for example, by motion detection.
[0033] Such a determination does not have to be carried out continuously, since the altitude angle of the camera typically remains unchanged for a long time. Therefore, considerable computing power can be used for the determination. The determination can also be carried out, for example, over a period of time using a number of camera images, e.g. a video section, so that the determination does not fail due to problems with a single image (such as over- or underexposure, sudden change in posture, false detection in a low-contrast image) or produce incorrect results. The posture can be determined for a sequence of camera images, and from the values determined in this way, those with a high probability of error can be rejected. A high probability of error arises, for example, if successive values for the posture show strong deviations from one another.This could indicate a misidentification or a change in posture. A high probability of error also arises when the determined values exceed an absolute expected range that defines anatomically probable postures. For example, it is unlikely that the ears are significantly below the shoulders. Furthermore, a high probability of error can be assumed if the body reference points on both sides (left / right) are very asymmetrical, i.e., if they cannot be essentially mapped onto one another by mirroring them across a possibly oblique axis.
[0034] In general, it is advantageous to delay further processing even with values that appear correct in principle until a certain number of images, i.e. a certain length of video, have been evaluated in order to ensure a high level of evaluation reliability. As the evaluation does not depend on the results being fast or frequent, a time period of 10 s, 20 s or even 1 minute can be used. As the individual images from the camera in today's standard video recording hardly differ from one another, as they are recorded at a rate of, for example, 40 ms, the evaluation of most of the resulting images can be omitted. For example, only one image can be evaluated in each 1 s interval. The evaluation can look at several images and reject those where the evaluation leads to a high probability of error.
[0035] At least two cameras, particularly one 3D-capable camera, can be used. An improvement in the accuracy of determining posture can be achieved if spatial information is available instead of purely two-dimensional image information. With such information, posture can be determined independently of sources of error such as spatial distortion and foreshortening.
[0036] The cameras can be independent cameras, such as a laptop camera and an additional freely adjustable camera. However, it can also be an integrated system for capturing three-dimensional image information, i.e., a 3D camera that also uses two cameras. It is advantageous for determining the three-dimensional position if the two cameras are positioned a few centimeters apart with parallel viewing directions.
[0037] The invention is described and explained in more detail below with reference to the exemplary embodiments shown in the figures. They show:
[0038] Figure 1 a person in a first, bad posture while working on a PC with a camera,
[0039] Figure 2 Steps of a procedure for determining the person’s posture,
[0040] Figure 3 shows the person working on a laptop with an integrated camera, and
[0041] Figure 4 shows the person in a second, good posture while working on the PC.
[0042] Figure 1 shows a person 1 in a first posture while working on a PC not shown in Figure 1. This first posture is curved forward, i.e., toward the PC screen 2. Such a posture is considered to be harmful to the back in the long term and is therefore disadvantageous. Figure 1 also shows a camera 3 connected to the PC, which is attached in the usual way to the top of the screen and is aligned such that it captures the head and upper body of person 1.
[0043] The person's posture is then determined by a program running on the PC. For this purpose, a method illustrated in Figure 2 is used, using an image stream from camera 3, i.e., its video output.
[0044] In a first step 31, the angle at which the camera 3 is viewing the person 1 is determined. In this exemplary embodiment, the program identifies the apparent vertical distance 4 between the ears and the shoulder of the person 1. Therefore, in this case, the angle of view onto this area, for example onto the ears, is of interest. The altitude angle 5, for example, can be used as a measure. This is the angle between a horizontal viewing direction and the viewing direction toward the ears.
[0045] Figure 3 shows a view analogous to Figure 1, in which an integrated laptop camera 6 is used as camera 3. As can be seen from Figure 3, the altitude angle 5 here is completely different from that of camera 3 in Figure 1.
[0046] The effect of this can be seen when comparing it with Figure 4. Figure 4 shows a straight and upright posture. When the posture changes from that shown in Figures 1 and 3, the apparent distance 4 between the ears and shoulders decreases for camera 3 in Figure 1 due to the camera's angle of view. In extreme cases, the distance could become practically zero if the person leans far forward. For the laptop camera 6, on the other hand, the distance actually increases slightly due to its angle of view.
[0047] The altitude angle 5 is determined by measuring straight
[0048] Lines are identified in the image, and from their position, assuming that they are horizontal and vertical lines, the altitude angle 5 is determined. For example, the edges of a table or vertical lines of a door or the edge of a room can be detected. If such lines cannot be detected, the program can provide a manual setting with which the altitude angle 5 can be determined.
[0049] In other embodiments of the invention, a 3D-capable camera can be used. In this case, spatial depth information is available, which can be used to determine the pose. This eliminates the dependence on the viewing angle, and the determination of the altitude angle 5 can be omitted.
[0050] As an alternative to determining the altitude angle 5, it is also possible to directly identify the position of body reference points by conducting a training session with person 1 beforehand. Person 1 can assume various postures, and the program records the resulting positions of the body reference points. This reduces the dependence of the method on the available angles and corresponding calculations, but increases the dependence on direct input from person 1.
[0051] In a second step 32, the person's ears and shoulders are identified for a series of camera images. In this embodiment, identification is attempted on both sides. Identification is performed using a neural network that was previously trained using images of people.
[0052] In a third step 33, an error probability for the identification results is determined. This includes whether an identification has led to results that do not deviate too far from the results of previous images, if previous images are available. Furthermore, it takes into account whether the symmetry of the ears and shoulders is within the realm of possibility and whether the resulting posture is even nearly vertical, i.e., upright, or significantly tilted.
[0053] If the error probability is too high, which is checked using a threshold value, the result for this image is discarded and the process returns to step 32 and the next image is evaluated.
[0054] If the error probability is sufficiently low, the evaluation result is saved. Depending on the configuration, the evaluation can continue with step 32 for a specific number of images, for example, until a specified number of images have been evaluated or a specific number of matching results have been obtained.
[0055] If such a number is present, a comparison of the determined body posture with a warning range is carried out in a step 33. In the simplest case, the determined distance 4 between ears and shoulders can be compared with a threshold value. In this case, the distance 4 must be corrected, for example, for the altitude angle 5. If the threshold value is not reached, the program can display a window containing a warning message. It can also be specified that such a warning message only appears when the body posture has been maintained for an extended period of time, for example 10 minutes.
[0056] Reference sign
[0057] 1 person
[0058] 2 Screen 3 Camera
[0059] 4 Distance
[0060] 5 Altitude angles
[0061] 6 Laptop camera
[0062] 31 first step 32 second step
[0063] 33 third step
Claims
Patent claims 1. Method for determining a person's posture (1) when working on an electronic device, in particular a PC, using the following steps: - Reading at least one image from a camera (3) of the electronic device, - Determining a value representing the body posture recognizable in the image, - Output of a warning signal if the value is within a warning range.
2. Method according to claim 1, wherein the value is determined from the relative apparent position of body reference points in the image.
3. The method of claim 2, wherein the body reference points are selected from: - Ears, - Jaw, - Shoulder, - Sternum.
4. Method according to claim 2 or 3, wherein a vertical distance between the reference points is determined as the relative apparent position.
5. Method according to one of claims 2 to 4, wherein the relative apparent position for both sides of the body is determined and a body posture deviating from the vertical is determined by comparison.
6. Method according to one of claims 2 to 5, wherein an altitude angle (5) is determined from the image, at which the camera (3) records the person (1) and the position of the reference points is corrected with the altitude angle (5).
7. Method according to one of the preceding claims, wherein the body posture is determined for a sequence of images from the camera (3) and those values determined in this way with a high probability of error are discarded.
8. Method according to one of the preceding claims, wherein at least two cameras (3) , in particular a 3D-capable camera is used.
9. Order for determining a person's posture (1) when working on an electronic device, in particular a PC, including: - a camera connected to the PC for image transmission (3) , - a program running on a PC for evaluating at least one image received from the camera, wherein a method according to one of the preceding claims is carried out with the program.
