Body movement / position risk assessment

Abstract

A body movement / position assessment tool comprises a mobile device for video recording of a subject, a processor for analysing the video recording to produce measurements of risk movements, a display for viewing the recorded video and a representation of the measurements of risk movements. The processor is configured to assess changes or differences in risk movements between different risk movements.

Description

BODY MOVEMENT / POSITION RISK ASSESSMENT

[0001] The present invention relates generally to a system and method for assessing work posture, and more specifically to making assessments of posture-related risks.BACKGROUND ART

[0002] Monitoring a person for the safe work activities is traditionally done in person with a safety assessor watching a worker perform an activity and interviewing or coaching the worker to avoid unsafe or risky movements I activities that could result in injury.

[0003] Video systems can record the activities of a person for later review of their work practices but this still requires in person review. There is little in the way of automation of the review of safe or hazardous work practices.

[0004] Automated video analysis is still in its early days. There are some early stage algorithms that can recognize a person in a video and can even recognize body parts and poses however using these to automate risk analysis is still in development.

[0005] It is also known to have one or more devices attached to corresponding body parts to take measurements of movement of the body part and to use the measurements to compute risk scores of the movements.

[0006] This prior art document is not useful in analysing the bio-mechanics of a subject in video images.

[0007] Any reference to prior art documents is not an admission that they form part of the common general knowledge of a skilled person in any jurisdiction.SUMMARY OF THE INVENTION

[0008] According to an aspect disclosed there is provided a body movement / position assessment tool comprising: a mobile device for video recording of a subject;a processor for analysing the video recording to produce measurements of risk movements; a display for viewing the recorded video and a representation of the measurements of risk movements; wherein the processor is configured to assess changes or differences in risk movements between different risk movements.

[0009] In an embodiment the processor is configured to identify a person in the video recording. In an embodiment the processor is configured to identify body parts of the person in the video recording.

[0010] In an embodiment the analysis comprises human pose detection. In an embodiment human pose detection comprises detection of core points of a human body in each frame of the video recording. In an embodiment detection of core points comprises detection of a person in the video recording, feature generation of the detected person, and body part angle calculation. In an embodiment the analysis comprises activity detection, and risk calculation for each activity.

[0011] In an embodiment the analysis comprises one or more of: posture analysis, distance checking, deviation detection, neck position checking, shoulder symmetry checking, eye level checking, position calibration, and posture analysis.

[0012] In an embodiment the recorded video is displayed with one or more charts showing one or more of the measurements of risk with respect to time. In an embodiment two different parts of the video recording, or two different video recordings are shown in relation, such as side by side, with the charts also shown in relation, such as side by side.

[0013] In an embodiment the display comprises an interpretation of the respective measurements and / or a comparison of the respective measurements.

[0014] In an embodiment the display comprises body part risk scores for a selected time in the video recording. In an embodiment the display comprises recommended changes to reduce the risk of injury for high risk potions of the video recording or of selected part of the video recording.

[0015] In an embodiment the processing selects portions of the video recording representing higher risk movements.

[0016] The display of the video with the measurement and / or the companion is a useful tool to provide feedback to the worker and of the benefit (or otherwise) of changes made to the way a task is performed.

[0017] In an embodiment the risk assessments performed are selected according to the work the person is performing in the video recording.

[0018] In an embodiment the processor is configured to classify the activity of the person as a type of work being performed by the person in the video recording.

[0019] Also according to an aspect disclosed there is provided a method of posture / movement modelling comprising: video recording a subject person with a mobile device; analysing the video recording to produce measurements of risk movements; assessing changes in risk between different risk movements; displaying the recorded video, a representation of the measurements of risk movements and the changes in risk between the different risk movements.

[0020] According to another aspect disclosed there is provides a work modelling tool comprising: a mobile device for video recording a person performing an activity; a processor for analysing the video recording to classify the activity of the person as a type of work being performed by the person in the video recording; and an output of the identified type of work.

[0021] In an embodiment the processor is configured to track 3D movement of body parts and / or objects in space for analysis.

[0022] Certain risk assessment methods are suited to assessment of certain activity type but are not suitable for assessing other activity types. The output type of work isuseful to select risk assessment methods suitable for assessing the risk associated with the activity the person in the video recording is performing.

[0023] In an embodiment the tool comprises a processor for selecting risk assessment methods to be used to assess the risk of the activity being performed based on the identified type of work.

[0024] In an embodiment the processor is configured to detect events within an activity. In an embodiment the processor is configured to identify risker events / actions with the activity.

[0025] The combination of activity classification, event detection within the activity and risk assessment of events within the activity is particularly advantageous.

[0026] In an embodiment the processor for analysing the video recording comprises a trained neural network classifier.

[0027] In an embodiment the neural network classifier is trained using a plurality of video recordings of people, where each person is performing a known activity.

[0028] In an embodiment the plurality of video recordings are each labelled with a respective label of the known activity that the person in the respective video reporting is performing.

[0029] According to another aspect disclosed there is provided a method of work modelling comprising: video recording a person performing a work activity; analysing the video recording to classify the activity of the person as a type of work being performed by the person in the video recording; and outputting the identified type of work.

[0030] Also according to an aspect disclosed there is provided a cloud-based system for workplace safety, comprising: a mobile application for video recording and analysis of workplace tasks; an online dashboard for viewing video and data from the analysis;Al-driven data analysis for measuring workplace risk movements and suggesting controls to keep the workforce safe.

[0031] The cloud-based system for workplace safety provides a mobile application for video recording and analysis, which allows for real-time monitoring of workplace tasks and identification of potential safety hazards.

[0032] The online dashboard provides a centralised location for viewing video and data of workplace tasks, allowing for easy access to information and efficient communication between team members.

[0033] The Al-driven data analysis reduces the risk of workplace injuries and improving overall safety.

[0034] In an example, the cloud-based system provides a mobile application and online dashboard for easy monitoring and automation of workplace safety.

[0035] In an example, the Al-driven data analysis offers useful insights into workplace risk movements and suggests controls to keep the workforce safe.

[0036] In an example, the video processing method comprises human pose detection, events detection, objects detection, and force detection, making it user-friendly and easy to use.

[0037] In an example, the comparison model for risk assessment provides a formula for overall improvement and line charts with a body part risk distribution, making it easy to evaluate and improve worker safety.

[0038] In an example, there is provided a job dictionary that allows for automatic job creation and pre-filling with risk data, making it easy to evaluate worker's ability to perform a particular job.

[0039] In an example, analysis comprises changing a threshold to a seated / standing work assessment. The suggestions comprise recommendations for improvingseated / standing posture, comprising recommendations about distance between a person and a monitor, making it easy to assess and improve posture.

[0040] In an example, a live model offers a processing engine for real-time processing of the analysis and an in-browser solution, making it easy to process and analyze data in real-time.

[0041] In an example, the tool may be used for office work posture assessment. The tool may comprise one or more of a coaching mode for real-time processing and immediate notifications, computer vision algorithms for posture analysis and deviation detection, unique algorithms for ideal position calibration, neck position check, shoulder symmetry check, eye to screen distance check, and eye height level check, and video recording and processing for building an interactive report that shows key problems and provides feedback, making it easy to assess and improve office work posture.

[0042] In an embodiment, the application provides a method for video processing, comprising one or more of:- Human pose detection functionality and algorithms for detecting one or more of core points of the human body, classifying front / side / back view, calculating body parts angles, detecting activity, calculating body parts and video risk, calculating twist, detecting highlights, and predicting RULA / REBA;- Events detection functionality and algorithms for detecting core events like pushing, pulling, bending, and lifting;- Objects detection and adding weight functionality and algorithms for detecting objects on the video and adding weight / force to video parts with detected objects; and- Force detection functionality and algorithms for calculating shear force and push force.

[0043] The method for video processing comprises human pose detection functionality and algorithms for detecting core points of the human body, allowing for accurate identification of body movements and potential safety hazards.

[0044] The events detection functionality is for detecting core events such as pushing, pulling, bending, and lifting, which may provide valuable insights into workplace tasks and potential safety hazards.

[0045] The force detection functionality is for calculating shear force and push force and may allow for accurate measurement of physical strain on the body, reducing the risk of workplace injuries and improving overall safety.

[0046] In a development, the application provides a comparison model for risk assessment, comprising:- a formula for comparing two videos and calculating overall improvement; and- a display, such as in the form of cards with each body part improvement and line charts with a body part risk distribution.

[0047] The comparison model for risk assessment provides a formula for comparing two videos and calculating overall improvement, allowing for easy identification of areas for further improvement and increased safety.

[0048] The cards with each body part improvement and line charts with the body part risk distribution provide valuable insights into potential safety hazards and areas for improvement.

[0049] According to a further aspect there is provided a real-time processing browser comprising: a processor for processing video of a person performing a work activity to identify workplace tasks and identify potential safety hazards; and a display output for displaying the video and identified workplace tasks and safety hazards.

[0050] According to a further aspect there is provided method of real-time processing work activity comprising: processing a video of a person performing a work activity to identify workplace tasks and identify potential safety hazards; and displaying the video and identified workplace tasks and safety hazards.

[0051] Embodiments of the invention are associated with various advantages and / or technical effects.

[0052] The invention may comprise any one or more combinations of the above aspects.

[0053] In this specification the terms “comprising” or “comprises” are used inclusively and not exclusively or exhaustively.

[0054] Other aspects, preferred forms and advantages of the present invention will be apparent from the present specification including the detailed description, drawings and claims.BRIEF DESCRIPTION OF DRAWINGS

[0055] The present disclosure is illustrated by way of example and not limited in the accompanying figures in which like reference numerals indicate similar elements. Embodiments of the application will now be described with reference to the attached drawings, in which:Figure 1 is a block diagram of a system according to an embodiment of the present invention;Figure 2 is a flow chart of a method according to an embodiment of the present invention;Figure 3 is a screen shot of a mobile application implementing an embodiment of the method of Figure 2;Figure 4 is a screen shot of a mobile application implementing an embodiment of the method of Figure 2;Figure 5 is a screen shot of a mobile application implementing an embodiment of the method of Figure 2;Figure 6 is a screen shot of a mobile application implementing an embodiment of the method of Figure 2;Figure 7 is a screen shot of a personal computer or tablet computer implementing an embodiment of the method of Figure 2;Figure 8 is a screen shot of a mobile application implementing an embodiment of the method of Figure 2;Figure 9 is a screen shot of a personal computer or tablet computer implementing an embodiment of the method of Figure 2;Figure 10 is a screen shot of a personal computer or tablet computer implementing an embodiment of the method of Figure 2; andFigure 11 is a screen shot of a personal computer or tablet computer implementing an embodiment of the method of Figure 2.DETAILED DESCRIPTION

[0056] The invention can take form of an entirely hardware embodiment, an entirely software embodiment or an embodiment containing both hardware and software elements. In a preferred embodiment, the invention is implemented in software, which comprises but is not limited to firmware, resident software, microcode, etc. Furthermore, the invention can take the form of a computer program product accessible from a computer-usable or computer-readable medium providing program code for use by or in connection with a computer or any instruction execution system. For the purposes of this description, a computer-usable or computer-readable can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.

[0057] Referring to Figure 1 , generally, there is a body movement / position assessment / modelling tool 10 (or system) comprising: a mobile device 12 for video recording of a subject; a processor 14 for analysing the video recording to produce measurements of risk movements; and a display 16 for viewing the recorded video and a representation of the measurements of risk movements.

[0058] The processor 14 is configured to assess changes in risk of movements between different movements. The mobile device 12, processor 14 and display 16 may be comprised in a single package, such as a smartphone, or these elements may be distributed, such as the processor 14 being ‘cloud’ based and the display 16 being of a tablet or desktop computer. The elements may for example communicate via network 18, such as the Internet, or internally within a single device via a bus and / or via messages.

[0059] In an example embodiment described and as shown in Figure 1, the elements are distributed. The mobile device 12 uploads the recorded video to the cloud- implemented processor 14. This may be a live feed from video captured by a camera ofthe mobile device 12 and stored (recorded) in the processor 14, or sent from a recording stored on the mobile device 12.

[0060] The processor 14 is configured to apply processing, such as in the form of algorithms or machine learning modules to produce the measurements of risk movements of a subject in the video recording(s).

[0061] The tool 10 is useful as a solution for workplace safety and may in some embodiments comprise data of workplace tasks and Al-driven data analysis for measuring workplace risk movements and implementing controls for the workforce. Video processing by the example tool 10 comprises human pose detection functionality, algorithms for detecting core points of the human body, classifying front / side / back views, and calculating body parts angles. The video processing also may comprise one or more of activity detection, video risk assessment, and twist highlights. Furthermore, it may comprise using one or more of RULA and REBA methodologies, events detection functionality, core events identification, and objects detection. The video processing may also comprise weight functionality for detected objects, force detection functionality, and shear force analysis. The comparison model for risk assessment compares videos, calculates overall improvement, and presents results. The results may be presented in cards, body part improvement line charts, and graphs of body part's risk distribution. The tool can be used for office work posture assessment by conducting real-time processing, immediate notifications, with computer vision algorithms for posture analysis, deviation detection, and ideal position calibration. This tool example may also comprises neck position check, shoulder symmetry check, eye to screen distance check, and back position check.

[0062] The tool 10 may be implemented by providing an application comprising a computer program for controlling the mobile device 12 and / or processor 14 to perform according to the requirements of the tool 10. The computer program may be in the form of instructions which control a CPU or other processing unit of the device or processor 14. The instructions may stored in a non-volatile manner, such as in a solid state memory or other storage device.

[0063] Referring to Figure 2, a method 50 is shown that for example uses tool 10, which commences with recording or selecting a video at 52. For example, a video may berecorded using a camera of a smart phone, or an existing recording that is stored in the smartphone may be selected. Figure 3 shows an example of selection of and then upload of selected videos.

[0064] The processor 14 then analyses the video at 54 by performing certain processing of the video.

[0065] The processing by the processor 14 may comprise identifying of an image of a person as distinguished from background in the video recording. That is, it identifies a region in each video frame that is of a human, rather than necessarily determining the identity of the person (although it may also do so). There are known algorithms available to do this. The processing may also identify at 62 body parts of the person in the video recording. Again, there are known algorithms available to do this. In an example the parts may comprise the head, shoulders, arms, trunk and legs of the person. Figure 5 shows an example of lines identifying forearms, upper arms, line between shoulders, lines between respective side’s shoulders and hips, line between hips, upper legs and lower legs. Image recognition processing can be used to determine the location of these lines in each video frame.

[0066] In an embodiment the analysis comprises human pose detection. Pose detection may be determined by for example processing the lines or by direct image processing to identify a pose of the person in the video. For example, in Figure 5 the person is leaning to their left with their arms above their head. In an embodiment human pose detection comprises detection at 64 of features comprising core points of human body in each frame of the video recording. In the image of a video in Figure 5, the points detected are wrists, elbows, shoulders, hips, knees, and ankles. In an embodiment detection of core points comprises detection of a person in the video recording, feature generation of the detected person, and body part angle calculation. Objects may also be detected, such as an object being lifted / pushed, or a display monitor or other equipment that a user is interacting with.

[0067] In an embodiment the analysis comprises activity detection at 66 (such as bending sideways), and risk calculation for each activity. For example, in Figure 5, a risk calculation of back angle being 132° has been calculated. This in turn or instead may be used to determine a risk score.

[0068] The analysis 54 may comprise one or more of: posture analysis, or distance checking, or deviation detection, or neck position checking, or shoulder symmetry checking, or eye level checking, or position calibration, or posture analysis.

[0069] In an embodiment at 56 the recorded video is displayed with one or more charts (or other display format) showing one or more of the measurements of risk with respect to time. In an embodiment at 58 at least two different parts of the video recording, or two different video recordings are shown in relation to each other, such as side by side, with the measurements (charts) also shown in relation, such as side by side, in order for a comparison to be made.

[0070] In an embodiment the display comprises an interpretation of the respective measurements (such as in the form of a risk score) and / or a comparison of the respective measurements.

[0071] In an embodiment the display comprises body part risk scores for a selected time in the video recording. Risk scores or ratings may be calculated as described further below. In Figure 4 parts of the video where a body part is preforming a higher risk movement (in this case medium risk movement) with the section of the video in which the risky movement is preformed is isolated (as a clip) and made available for display to demonstrate to the user, or other person the situation in which the risky movement is being performed. In the example shown in Figure 4, an overall general risk score (number) and (word form) rating, and various movements is displayed, as well as a risk score and rating, for individual body parts, are calculated and displayed for the video(s).

[0072] In an embodiment the processor is configured to identify events from the measurements of the body parts and / or the processor is configured to identify events by use of image recognition. Events may be for example picking, pushing, bending and / or twisting.

[0073] In an embodiment the processor may be configured to cut the video into clips of identified movements of the person. In this manner each clip of each movement can be used to show the risk involved with that particular movement within the video of the task being performed. For example, the task might be to pick up a box and put it on a shelf.The first clip might be the bending down movement to grab the box, the next clip might be movement to lift up the box and the final clip might be the movement to placement of the box on shelf. An algorithm of the processor may be employed to automatically identify discrete movements in the video and to segment the video into the clips according to the identified discrete movements. The algorithm may be a neural network classifier trained to identify each type of movement, or a rules based classifier or a combination.

[0074] In an embodiment the display comprises recommended changes to reduce the risk of injury for high risk potions of the video recording or of selected part of the video recording. For example, in Figure 5, the detection of a high back angle triggers the display of a suggested / recommended change (under the heading ‘How to Fix’).

[0075] When two videos are loaded for comparison they can be compared. The latter one of the comparisons may be compared on the basis of being an improvement in which riskier movements are attempted to be corrected. The comparison may then be an indication on whether there has been an improvement and optionally the degree of improvement. For example, in Figure 6 the overall improvement between movements in two videos is indicated as -30%. Individual body parts also have their improvements quantified. In this case the body parts are neck bending, back flexion, back twist, left arm bending, right arm bending, left leg bending and right leg bending. In this comparison there are improvements (as indicated by negative values and in green) in neck bending and left arm bending, but worsening (as indicated by positive values and in red) of back flexion and both leg bending. Further the change in the degree of risk is also indicated, such as high to low regarding neck bending, low to medium regarding back flexion, medium to low for left arm bending, low to medium for left leg bending and remaining low risk for right leg bending.

[0076] In an embodiment the processing selects portions of the video recording representing higher risk movements. In an embodiment the risk of movements is calculated over time and displayed over time. For example, Figure 7 shown the degree of risk shown over time as a graph for the overall risk and as different colours signifying different degrees of risk as a percentage of time in bars for each body part. Further a frontal human silhouette image is displayed with risk scores and colours showingaverage or peak degrees of risk of movements recorded in the video. The display may also show the time spent in a risky position for one or more of the body parts.

[0077] An example of rules based risk identification for body parts angles can be created from the table below.

[0078] The display of the video with the measurement and or the companion is a useful tool to provide feedback to the worker and of the benefit (or otherwise) of changes made to the way a task is performed.

[0079] For example, a task of putting boxes on a self may be video recorded, as per Figure 8, where different parts of the movement in that action are shown. In one instance of the lifting technique to put the box on the shelf the box is placed higher than in another instance (see the respective top images of a person lifting the box in the respective instances in Figure 9).

[0080] Each video (or a clip thereof) is processed and the degree of risk of each part of the movement calculated, on an overall basis which is displayed in the graphs over time and for each body part, which are displayed as percentages of degree of risk as a proportion of the whole task. It can be seen that the left high (above the head) lift is more risky than the right (waist height) lift. Each video can be played to track the risk as a given moment in time that the video is up to. This allows visualization of the what the person is actually doing and what the risk is of doing it. Further the side by side comparison allows understanding of the difference in risk between the different techniques, both overall and at a specific moment during the overall movement to perform the task.

[0081] Referring to Figure 10, another version of comparison display is shown wherein the risk of each version of the video is compared (as a before and after comparison).

[0082] The comparison table shows how the risk is allocated and how the method affects overall improvement, demonstrating both risk improvement by each body part and overall improvement in percentage.

[0083] Comparison by each body part helps to understand improvement in each body such as part (neck, shoulder, spine). For example, Figure 11 shows comparison of 2 videos of the task cleaning and back risk on the left video is higher than on the right video, thus, in Figure 10 it shows 75% risk reduction.

[0084] Figure 11 shows another comparison of videos of the task of cleaning, with the left being of cleaning without assistance of a tool and the right being with the assistance of a tool. The first risk analysis identified high risk movement that make the subject of alteration to reduce the risk. The comparison of risk assessments allows quantification of the improvement (or otherwise) as a result of alteration of the movements in performance of the task (in this case of cleaning) and may also identify further room for improvement.

[0085] The processing may allow comparison of one point in time against another point in time of the respective videos (or clips) but also may allow comparison of type of movement against the same type of movement, even if the same movement (or substantially the same, subject to an alteration) is at a different point in time in the respective videos. The processing can align the types of movement with each video clip so that both can be seen in relation to each other.

[0086] Typically, the type of movement identified and then one or more types of an ergonomic assessment are made. Types of ergonomic assessment comprise an eye to screen level / height assessment; neck angle or position check; body parts angles, comprising twisting and bending; distance to an object (such as eye to a computer screen); seated / standing work assessment, or seated / standing posture or other pose; weight or force detection functionality, comprising shear force and push force; ergonomic event detection, such as lifting; ideal position calibration; and shoulder symmetry check.

[0087] The ergonomic assessments are for identification of body movements, measurement of physical strain / stress on the body, movement event determination, risk scoring and identification of potential safety hazards.

[0088] The video may be annotated with a label of a task.

[0089] In another aspect there is a comparison model for risk assessment, comprising a processor for comparing two videos and calculating overall improvement and / or improvements to individual body part strain / stress.

[0090] The comparison may be displayed in the form of cards with each body part improvement and line charts with a body part’s risk distribution. Figures 4, 6, 7, 9 to 11 show examples of cards.

[0091] The comparison allows for easy identification of areas for improvement and increased safety.

[0092] In a development, the application provides a job dictionary of categorized activities for evaluating worker's ability to perform a particular job, comprising:- Automatic job creation and pre-filling with some risk data;- Automatic task profiles creation;- Attaching video to task profile;- Providing job dictionary hierarchy; and- Adding one or more job roles to the dictionary.

[0093] The job dictionary for evaluating worker's ability to perform a particular job provides automatic job creation and pre-filling with some risk data, reducing the time and effort required for job creation.

[0094] The automatic task profiles creation and attaching videos to task profiles provide valuable insights into potential safety hazards and areas for improvement.

[0095] The job dictionary hierarchy and adding multiple job roles to task video analysis allows for efficient communication and collaboration between team members, improvingoverall safety in the workplace.

[0096] In another aspect there is a method of evaluating a seated / standing work assessment, comprising:- changing the risk model and algorithms of the assessment to evaluate seated / standing work posture according to a selected threshold; and- providing recommendations for improving seated / standing posture, comprising recommendations about distance between a person and a display monitor.

[0097] The recommendations provide valuable insights into potential safety hazards and areas for improvement. The selection of the threshold allows for continuous improvement and increased safety in the workplace.

[0098] In another aspect there is a model for real-time processing, comprising:- a processing engine for detecting body parts angles and calculating risks from a video feed;- an in-browser display of the body parts angles and calculating risks.

[0099] The processing engine allows for real-time processing of body part angles and risk calculation, providing immediate feedback to the user and allowing for quick adjustments to be made to improve posture and reduce the risk of injury.

[0100] The in-browser display provides a convenient and accessible platform for users to access the risks, allowing for easy integration into daily office routines and promoting consistent use.

[0101] The use of computer vision algorithms, posture analysis and deviation detection algorithms in the tool for office work posture assessment allows for a comprehensive assessment of posture and identification of key problem areas, leading to more targeted and effective feedback and coaching.

[0102] In another aspect there is a tool for office work posture assessment, comprising:- a coaching mode module for real-time processing and immediate notifications;- computer vision algorithms for posture analysis and deviation detection;- algorithms for ideal position calibration, neck position check, shoulder symmetry check, eye to screen distance check, and eye height level check; and- video recording and processing for building an interactive report that shows key problems and provides feedback.

[0103] The coaching mode module provides real-time processing and immediate notifications, allowing for timely feedback and coaching to improve posture and reduce the risk of injury.

[0104] The use of video recording and processing in the tool allows for the creation of an interactive report that shows problems and provides feedback, promoting user engagement and motivation to improve posture. The algorithms provide a comprehensive assessment of posture and identify specific areas for improvement, leading to more targeted and effective feedback and coaching.

[0105] The cloud-based solution for workplace safety comprises a mobile application for video recording, analysis, an online dashboard, and viewing capabilities. This solution comprises data of workplace tasks and Al-driven data analysis for measuring workplace risk movements and implementing controls for the workforce. The mobile application for video recording is part of the cloud-based solution, as are the analysis and online dashboard components.

[0106] Video processing within the cloud-based solution comprises human pose detection functionality, algorithms for detecting core points of the human body, classifying front / side / back views, and calculating body parts angles. The video processing also comprises of activity detection, video risk assessment, and twist highlights. Furthermore, it comprises RLILA and REBA methodologies, events detection functionality, core events identification, and objects detection.

[0107] The video processing also comprises weight functionality for detected objects, force detection functionality, and shear force analysis.

[0108] The comparison model for risk assessment uses a formula for comparing videos, calculating overall improvement, and presenting results in cards, body partimprovement line charts, and body part's risk distribution.

[0109] The model for real-time processing comprises a processing engine, detecting body parts angles, calculating risks, and serving as the basis for the inbrowser display. The processing engine and risks are part of the model for real-time processing.

[0110] The tool for office work posture assessment comprises a coaching mode for real-time processing, immediate notifications, computer vision algorithms for posture analysis, deviation detection, and unique algorithms for ideal position calibration. This tool also comprises of neck position check, shoulder symmetry check, eye to screen distance check, and back position check.Human pose detection functionality and algorithms

[0111] In an example the processor detects core points of human body using these algorithms:

[0112] 1. Focus on one or more persons: person detection, in 2d and 3d trained using Intersection over Union (loU) metric performance training.

[0113] 2. Human pose estimation improvements (based on open source libraries):(a) multiple tries using different boxes;(b) person keypoints interpolation;(c) 1-euro filtering for person keypoints / position smoothing;(d) Nose, eyes and ears restoration(e) Body flip detection for better understanding left / right of the body: distance between corresponding body keypoints, angles between body keypoints

[0114] 3. Features generation: classifying front / side / back view with 2d, 3d, machine learning (ML)

[0115] 4. Body parts angles calculation: 2d angles, 3d angles; formula using sin / cos or / and machine learning (ML) algorithm; improved by interpolation, smoothingand filtering

[0116] 5. Activity detection: 2d, 3d, machine learning (ML) to detect activity such as sitting, walking, squatting, bending

[0117] 6. Body parts and video risk calculation via the aggregated data using formula to calculate risks and some coefficients which make body parts more valuable

[0118] 7. Twist calculation: ML using 2d, 3d etc features

[0119] 8. Highlights detection: using 2d, 3d, ML detects the most risky parts of the video

[0120] 9. RULA / REBA prediction: using 2d, 3d, dataset with assessments, MLEvents detection functionality and algorithms

[0121] In an embodiment the processor detects core events like pushing, pulling, bending, lifting using these algorithms:

[0122] 1. Pretrained classification to classify highlights into events

[0123] 2. Detects events using 2d, 3d, machine learning (ML)Objects detection and adding weight functionality and algorithms

[0124] In one version a user will select video parts manually and enters the weight of an object being lifted.

[0125] In another version, the processor detects objects on the video using classic computer vision algorithms and neural networks and makes the weight detection process half automated - provide preselected video parts with objects. With object detection a user can add weight / force to one video part and it will be added automatically to other parts with detected object.Force detection functionality and algorithms

[0126] Formula shear force applied to the spine during picking:Shear Force = (Mass of Object x (Lifting Acceleration + 9.8 m / sA2)) x(Distance from Object to Spine I Distance from Feet to Spine) x (cos(0) + sin(0) x cos(a))- Distance from Object to Spine is the horizontal distance between the centre of object and the L1 / L2 vertebra- Distance from Feet to Spine is the vertical distance between the ankle and the L1 / L2 vertebra- 0 is the angle between spine and ground- a is the angle between spine and arm

[0127] Formula shear force applied to the spine during pushing:Shear force = (Coefficient of friction x Weight of object x cos(a) x tan(0))+ (Mass of object x Acceleration x cos(a) x Handle height I Distance from hand to spine)- a - is pull / push angle- 0 - is spine angle- Distance from hand to spine is the horizontal distance between the wrist and the L1 / L2 vertebra

[0128] The actual distance does not need to be measured, because only their ratio is important, so the distance in pixels can be used.

[0129] As an alternative or in addition to these formulas a trained ML algorithm may be used.Comparison model

[0130] This feature gives a user the ability to compare two videos, for example with the wrong and the right technique or before and after an intervention formula which compares each body part and calculate overall improvement. Two videos are selectedto be compared.

[0131] On a comparison screen / tab a user can see a score of the overall improvement. This number is calculated by a formula that sums the risks for each body part multiplied by the factors of importance, and then calculates the improvement for the comparison. There are cards with each body part improvement and line charts with a body part risk distribution.

[0132] In an embodiment the manner of calculating a risk rate for each body part in the form of computer program instructions is as follows:return risk__coef def get body part r isk rate (body part medium risk time ,return body__part__risk__rate

[0133] Get risk level for each body part:um else : body__part__risk__level : = Low

[0134] Get video risk rate: def get__vedeo risk rate (all body part risk rat.es) : return0 . 8 * (lef t__arm__risk__rat.e + right__arm__risk__rat.e) +SUBSTITUTE SHEET (RULE 26)0 .25 * (lef t__leg__risk__rate 4- right__leg__risk__rate)

[0135] Get video risk level (for Comparison section): def get__video__risk__level (all body__part__risk__rat.es) : video__risk__rate 0 video__risk__rate = c (all body__part__risk__rates) if video__risk__rate >= 2 : video__risk__level : = High el if video__risk__rate >= 1 : videe risk level : = Medium else : v Low return

[0136] Calculate body_part improvement (for Comparison section): def aalculate__improvement (risk__rate__f irst , risk__rate__seaond , risk__level__f irst , risk__level__second) : current__improvement = 0 if risk rate f irst > 0 : current improvement = risk__rate___second / risk__rate__f irst - 1new impr = 1 ~ risk__rate__first / risk__rate__second if risk__rats__f irst == 0 and risk__rate__seoond ? — 0 : new__impr = 1 if risk level f irst::::::Low and risk level seoond::::::Low : improvement = improvement * 0 . 15 elif risk lavel f irst ! = High and risk__level__second ! = High : improvement = improvement * 0 .4 return new__impr

[0137] Calculation formula for the whole video improvement: (for Comparison section): ov<srall__risk__rate__f irst = get__vedeo__risk__rate (all bcdy par i risk rates f i rst)(all body part risk ra tas saccEiid) ovsrall__risk__level__f irst:::ge t__video__risk__level (all body__part__risk__rat<s__f irst)SUBSTITUTE SHEET (RULE 26)overall__risk__level__seaond = get__video__risk__level (all body__part__risk__rate__second) overall_improvement = calculate__improvement ( overall__risk__rate__first, overall__risk__rate__second, overall__risk__level_first, overall__risk__level__second, )

[0138] The algorithms applied when video processing analyse the movements amplitudes and identify 3 risk zones I categories I ratings for different body parts by overlaying the body with red, yellow, or green lines representing the level of risk of the movement amplitude.

[0139] A Task Risk Group (TRG) for each body part (low, medium, or high) is assigned based on the average time each body part spends in various movement amplitude risk zones (low, medium, and high) during a task. The percentage of time each body part is in each risk zone is calculated by analysing the duration of the task.

[0140] For example, for the body part Back, and a task that lasts 30 seconds, with an 80% low-risk zone back movement, 20% medium-risk zone back movement, and O.This task would be classified as a Medium risk group for the Back. The Task Risk level is calculated using the sum of percentages for each body part in each risk zone, considering all six body parts.

[0141] Risks may be calculated for multiple body parts and / or for multiple movements that are part of the task. Risk can be calculated tasks comprising movements such as:-Bending-Lifting-Twisting the back-Sustained awkward postures-Arm elevation-Pushing & pulling movementsAnd idle movements like:-StandingSUBSTITUTE SHEET (RULE 26)-Walking- Sitting

[0142] And another movements such as:- StoopingFor spine, shoulder and knee risk scores may subject to low, middle and high risk.

[0143] Video generated data and body part measurement device data are collated to job logs for overall job role assessment. The Job Dictionary is automatically filled in based on the video generated data. When job profiles are created, they include the tasks and risks, which are calculated for every job profile and task profile.Job dictionary

[0144] Job Dictionary evaluates a worker's ability to perform a particular job comparing job roles and tasks, comprising a system of risk assessment for performing different tasks subject to low, middle and high risk for spine and shoulder body areas. This product collates device data from sensors attached to one or more body parts with video risk data to job logs for overall job role assessment, combines spine & shoulder data with risk estimation and create job profiles to quantify risk in each role and task, comparing the distribution of time and number of movements per hour in different angles, what risk corresponds to these tasks, percentages of time (for bending, twisting, arm elevation).

[0145] There is automatic creation of jobs and pre-filling it with some risk data. A user can create a Job Profile when a user adds a new Job Role. There can be default company categories as departments, sites, job roles and tasks. When a category job role is populated with job roles names, in the job dictionary job profiles are automatically created (job profiles correspond to job roles names).

[0146] There may also be automatic creation of task profiles. A user can create a Task Profile when a user adds a new Task Name to video or attach a task from a user page. When Job Roles are attached to a user, Task Profiles are created for each theirtasks. If a video has few Job Roles Task Profiles can be created for each Job Profile. If a user or video has no Job Role then a Profile need not be created.

[0147] The user can attach a video to a Task Profile. After the user uploads a video and sets Job Role and Task Name the Task Profile can be attached to it.

[0148] The user can provide a Job Dictionary hierarchy. User can see the Task Profile directly for a particular Job Role. A new Task Profile can be created for each case of attaching a Task Name to a Job Role.

[0149] The user can adding multiple Job Roles to a video. The same Task Profile can be created with this video for every added Job Role.Threshold changing to the seated / standing work assessment

[0150] As a seated / standing work neutral position is completely different from manual handling neutral position, so the risk model and algorithms of this assessment are different. The user can select another type of assessment (seated / standing work) and the processor will process the video with changed thresholds. Recommendations for a video will be directed to improving seated / standing posture comprising recommendations about distance between a person and a monitor.Live model

[0151] In a live model the processing engine starts with posture calibration. A user sets up their position and workstation in a proper way that is described in a step- by-step tutorial.

[0152] Then the system makes an assessment and analyses human posture using computer vision algorithms and runs real-time coaching where a user will be notified about each posture deviation to correct a posture.Computer vision algorithms

[0153] The MoveNet model can be used to extract key points from video. The key points are then passed through filters and used in algorithms to estimate posture deviation from the correct posture.

[0154] All algorithms take as input the sensitivity threshold and two postures: correct and current. The output of the algorithms is a logical value: true / false - whether the posture deviates enough or not. If the key points necessary for the algorithm to work are not found, it outputs a value of null.Ideal position calibration algorithm

[0155] We have several calibration steps with instructions on how to move to the ideal position to get the ideal position for each user. This algorithm helps to adapt to most personal differences and features and allows for a more personal analysis of a person's posture.

[0156] The video processing can be used to verify that the instructions have been followed and / or are effective.Neck position check

[0157] The head points of the current pose are centred around the centre of the head points of the correct pose. Then the average shift of the corresponding points is calculated. The shift is normalized by the distance between the shoulders of the correct pose and compared to the sensitivity threshold.Shoulder symmetry check

[0158] The angle between the shoulder vectors is calculated and compared to the sensitivity threshold.Eye to screen distance check

[0159] The relative change in eye distance of the current pose relative to the correct pose is calculated and compared to the sensitivity threshold.Eye height level check

[0160] The absolute change in average eye height is calculated, normalized by the distance between the eyes of the correct pose and compared to the sensitivity threshold.

[0161] Sensitivity thresholds are selected heuristically.

[0162] Modifications and variations as would be apparent to the person skilled in the art are intended to be fall within the scope of the present invention.

Claims

Claims1. A body movement / position assessment tool comprising: a mobile device for video recording of a subject; a processor for analysing the video recording to produce measurements of risk movements; a display for viewing the recorded video and a representation of the measurements of risk movements; wherein the processor is configured to assess changes or differences in risk movements between different risk movements.

2. A method of posture / movement modelling comprising: video recording a subject person with a mobile device; analysing the video recording to produce measurements of risk movements; assessing changes in risk between different risk movements; displaying the recorded video, a representation of the measurements of risk movements and the changes in risk between the different risk movements.

3. A work modelling tool comprising: a mobile device for video recording a person performing an activity; a processor for analysing the video recording to classify the activity of the person as a type of work being performed by the person in the video recording; and an output of the identified type of work.

4. A method of work modelling comprising: video recording a person performing an activity; analysing the video recording to classify the activity of the person as a type of work being performed by the person in the video recording; and outputting the identified type of work.

5. A cloud-based solution for workplace safety, comprising: a mobile application for video recording and analysis; an online dashboard for viewing video and data of workplace tasks;Al-driven data analysis for measuring workplace risk movements and implementing controls to keep the workforce safe.

6. A real-time processing browser comprising: a processor for processing video of a person performing a work activity to identify workplace tasks and identify potential safety hazards; and a display output for displaying the video and identified workplace tasks and safety hazards.

7. A method of real-time processing work activity comprising: processing a video of a person performing a work activity to identify workplace tasks and identify potential safety hazards; and displaying the video and identified workplace tasks and safety hazards.

Images