Double-camera sit-up test method and device based on computer vision

Abstract

According to the double-camera sit-up test method and device based on computer vision, double cameras are used for parallel detection and counting based on a human body key point detection model, and voice broadcasting, test start judgment, violation judgment, accurate counting and real-time score reporting can be carried out on testers under the unsupervised condition; the problems that a manual sit-up test method is high in manpower overhead and large in test standard deviation can be solved, and the problems that a traditional contact type sensor sit-up test method is low in counting accuracy and poor in compatibility with different subjects, and a contact type device interferes with movement of the subjects are solved. The problem that the real-time performance is not considered during violation judgment and score reporting voice notification of a sit-up test method after video acquisition is solved, and the problem that single-hand violation detection of a single-camera sit-up test method is inaccurate is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision, and in particular to a dual-camera sit-up test method and device based on computer vision. Background Art

[0002] The existing sit-up test methods mainly include: manual sit-up test method, traditional contact sensor sit-up test method, video post-processing sit-up test method, and single camera sit-up test method. These methods have problems: high manpower cost, different test standards, low test efficiency, interference test, safety hazards, poor real-time performance, inaccurate violation judgment, etc.

[0003] 1. Manual sit-up test method: The manual sit-up test method uses a one-to-one or one-to-many ratio of referees to testers to perform sit-up tests. This method has the following problems: (1) High manpower cost: multiple referees or a single referee is required to supervise for a long time; (2) Inconsistent test standards: different referees have different counting standards, or fatigue may cause referees to relax in counting standards; (3) Difficulty in reviewing and verifying test results: testers cannot review their own test process for verification and summary.

[0004] 2. Traditional contact sensor sit-up test method: The traditional contact sensor sit-up test method is to test the tester by wearing sensor clothing or directly contacting the sensor device. This method has the following problems: (1) Interference test: Wearing sensor clothing will bring additional exercise to the tester, affecting the test results; (2) There are safety hazards: Wearable clothing or sensor devices may carry electric current, which may pose a risk to the life of the tester; (3) Inaccurate violation judgment: The inability to detect the hand position leads to illegal actions.

[0005] 3. Video post-processing sit-up test method: The video post-processing sit-up test method collects the video of the entire test process of the tester and saves it offline, and then calculates the score by analyzing and judging the offline video. This method has the following problems: (1) Lack of real-time performance: During the test, the tester cannot know in real time whether the sit-ups done in the current round meet the action standards and whether they are included in the final score. This may cause the tester to be unable to adjust the action in time to meet the standards during the test.

[0006] 4. Single-camera sit-up test method: The single-camera sit-up test method performs video acquisition and analysis counting by setting a camera on one side of the test subject. There are problems with this method: (1) Inaccurate violation determination: Obvious errors will occur in the analysis and counting of information such as key points of the arms and legs on the occluded side. This error may lead to the failure to recognize violations of hand or leg movements on that side, resulting in score errors. Summary of the Invention

[0007] In order to overcome the above technical deficiencies, the present invention provides a human key point detection technology, and a double-camera sit-up test method and device with features such as intelligent detection, automatic counting, and violation determination.

[0008] The technical solution adopted by the present invention to overcome its technical problems is: A double-camera sit-up test method based on computer vision, comprising: Define a test area, and install cameras on both sides of the test area. The cameras continuously collect the pose key point information of the test subject from the left and right perspectives. Calibrate the test area under the perspectives of the two cameras to obtain a calibrated area. The test subject sets the timing duration. When the test subject enters the test area and is in the sit-up preparation posture, the test subject is reminded by voice to start the test, and the timing starts. Use the collected pose key point information of the test subject to determine whether the test subject is in a lying state or a sitting-up state. When the test subject performs one time from the lying state to the sitting-up state, it is determined that one sit-up is completed and counted. If it is determined that the test subject has a violation action, this action is not counted. After the timing duration is completed, the number of sit-ups is announced to the test subject.

[0009] Furthermore, the distance between the two cameras is 4 - 5 meters, the height between the two cameras and the ground is 0.5 - 1.3 meters, and the connection line between the two cameras is perpendicular to the lying direction of the test subject and passes through the center of the test area.

[0010] Furthermore, the method for calibrating the test area under the perspectives of the two cameras is: Establish a quadrilateral area by setting the four corner points of the quadrilateral area. This quadrilateral area is used as the calibrated area. The whole body of the test subject is located within the quadrilateral area during the test. The part outside the calibrated area is removed by masking.

[0011] Furthermore, after the test subject sets the timing duration, the test subject is notified to get ready by voice.

[0012] Furthermore, after the tester enters the test area, the tester's preparation posture is determined to be completed when the distance between the tester's left wrist key point and the nose key point in the left camera perspective is less than the distance between the left shoulder key point and the left elbow key point, and the distance between the tester's right wrist key point and the nose key point in the right camera perspective is less than the distance between the right shoulder key point and the right elbow key point, and the angle formed by the tester's left ankle key point, left knee key point and left hip key point in the left camera perspective is less than the preset maximum threshold of 120 degrees and greater than the preset minimum threshold of 60 degrees, and the angle formed by the tester's right ankle key point, right knee key point and right hip key point in the left camera perspective is less than the preset maximum threshold of 120 degrees and greater than the preset minimum threshold of 60 degrees.

[0013] The test person is judged to be in a lying position when the angle formed by the test person's left knee key point, left hip key point and left shoulder key point in the test area from the perspective of the left camera is greater than the preset supine threshold of 80 degrees, and the angle formed by the test person's right knee key point, right hip key point and right shoulder key point in the test area from the perspective of the right camera is greater than the preset supine threshold of 80 degrees.

[0014] Furthermore, the tester is judged to be in a sitting-up state when the angle formed by the tester's left knee key point, left hip key point and left shoulder key point in the test area under the perspective of the left camera is less than a preset sitting-up threshold of 10 degrees, and the angle formed by the tester's right knee key point, right hip key point and right shoulder key point in the test area under the perspective of the right camera is less than a preset sitting-up threshold of 10 degrees.

[0015] Furthermore, if the tester in the test area satisfies the conditions that the distance between the tester's left wrist key point and the nose key point in the left camera's perspective is less than the distance between the left shoulder key point and the left elbow key point, and the distance between the tester's right wrist key point and the nose key point in the right camera's perspective is less than the distance between the right shoulder key point and the right elbow key point, and the angle formed by the tester's left ankle key point, left knee key point and left hip key point in the left camera's perspective is less than the preset maximum threshold of 120 degrees and greater than the preset minimum threshold of 60 degrees, and the angle formed by the tester's right ankle key point, right knee key point and right hip key point in the right camera's perspective is less than the preset maximum threshold of 120 degrees and greater than the preset minimum threshold of 60 degrees, then the tester is determined to have violated the regulations.

[0016] A device for a dual-camera sit-up test method based on computer vision, characterized in that it includes: An image acquisition module is used to obtain images from two cameras for frame capture and frame extraction operations; The voice broadcast module is used to broadcast the preparation prompt, timing start, real-time counting during the test, end timing, and the number of sit-ups; A calibration module is used to calibrate the test area under the dual camera viewing angle to obtain a calibration area; Timing module, used to set the duration for timing; The test module is used to analyze the tester's movements, determine whether the tester is in a lying or sitting state, and determine whether the tester has any illegal movements; The communication module is used for communication between the image acquisition module, the voice broadcast module, the calibration module, the timing module and the test module.

[0017] The beneficial effects of the present invention are: based on the human key point detection technology in the field of computer vision, a dual-camera sit-up test device and method with the characteristics of intelligent detection, automatic counting, violation determination, etc. is realized. The present invention is based on the human key point detection model, uses dual cameras for parallel detection and counting, can perform voice broadcasting, test start determination, violation determination, accurate counting, and real-time score reporting for test personnel under unsupervised conditions, can avoid the problems of high manpower cost and large test standard variance of the manual sit-up test method, avoid the problems of low counting accuracy, poor compatibility with different subjects, and contact device interference with subject movement of the traditional contact sensor sit-up test method, avoid the problem of not considering real-time in violation determination and score reporting voice notification of the sit-up test method after collecting video, and avoid the problem of inaccurate one-handed violation detection of the single-camera sit-up test method. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a flow chart of the method of the present invention; Figure 2 A top view of the placement of the camera and the test area of ​​the present invention; Figure 3 It is a side view of the placement of the camera and the test area of ​​the present invention. DETAILED DESCRIPTION

[0019] The following is combined with Figure 1 , Attachment Figure 2 , Attachment Figure 3 The present invention is further described.

[0020] As attached Figure 1 A dual-camera sit-up test method based on computer vision is shown, comprising: Delineate the test area, such as Figure 2As shown, cameras are installed on both sides of the test area. The cameras collect the key point information of the left and right perspectives of the tester in real time. When placing the cameras, it is necessary to ensure that the test area is always within the captured image screen and is not blocked. The tester does not have direct contact with any test equipment during the entire test process.

[0021] Calibrate the test area under the dual camera viewing angles to obtain the calibration area.

[0022] The tester sets the timing duration. In the preparation phase, the communication module will notify the image acquisition module to start, that is, to collect images from two cameras in parallel, and the image acquisition module will run until the test is completed. Once the preparation phase begins, the dual cameras will collect image information from the left and right sides of the tester in real time, and obtain the key points of the human body on the left and right sides based on the computer vision model reasoning (the number of all key points of the human body obtained by a single camera is 26) and return the information to the test module.

[0023] When the tester enters the test area and is in the sit-up preparation posture, the voice reminds the tester to start the test and starts the timing. Test start judgment: When the tester enters the test area and is in the sit-up preparation posture, the voice broadcast module will notify the tester to start the test, and the timing module will start the timing module to start the timing. When the tester lies down, he only needs to ensure that there are cameras on both sides, and there is no need to consider the direction of his head.

[0024] The key point information of the test person's posture is collected to determine whether the test person is in a lying state or a sitting state. When the test person performs a sit-up from a lying state to a sitting state, it is considered to have completed one sit-up and counted. If the test person is judged to have violated the rules, the action will not be counted.

[0025] After the timing is completed, the number of sit-ups will be broadcast to the tester. After the timing module completes the preset timing, it will enter the completion stage. During the completion stage, the voice broadcast module will notify the tester that the timing is over and broadcast the tester's test results in real time. At the same time, the communication module will save the test video offline and synchronously display the current time test number for the tester to summarize and verify.

[0026] The dual-camera sit-up test device and method based on computer vision, based on the human key point detection technology in the field of computer vision, realizes a dual-camera sit-up test device and method with the characteristics of intelligent detection, automatic counting, violation judgment, etc. The present invention is based on the human key point detection model, uses dual cameras for parallel detection and counting, and can perform voice broadcasting, test start judgment, violation judgment, accurate counting, and real-time score reporting for test personnel under unsupervised conditions.

[0027] In one embodiment of the present invention, the distance between the two cameras is 4-5 meters, the height between the two cameras and the ground is 0.5-1.3 meters, and the line between the two cameras is perpendicular to the lying direction of the test person and passes through the center of the test area.

[0028] In one embodiment of the present invention, the method for calibrating the test area under the dual camera viewing angle is: A quadrilateral area is established by setting the four corner points of the quadrilateral area, and the quadrilateral area is used as the calibration area. The calibration order of the four corner points can be set by yourself, and the calibration module will automatically sort them according to the relative positions of the corner points. When calibrating the test area, it is necessary to ensure that the tester can be in the quadrilateral area with his whole body during the entire test process, and the two cameras need to calibrate the test area under their perspective respectively. The part outside the calibration area is eliminated by masking. Therefore, the activities of people who are within the camera's perspective but outside the test area will not affect the test process of people in the test area.

[0029] In one embodiment of the present invention, after the tester sets the timing duration, the tester is notified to prepare by voice.

[0030] In one embodiment of the present invention, after the tester enters the test area, the tester's preparation posture is determined to be completed when the distance between the tester's left wrist key point and the nose key point under the left camera perspective is less than the distance between the left shoulder key point and the left elbow key point, and the distance between the tester's right wrist key point and the nose key point under the right camera perspective is less than the distance between the right shoulder key point and the right elbow key point, and the angle formed by the tester's left ankle key point, left knee key point and left hip key point under the left camera perspective is less than a preset maximum threshold of 120 degrees and greater than a preset minimum threshold of 60 degrees, and the angle formed by the tester's right ankle key point, right knee key point and right hip key point under the left camera perspective and the right camera perspective is less than the preset maximum threshold of 120 degrees and greater than the preset minimum threshold of 60 degrees.

[0031] In one embodiment of the present invention, the test person is determined to be in a lying state when the angle formed by the test person's left knee key point, left hip key point, and left shoulder key point in the test area under the perspective of the left camera is greater than a preset supine threshold of 80 degrees, and the angle formed by the test person's right knee key point, right hip key point, and right shoulder key point in the test area under the perspective of the right camera is greater than a preset supine threshold of 80 degrees.

[0032] In one embodiment of the present invention, the tester is determined to be in a sitting state when the angle formed by the tester's left knee key point, left hip key point, and left shoulder key point in the test area under the perspective of the left camera is less than a preset sitting-up threshold of 10 degrees, and the angle formed by the tester's right knee key point, right hip key point, and right shoulder key point in the test area under the perspective of the right camera is less than a preset sitting-up threshold of 10 degrees.

[0033] After the tester enters the test phase, the test module will use the key point information of the human body obtained by reasoning to analyze the tester's lying and sitting states, and count the number of sit-ups. At the same time, the test module will also use the key point information to analyze whether the tester has made any illegal actions. The number of sit-ups completed under illegal actions will not be counted in the total score. The voice broadcast module will use real-time voice to inform the tester of the number of sit-ups currently completed throughout the sit-up process, so as to avoid the situation where the tester is unaware of the illegal action and does not count. Specifically, in one embodiment of the present invention, the tester is judged to have violated the rules when the distance between the left wrist key point and the nose key point of the tester in the test area under the perspective of the left camera is less than the distance between the left shoulder key point and the left elbow key point, and the distance between the right wrist key point and the nose key point of the tester in the perspective of the right camera is less than the distance between the right shoulder key point and the right elbow key point, and the angle formed by the left ankle key point, left knee key point and left hip key point of the tester in the perspective of the left camera is less than the preset maximum threshold of 120 degrees and greater than the preset minimum threshold of 60 degrees, and the angle formed by the right ankle key point, right knee key point and right hip key point of the tester in the perspective of the right camera is less than the preset maximum threshold of 120 degrees and greater than the preset minimum threshold of 60 degrees.

[0034] The present invention also relates to a device for a dual-camera sit-up test method based on computer vision, characterized in that it comprises: The image acquisition module is used to obtain images from two cameras for frame capture and frame fetching operations. This module is executed in parallel with other modules, and frames are transmitted through the communication module. The obtained frames will be used for real-time testing of video streams and offline storage of video streams. At the same time, this module can warn when no frames are entered for a long time.

[0035] The voice broadcast module is used to voice broadcast the preparation prompts, timing start, real-time counting during the test, end timing, and number of sit-ups.

[0036] The calibration module is used to calibrate the test area under the dual camera view to obtain the calibration area. For any camera to collect images, the calibration module will automatically process the four corner points of the test area with the calibration order disrupted, and pass it to the test module through the communication module to limit the test area during the test to avoid interference from other people in the camera.

[0037] The timing module is used to set the duration for timing.

[0038] The test module is used to analyze the tester's movements, determine whether the tester is lying down or sitting up, and determine whether the tester has any illegal movements. It realizes the functions of motion analysis, starting judgment, detection counting, and violation judgment of the tester. Based on the human key point detection model in computer vision, the human key points are obtained by inputting the pre-processed (cropped, masked, filled, etc.) input acquisition image into the model for inference, and then the current state of the tester (supine state, sitting up state, violation state, leaving state) is analyzed and judged through the combination of these key points to realize further counting, violation judgment and other functions.

[0039] The communication module is used for communication between the image acquisition module, the voice broadcast module, the calibration module, the timing module and the test module.

[0040] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A dual-camera sit-up test method based on computer vision, It is characterized in that include: Define the test area and install cameras on both sides of the test area. The cameras collect the key point information of the tester's posture from the left and right perspectives in real time. Calibrate the test area under the dual camera viewing angle to obtain the calibration area; The tester sets the timing duration; When the tester enters the test area and is in the sit-up preparation posture, a voice reminds the tester to start the test and starts the timer; The key point information of the tester's posture is collected to determine whether the tester is in a lying state or a sitting state. When the tester performs a sit-up from a lying state to a sitting state, it is considered to have completed one sit-up and counted. If the tester is judged to have violated the rules, the action will not be counted. After the timing duration is completed, the number of sit-ups will be reported to the tester.

2. The dual-camera sit-up test method based on computer vision according to claim 1, Features: The distance between the two cameras is 4-5 meters, the height between the two cameras and the ground is 0.5-1.3 meters, and the line between the two cameras is perpendicular to the direction in which the test person lies and passes through the center of the test area.

3. The dual-camera sit-up test method based on computer vision according to claim 1, It is characterized in that The method for calibrating the test area under the dual camera perspective is: A quadrilateral region is established by setting four corner points of the quadrilateral region, and the quadrilateral region is used as a calibration region; During the test, the tester's whole body is within the quadrilateral area; The parts outside the calibration area are eliminated by masking.

4. The dual-camera sit-up test method based on computer vision according to claim 1, Features: After the tester sets the timing duration, the tester is notified by voice to prepare.

5. The dual-camera sit-up test method based on computer vision according to claim 1, Features: After the tester enters the test area, the tester's preparation posture is completed when the distance between the tester's left wrist key point and the nose key point from the left camera's perspective is less than the distance between the left shoulder key point and the left elbow key point, and the distance between the tester's right wrist key point and the nose key point from the right camera's perspective is less than the distance between the right shoulder key point and the right elbow key point, and the angle formed by the tester's left ankle key point, left knee key point and left hip key point from the left camera's perspective is less than the preset maximum threshold of 120 degrees and greater than the preset minimum threshold of 60 degrees, and the angle formed by the tester's right ankle key point, right knee key point and right hip key point from the left camera's perspective is less than the preset maximum threshold of 120 degrees and greater than the preset minimum threshold of 60 degrees.

6. The dual-camera sit-up test method based on computer vision according to claim 1, Features: The test person is judged to be in a lying position when the angle formed by the test person's left knee key point, left hip key point and left shoulder key point in the test area from the perspective of the left camera is greater than the preset supine threshold of 80 degrees, and the angle formed by the test person's right knee key point, right hip key point and right shoulder key point in the test area from the perspective of the right camera is greater than the preset supine threshold of 80 degrees.

7. The dual-camera sit-up test method based on computer vision according to claim 1, Features: The test person is judged to be in a sitting state when the angle formed by the test person's left knee key point, left hip key point and left shoulder key point in the test area of ​​the left camera's perspective is less than a preset sitting-up threshold of 10 degrees, and the angle formed by the test person's right knee key point, right hip key point and right shoulder key point in the test area of ​​the right camera's perspective is less than a preset sitting-up threshold of 10 degrees.

8. The dual-camera sit-up test method based on computer vision according to claim 1, Features: The tester is judged to have violated the rules if, in the test area, the distance between the tester's left wrist key point and the nose key point from the left camera's perspective is less than the distance between the left shoulder key point and the left elbow key point, the distance between the tester's right wrist key point and the nose key point from the right camera's perspective is less than the distance between the right shoulder key point and the right elbow key point, the angle formed by the tester's left ankle key point, left knee key point and left hip key point from the left camera's perspective is less than the preset maximum threshold of 120 degrees and greater than the preset minimum threshold of 60 degrees, and the angle formed by the tester's right ankle key point, right knee key point and right hip key point from the right camera's perspective is less than the preset maximum threshold of 120 degrees and greater than the preset minimum threshold of 60 degrees.

9. A device for implementing any one of the computer vision-based dual-camera sit-up test methods of claims 1 to 8, It is characterized in that include: An image acquisition module is used to obtain images from two cameras for frame capture and frame extraction operations; The voice broadcast module is used to broadcast the preparation prompt, timing start, real-time counting during the test, end timing, and the number of sit-ups; A calibration module is used to calibrate the test area under the dual camera viewing angle to obtain a calibration area; Timing module, used to set the duration for timing; The test module is used to analyze the tester's movements, determine whether the tester is in a lying or sitting state, and determine whether the tester has any illegal movements; The communication module is used for communication between the image acquisition module, the voice broadcast module, the calibration module, the timing module and the test module.

Images