Automatic detection method and device for ankle flexibility based on computer vision

Abstract

The present invention belongs to the technical field of computer vision, and discloses an automatic detection method and device for ankle flexibility based on computer vision, which collects the videos of ankle dorsiflexion and plantar flexion and ankle inversion and eversion of the subject's ankle joint in succession, extracts the position images of the first metatarsal, ankle and sole plane; judges the ankle flexibility of the subject according to the maximum dorsiflexion angle and plantar flexion angle of the subject; and judges the ankle flexibility of the subject according to the maximum eversion angle and maximum inversion angle of the ankle joint of the subject. The present invention is oriented to two tests of the maximum angle of ankle dorsiflexion and plantar flexion and the maximum angle of inversion and eversion, and detects the changes of angles and positions in real time by quickly extracting the positions of the first metatarsal, ankle and sole plane, which can reduce the workload of the evaluation personnel, reduce the equipment cost, improve the evaluation efficiency, and improve the stability and accuracy of the test results.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and in particular relates to a method and device for automatically detecting ankle flexibility based on computer vision. Background Art

[0002] The ankle is an important joint that is indispensable for the human body to achieve walking, running, jumping and other movements. Its flexibility and stability are basic and important indicators for the assessment of ankle joint function. The ankle flexibility measuring instrument mainly measures the range of motion of the ankle joint, including the motion arc (trajectory) of the links that constitute the joint during joint movement or the angle of rotation of the links around a certain joint axis. The measurement of ankle joint range of motion generally includes dorsiflexion, plantar flexion, inversion, eversion and other angles. The process in which the dorsum of the foot approaches the calf and foot and the angle between the calf and the foot decreases is called dorsiflexion. On the contrary, the process in which the dorsum of the foot moves away from the calf and the angle between the foot and the calf increases is called plantar flexion. When plantar flexing, the foot can make a certain range of lateral movement. The sole of the foot turns inward for inversion and the sole of the foot turns outward for eversion.

[0003] At present, the methods for measuring ankle flexibility in China mainly include the following: 1. Angle meter measurement method: such as the protractor measurement method, using a protractor to directly measure the maximum angle of ankle dorsiflexion, plantar flexion, inversion, eversion and other movements to evaluate the flexibility of the ankle. This method is simple and easy, but the measurement accuracy is greatly affected by the operator's experience, and the consistency of repeated measurements is poor. 2. Visual method: observe the ankle movement with the naked eye to evaluate the flexibility of the ankle. This method is highly subjective, and the measurement results are greatly affected by the observer's experience and technical level. 3. Electronic instrument measurement method: use electronic instruments (such as ankle flexibility measuring instruments) for measurement, which has high accuracy, strong objectivity, and good repeatability, but the price is high, and the operation and maintenance costs are also high. 4. Imaging examination method: observe the ankle joint gap, articular surface and surrounding soft tissue through imaging examination methods such as X-ray, CT or MRI, so as to indirectly evaluate the range of motion of the ankle joint. It can provide detailed structural information inside the joint and provide a basis for evaluation. However, the cost is high and professional image interpretation ability is required. In addition, imaging examinations can usually only reflect the joint condition in a static state and cannot directly measure dynamic range of motion. 5. Three-dimensional camera measurement system: Use three-dimensional camera technology to capture the dynamic changes of the ankle joint during movement, and obtain data on joint range of motion through computer analysis. The measurement accuracy is high and can intuitively display the movement trajectory and range of the joint. However, the equipment is expensive and requires professional personnel to operate and analyze. Summary of the invention

[0004] The purpose of the present invention is to provide a method and device for automatic detection of ankle flexibility based on computer vision to solve the above-mentioned technical problems.

[0005] In order to solve the above technical problems, the specific technical solutions of the method and device for automatic ankle flexibility detection based on computer vision of the present invention are as follows:

[0006] A method for automatically detecting ankle flexibility based on computer vision comprises the following steps:

[0007] Step 1: photograph the subject's ankle dorsiflexion and plantar flexion movements, extract the position images of the first metatarsal and ankle, and perform image preprocessing;

[0008] Step 2: Extract the dorsiflexion and plantar flexion angles of the subject's ankle joint;

[0009] Step 3: Determine the ankle flexibility of the subject by calculating the maximum dorsiflexion angle and plantar flexion angle of the subject;

[0010] Step 4: photograph the inversion and eversion movements of the subject's ankle joint, extract the position image of the sole plane and perform image preprocessing;

[0011] Step 5: Extract the inversion and eversion angles of the subject's ankle joint;

[0012] Step 6: Determine the ankle flexibility of the subject by calculating the maximum eversion angle and maximum inversion angle of the ankle joint of the subject.

[0013] Furthermore, the step 1 includes the following specific steps:

[0014] Step 1.1: Generate a sample image of the ankle side detection;

[0015] The subject rotates the ankle joint to perform ankle dorsiflexion and plantar flexion movements, and the side of the subject's ankle is photographed to obtain the subject's ankle dorsiflexion and plantar flexion movement video. The frame rate F is selected, and the image stream in the video is extracted with 1 / F as the time interval. n The image is A( n ) means that the sequence A(1), A(2), …, A( n ) forming a lateral ankle detection sample for subsequent extraction and analysis of the first metatarsal and ankle joint positions;

[0016] Step 1.2: Preprocess the ankle side detection sample image to obtain a preprocessed black and white image.

[0017] Furthermore, the step 2 includes the following specific steps:

[0018] Step 2.1: Extract the position coordinates of the subject's first metatarsal and ankle joint;

[0019] Starting from the first row of the black-and-white image preprocessed in step 1, the number of pixels with a pixel value of 0 in each row is calculated. When the number of pixels reaches the set first metatarsal pixel threshold, the row number at this time is determined as the ordinate of the first metatarsal y M , then take the average value of the horizontal coordinates of the pixels with a pixel value of 0 under the vertical coordinate of the first metatarsal bone. The average value at this time is the horizontal coordinate of the first metatarsal bone x M After confirming the coordinates of the first metatarsal, continue to read and calculate downward in rows. When the number of pixels with a pixel value of 0 reaches the set ankle joint pixel threshold, the row number at this time is determined to be the ordinate of the ankle joint. y A , then take the average value of the horizontal coordinates of the pixels with a pixel value of 0 under the vertical coordinate of the ankle joint. The average value at this time is the horizontal coordinate of the ankle joint x A ;

[0020] Step 2.2: Extract the angle between the line connecting the first metatarsal bone and the ankle joint relative to the vertical direction, and determine whether it is a dorsiflexion angle or a plantar flexion angle;

[0021] The coordinates of the first metatarsal and ankle joint are atan Transformation, we get the angle between the two lines relative to the vertical direction: θ = atan ( ), , according to the ankle placement direction of the subject's sole on the left and the dorsum on the right in the image, when θ>0, it means that the ankle joint is doing plantar flexion movement, and the plantar flexion angle is output When θ<0, it means the ankle joint is dorsiflexing, and the output is the dorsiflexion angle , when θ=0, it means that the first metatarsal on the image is vertically above the ankle joint.

[0022] Furthermore, the step 3 includes the following specific steps:

[0023] After calculating the dorsiflexion and plantar flexion angles of the subjects' ankle joints, the dorsiflexion angles of all ankle joints were α ( n ) and plantar flexion angle β ( n ) for comparison, where n Indicates n Detection sample images; Get the maximum dorsiflexion angle of the ankle joint of the subject α max and maximum plantar flexion angle β max , if 15°≤ α max ≤20°, the subject's ankle dorsiflexion flexibility is normal. αmax <15°, the subject's ankle dorsiflexion flexibility is insufficient. α max >20°, the subject's ankle dorsiflexion flexibility is considered to be abnormal; if 40°≤ β max ≤50°, the ankle plantar flexion flexibility of the subject is normal. β max <40°, the subject's ankle plantar flexion flexibility is insufficient. β max >50°, indicating that the subject's ankle plantar flexion flexibility is extraordinary.

[0024] Furthermore, the step 4 includes the following specific steps:

[0025] Step 4.1: Generate a sample image of the top surface of the toe;

[0026] The subject performs ankle inversion and eversion movements, shoots the top surface of the subject's toes, obtains the subject's ankle dorsiflexion and plantar flexion movement video, selects the frame rate F, extracts the image stream in the video with 1 / F as the time interval, and n The image is A( n ) means that the sequence A(1), A(2), …, A( n ) forms a toe top surface detection sample for subsequent extraction and analysis of the sole plane position;

[0027] Step 4.2: Preprocess the sample image of the toe top surface detection to obtain a preprocessed black and white image.

[0028] Furthermore, the step 5 includes the following specific steps:

[0029] Step 5.1: Extract the position coordinates of both sides of the subject's foot;

[0030] Starting from the first row of the black-and-white image preprocessed in step 4, the number of pixels with a pixel value of 0 in each column is calculated. When the number of pixels reaches the set toe pixel threshold, the column number at this time is determined as the horizontal coordinate of the toe. x T , then take the average of the vertical coordinates of the pixels with a pixel value of 0 under the horizontal coordinate of the toe. The average value at this time is the vertical coordinate of the toe y T ;

[0031] The horizontal coordinate at the toe is ( x T +5) column, from y T Start retrieving pixel values ​​upwards, when B( x T +5,y T - i )=0, and B( x T +5, y T -( i+ 1)) = 255, the coordinates are obtained ( x T +5, y T - i ),in i =1,2,…; at the same time from y T Start to retrieve pixel values ​​downwards, when B( x T +5, y T + k )=0, and B( x T +5, y T +( k +1)) = 255, extract the coordinates ( x T +5, y T + k ),in k =1,2,…; repeat in ( x T +5 n ) columns, where n =1,2,…, get the coordinates of the edge of the sole, and compare each coordinate with the previous coordinate to get the change of the ordinate of the edge of the sole, until the change of the ordinate exceeds the threshold we set, then stop the above retrieval and coordinate extraction, and discard the coordinate;

[0032] Extract the maximum horizontal coordinate for upward retrieval and its corresponding ordinate , and the maximum horizontal coordinate for downward retrieval and its corresponding ordinate ,calculate( x T , y T )、 The distance between two coordinate points ,calculate( x T , y T )、 The distance between two coordinate points ,Will M1 and M 2 is compared with the foot width threshold range we set. The position of the foot that falls within the threshold range is the foot position of the subject. Finally, the position coordinates of both sides of the foot are determined and recorded as ( x T , y T )、 ;

[0033] Step 5.2: Extract the angle between the sole of the foot and the vertical direction, and determine whether it is an inversion angle or a valgus angle;

[0034] The coordinates of the two sides of the sole are atan Transformation, we get the angle between the two lines relative to the vertical direction: θ = atan ( ), , according to the placement direction of the subject's ankle with the sole on the left and the dorsum on the right in the image, when θ>0, if it is the left foot, it means that the ankle joint is everting, and if it is the right foot, it means that the ankle joint is inverting; when θ<0, if it is the left foot, it means that the ankle joint is inverting, and if it is the right foot, it means that the ankle joint is everting;

[0035] Define a parameter D. When the measured foot is the left foot, D=1, and when the measured foot is the right foot, D=-1. When the ankle joint performs eversion movement, the output eversion angle is , When , it means the ankle joint is inverted, and the output is the inversion angle .

[0036] Furthermore, the step 6 includes the following specific steps:

[0037] After calculating the inversion and eversion angles of the ankle joint, the eversion angles of all ankle joints are calculated. and varus angle For comparison, n Indicates n Detection sample images; Get the maximum eversion angle of the ankle joint of the subject and maximum varus angle ,like , the test subject's ankle eversion flexibility is normal. , it is concluded that the subject's ankle eversion flexibility is insufficient. , it is concluded that the subject's ankle eversion flexibility is extraordinary; if , the ankle inversion flexibility of the tested person is normal. , it is concluded that the subject's ankle inversion flexibility is insufficient. , and it was concluded that the subjects had extraordinary ankle inversion flexibility.

[0038] Furthermore, the step 1.2 includes the following specific steps:

[0039] Step 1.2.1: Detect sample A on the side of the ankle ( n ) performs grayscale processing to generate a grayscale image of the ankle side detection sample;

[0040] Extract the detection sample image A( n ) are weighted averaged according to different weights: , ( x , y ) represents the coordinates of any pixel point on the original image, R ( x , y )、 G ( x , y )、 B ( x , y ) represent the red, green and blue component values ​​of the pixel respectively, and finally generate the corresponding grayscale image;

[0041] Step 1.2.2: Perform two-dimensional median filtering on the grayscale image of the ankle side detection sample;

[0042] set up b * b The square template roams in the grayscale image of the detection sample, and the center of the template coincides with each pixel position in the image in turn; reads the grayscale value of each corresponding pixel under the template; sorts these grayscale values ​​from small to large; extracts the middle value of the grayscale value; assigns this middle value to the pixel at the center position of the corresponding template as the output result; the output of the two-dimensional median filter is written as , N ( x , y ) represents the neighborhood of the current pixel, that is, b * b Template coverage area, , Gray ( s , t ) represents the original grayscale value of each pixel in the neighborhood, and its output value is the median of the grayscale values ​​of each point in the current neighborhood, and is assigned to the current pixel, recorded as g median ( x , y ), for a template size of b * b The median filter has an output value of b * bThe median of gray values, the mutation area in the image whose size is less than half of the template size is removed after filtering to deal with random and isolated noise points in the sample image under test;

[0043] Step 1.2.3: Perform grayscale transformation on the ankle side detection sample image after two-dimensional median filtering;

[0044] Extract the pixel grayscale value in the detection sample image after two-dimensional median filtering g , through the transformation function T, the gray value g Map to grayscale values r , , c and a is the corresponding coefficient under the image condition, so that the original image is enhanced;

[0045] Step 1.2.4: Binarize the ankle side detection sample image after grayscale transformation;

[0046] The histogram of the detected sample image after grayscale transformation is defined as h ( r k ) =n k ; r k It is k The gray value of the level, n k The gray level in the image is r k The number of pixels, k =0, 1, 2, ..., L- 1. Here L is the grayscale level; by enumerating and finding a suitable threshold, the sample image after grayscale transformation is binarized; when the grayscale value is greater than the threshold, B ( x,y )=255, when the gray value is less than the threshold, let B ( x,y )=0, where B ( x,y ) represents the pixel value of a pixel in the image. After binarization, the image is converted into a black and white image, which further highlights the characteristic information of the foot.

[0047] Furthermore, the pretreatment step of step 4.2 is the same as step 1.2.

[0048] The present invention also discloses an automatic ankle flexibility detection device based on computer vision, which is used to implement the steps of the method. The device includes a monitoring device, a single-chip microcomputer and a computer, which are interconnected. The monitoring device has a camera inside, which is used to capture the position information of the object to be measured and send it to the single-chip microcomputer. The single-chip microcomputer includes an audio-visual prompt module, which is used to issue an audio-visual prompt after receiving the position information of the subject collected by the monitoring device, prompting the subject's ankle to reach the designated camera video capture position. The computer is used to collect images captured by the monitoring device and perform image processing.

[0049] The computer vision-based automatic ankle flexibility detection method and device of the present invention have the following advantages: the computer vision-based automatic ankle flexibility detection method and device of the present invention are oriented to two tests of the maximum angles of ankle dorsiflexion and plantar flexion and the maximum angles of inversion and eversion. By quickly extracting the positions of the first metatarsal, ankle and sole plane, the changes in angles and positions can be detected in real time, which can reduce the workload of evaluators, reduce equipment costs, improve evaluation efficiency, and improve the stability and accuracy of test results. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Picture 1 It is a schematic diagram of the automatic detection device of ankle flexibility based on computer vision of the present invention.

[0051] Picture 2 It is a schematic diagram of the structure of the monitoring device of the present invention.

[0052] Picture 3 It is a flow chart of the automatic ankle flexibility detection method of the present invention.

[0053] Picture 4 It is a schematic diagram of image coordinate axis segmentation of the present invention.

[0054] Explanation of the markings in the figure: 1. Monitoring device; 11. Bottom background plate; 12. Side background plate; 13. Moving track; 14. Camera; 15. Shooting point; 2. Single chip microcomputer; 3. Computer. DETAILED DESCRIPTION

[0055] In order to better understand the purpose, structure and function of the present invention, the following is a further detailed description of a method and device for automatic ankle flexibility detection based on computer vision of the present invention in conjunction with the accompanying drawings.

[0056] like Picture 1 As shown, the automatic ankle flexibility detection device based on computer vision of the present invention comprises a monitoring device 1, a single chip microcomputer 2 and a computer 3, which are interconnected. The monitoring device 1 is used to collect video information of the object to be tested and send it to the single chip microcomputer 2. Picture 2As shown, the monitoring device 1 includes a bottom background plate 11, a side background plate 12, a moving track 13 and a camera 14. The bottom background plate 11 and the side background plate 12 are black. The bottom background plate 11 is arranged below the shooting point 15, and the side background plate 12 is arranged on one side of the shooting point. A plurality of moving tracks 13 are respectively arranged on the other side, the front and the top of the shooting point 15. A plurality of cameras 14 are respectively arranged on each moving track 13 for flexibly collecting the video of the object under test. The single-chip computer 2 includes an audio-visual prompt module for issuing an audio-visual prompt after receiving the position information of the subject collected by the monitoring device 1, prompting the subject's ankle to reach the designated camera video capture position. The computer 3 is used to collect the image captured by the monitoring device 1 and perform image processing.

[0057] like Picture 3 As shown, a method for automatically detecting ankle flexibility based on computer vision of the present invention comprises the following steps:

[0058] Step 1: photograph the subject's ankle dorsiflexion and plantar flexion movements, extract the position images of the first metatarsal and ankle, and perform image preprocessing;

[0059] Step 1.1: Generate a sample image of the ankle side detection;

[0060] During the measurement, the subject sits with legs straight. The monitoring device 1 receives the subject's ankle position information fed back by the camera 14, and controls the single-chip microcomputer 2 to emit sound and light prompts, so that the subject's ankle reaches the designated camera 14 video capture position. The subject rotates the ankle joint to perform ankle dorsiflexion and plantar flexion. The monitoring device 1 shoots the side of the subject's ankle to obtain the subject's ankle dorsiflexion and plantar flexion movement video. The frame rate F is selected, and the image stream in the video is extracted at a time interval of 1 / F. n The image is A( n ) means that the sequence A(1), A(2), …, A( n ) forming a lateral ankle detection sample for subsequent extraction and analysis of the first metatarsal and ankle joint positions;

[0061] Step 1.2: Preprocess the ankle side detection sample image to obtain a preprocessed black and white image.

[0062] Step 1.2.1: Detect sample A on the side of the ankle ( n ) performs grayscale processing to generate a grayscale image of the ankle side detection sample;

[0063] like Picture 4 As shown, the detection sample image A( n ) are weighted averaged according to different weights: , ( x , y) represents the coordinates of any pixel point on the original image, R ( x , y )、 G ( x , y )、 B ( x , y ) represent the red, green and blue component values ​​of the pixel respectively, and finally generate the corresponding grayscale image;

[0064] Step 1.2.2: Perform two-dimensional median filtering on the grayscale image of the ankle side detection sample;

[0065] set up b * b The square template roams in the grayscale image of the detection sample, and the center of the template coincides with each pixel position in the image in turn. b Generally, an odd number such as 3, 5, 7, etc. is used; the grayscale values ​​of the corresponding pixels under the template are read; these grayscale values ​​are sorted from small to large; the middle value of the grayscale value is extracted; this middle value is assigned to the pixel at the center of the corresponding template as the output result; the output of the two-dimensional median filter is written as , N ( x , y ) represents the neighborhood of the current pixel, that is, b * b Template coverage area, , Gray ( s , t ) represents the original grayscale value of each pixel in the neighborhood, and its output value is the median of the grayscale values ​​of each point in the current neighborhood, and is assigned to the current pixel, recorded as g median ( x , y ). For a template size of b * b The median filter has an output value of b * b The median of gray values ​​is taken as the image size. The mutation area whose size is less than half of the template size is removed after filtering to process the random and isolated noise points in the sample image under test.

[0066] Step 1.2.3: Perform grayscale transformation on the ankle side detection sample image after two-dimensional median filtering;

[0067] Extract the pixel grayscale value in the detection sample image after two-dimensional median filtering g , through the transformation function T, the gray value g Map to grayscale values r, , c and a is the corresponding coefficient under the image condition, which enhances the original image.

[0068] Step 1.2.4: Binarize the ankle side detection sample image after grayscale transformation;

[0069] The histogram of the detected sample image after grayscale transformation is defined as h ( r k ) =n k ; r k It is k The gray value of the level, n k The gray level in the image is r k The number of pixels, k =0, 1, 2, ..., L- 1. Here L is the grayscale level; by enumerating and finding a suitable threshold, the sample image after grayscale transformation is binarized; when the grayscale value is greater than the threshold, B ( x,y )=255, when the gray value is less than the threshold, let B ( x,y )=0, where B ( x,y ) represents the pixel value of a pixel in the image. After binarization, the image is converted into a black and white image, which further highlights the characteristic information of the foot.

[0070] Step 2: Extract the dorsiflexion and plantar flexion angles of the subject's ankle joint;

[0071] Step 2.1: Extract the position coordinates of the subject's first metatarsal and ankle joint;

[0072] Starting from the first row of the black-and-white image preprocessed in step 1, the number of pixels with a pixel value of 0 in each row is calculated. When the number of pixels reaches the set first metatarsal pixel threshold, the row number at this time is determined as the ordinate of the first metatarsal y M , then take the average value of the horizontal coordinates of the pixels with a pixel value of 0 under the vertical coordinate of the first metatarsal bone. The average value at this time is the horizontal coordinate of the first metatarsal bone x M After confirming the coordinates of the first metatarsal, continue to read and calculate downward in rows. When the number of pixels with a pixel value of 0 reaches the set ankle joint pixel threshold, the row number at this time is determined to be the ordinate of the ankle joint. y M, then take the average value of the horizontal coordinates of the pixels with a pixel value of 0 under the vertical coordinate of the ankle joint. The average value at this time is the horizontal coordinate of the ankle joint x A .

[0073] Step 2.2: Extract the angle between the line connecting the first metatarsal bone and the ankle joint relative to the vertical direction, and determine whether it is a dorsiflexion angle or a plantar flexion angle;

[0074] The coordinates of the first metatarsal and ankle joint are atan Transformation, we get the angle between the two lines relative to the vertical direction: θ = atan ( ), , according to the ankle placement direction of the subject's sole on the left and the dorsum on the right in the image, when θ>0, it means that the ankle joint is doing plantar flexion movement, and the plantar flexion angle is output When θ<0, it means the ankle joint is dorsiflexing, and the output is the dorsiflexion angle , when θ=0, it means that the first metatarsal on the image is vertically above the ankle joint.

[0075] Step 3: Determine the ankle flexibility of the subject by calculating the maximum dorsiflexion angle and plantar flexion angle of the subject;

[0076] After calculating the dorsiflexion and plantar flexion angles of the subjects' ankle joints, the dorsiflexion angles of all ankle joints were α ( n ) and plantar flexion angle β ( n ) for comparison, where n Indicates n Detection sample images; Get the maximum dorsiflexion angle of the ankle joint of the subject α max and maximum plantar flexion angle β max If 15°≤ α max ≤20°, the subject's ankle dorsiflexion flexibility is normal. α max <15°, the subject's ankle dorsiflexion flexibility is insufficient. α max >20°, the subject's ankle dorsiflexion flexibility is considered to be abnormal; if 40°≤ β max ≤50°, the ankle plantar flexion flexibility of the subject is normal. β max <40°, the subject's ankle plantar flexion flexibility is insufficient. β max >50°, indicating that the subject's ankle plantar flexion flexibility is extraordinary.

[0077] Step 4: photograph the inversion and eversion movements of the subject's ankle joint, extract the position image of the sole plane and perform image preprocessing;

[0078] Step 4.1: Generate a sample image of the top surface of the toe;

[0079] During the measurement, the subject sits with legs straight and toes pointing vertically upward. The monitoring device 1 uses the camera 14 to feed back the subject's ankle position information, and controls the single-chip microcomputer 2 to emit sound and light prompts, so that the subject's ankle reaches the designated camera 14 video capture position. The subject performs ankle inversion and eversion movements, and the camera 14 on the top surface of the monitoring device 1 captures the top surface of the subject's toes to obtain the subject's ankle dorsiflexion and plantar flexion movement video. The frame rate F is selected, and the image stream in the video is extracted at a time interval of 1 / F. n The image is A( n ) means that the sequence A(1), A(2), …, A( n ) forms a toe top surface detection sample for subsequent extraction and analysis of the sole plane position;

[0080] Step 4.2: Preprocess the sample image of the toe top surface detection to obtain a preprocessed black and white image.

[0081] Step 4.2.1: grayscale the toe top surface detection sample A(n) to generate a grayscale image of the toe top surface detection sample;

[0082] Extract the values ​​of the three components of R, G, and B of the pixel in the detection sample image A(n), and perform weighted averaging according to different weights: , ( x , y ) represents the coordinates of any pixel point on the original image, R ( x , y )、 G ( x , y )、 B ( x , y ) represent the red, green and blue component values ​​of the pixel respectively, and finally generate the corresponding grayscale image;

[0083] Step 4.2.2: Perform two-dimensional median filtering on the grayscale image of the toe top surface detection sample;

[0084] set up b*b The square template roams in the grayscale image of the detection sample, and the center of the template coincides with each pixel position in the image in turn. bGenerally, an odd number such as 3, 5, 7, etc. is used; the grayscale values ​​of the corresponding pixels under the template are read; these grayscale values ​​are sorted from small to large; the middle value among these values ​​is extracted; this middle value is assigned to the pixel at the center position of the corresponding template as the output result; the output of the two-dimensional median filter is written as , N ( x , y ) represents the neighborhood of the current pixel, that is, b * b Template coverage area, , Gray ( s , t ) represents the original grayscale value of each pixel in the neighborhood, and its output value is the median of the grayscale values ​​of each point in the current neighborhood, and is assigned to the current pixel, recorded as g median ( x , y ). For a template size of b * b The median filter has an output value of b * b The median of gray values ​​is taken as the image size. The mutation area whose size is less than half of the template size is removed after filtering to process the random and isolated noise points in the sample image under test.

[0085] Step 4.2.3: Perform grayscale transformation on the toe top surface detection sample image after two-dimensional median filtering;

[0086] Extract the pixel grayscale value in the detection sample image after two-dimensional median filtering g , through the transformation function T, the gray value g Map to grayscale values r , , c and a is the corresponding coefficient under the image condition, which enhances the original image.

[0087] Step 4.2.4: Binarize the toe top surface detection sample image after grayscale transformation;

[0088] The histogram of the detected sample image after grayscale transformation is defined as h ( r k ) =n k ; r k It is k The gray value of the level, n k The gray level in the image is r kThe number of pixels, k =0, 1, 2, ..., L- 1. Here L is the grayscale level; by enumerating and finding a suitable threshold, the sample image after grayscale transformation is binarized; when the grayscale value is greater than the threshold, B ( x,y )=255, when the gray value is less than the threshold, let B ( x,y )=0, where B ( x,y ) represents the pixel value of a pixel in the image. After binarization, the image is converted into a black and white image, which further highlights the characteristic information of the foot.

[0089] Step 5: Extract the inversion and eversion angles of the subject's ankle joint;

[0090] Step 5.1: Extract the position coordinates of both sides of the subject's foot;

[0091] Starting from the first row of the black-and-white image preprocessed in step 4, the number of pixels with a pixel value of 0 in each column is calculated. When the number of pixels reaches the set toe pixel threshold, the column number at this time is determined as the horizontal coordinate of the toe. x T , then take the average of the vertical coordinates of the pixels with a pixel value of 0 under the horizontal coordinate of the toe. The average value at this time is the vertical coordinate of the toe y T .

[0092] The horizontal coordinate at the toe is ( x T +5) column, from y T Start retrieving pixel values ​​upwards, when B( x T +5, y T - i )=0, and B( x T +5, y T -( i+ 1)) = 255, the coordinates are obtained ( x T +5, y T - i ),in i =1,2,…; at the same time from y T Start to retrieve pixel values ​​downwards, when B( x T +5, y T+ k )=0, and B( x T +5, y T +( k +1)) = 255, extract the coordinates ( x T +5, y T + k ),in k =1,2,…; repeat in ( x T +5 n ) columns, where n =1,2,…, get the coordinates of the edge of the sole, and compare each coordinate with the previous coordinate to get the change of the longitudinal coordinate of the edge of the sole, until the change of the longitudinal coordinate exceeds the threshold we set, then stop the above retrieval and coordinate extraction, and discard the coordinate.

[0093] Extract the maximum horizontal coordinate for upward retrieval and its corresponding ordinate , and the maximum horizontal coordinate for downward retrieval and its corresponding ordinate ,calculate( x T , y T )、 The distance between two coordinate points ,calculate( x T , y T )、 The distance between two coordinate points ,Will M 1 and M 2 is compared with the foot width threshold range we set. The position of the foot that falls within the threshold range is the foot position of the subject. Finally, the position coordinates of both sides of the foot are determined and recorded as ( x T , y T )、 .

[0094] Step 5.2: Extract the angle between the sole of the foot and the vertical direction, and determine whether it is an inversion angle or a valgus angle;

[0095] The coordinates of the two sides of the sole are atan Transformation, we get the angle between the two lines relative to the vertical direction: θ = atan ( ), According to the placement direction of the subject's ankle in the image, with the sole on the left and the dorsum on the right, when θ>0, if it is the left foot, it means that the ankle joint is doing eversion movement, and if it is the right foot, it means that the ankle joint is doing inversion movement; when θ<0, if it is the left foot, it means that the ankle joint is doing inversion movement, and if it is the right foot, it means that the ankle joint is doing eversion movement.

[0096] Define a parameter D. When the measured foot is the left foot, D=1, and when the measured foot is the right foot, D=-1. When the ankle joint performs eversion movement, the output eversion angle is , When , it means the ankle joint is inverted, and the output is the inversion angle .

[0097] Step 6: Determine the ankle flexibility of the subject by calculating the maximum eversion angle and maximum inversion angle of the ankle joint of the subject;

[0098] After calculating the inversion and eversion angles of the ankle joint, the eversion angles of all ankle joints are calculated. and varus angle For comparison, n Indicates n Detection sample images; Get the maximum eversion angle of the ankle joint of the subject and maximum varus angle ,like , the test subject's ankle eversion flexibility is normal. , it is concluded that the subject's ankle eversion flexibility is insufficient. , it is concluded that the subject's ankle eversion flexibility is extraordinary; if , the ankle inversion flexibility of the tested person is normal. , it is concluded that the subject's ankle inversion flexibility is insufficient. , and it was concluded that the subjects had extraordinary ankle inversion flexibility.

[0099] It is to be understood that the present invention is described by some embodiments, and it is known to those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the scope of protection of the present invention.

Claims

1. A method for automatically detecting ankle flexibility based on computer vision, comprising an automatic detection device for ankle flexibility based on computer vision, the device comprising a monitoring device (1), a single chip microcomputer (2) and a computer (3), the three being connected to each other, the monitoring device (1) being used to collect video information of a detected object and send it to the single chip microcomputer (2), the monitoring device (1) comprising a bottom background plate (11), a side background plate (12), a moving track (13) and a camera (14), the bottom background plate (11) and the side background plate (12) being black, the bottom background plate (11) being arranged at a shooting point (14) 5) below, the side background plate (12) is arranged on one side of the shooting point, a plurality of movable tracks (13) are respectively arranged on the other side, front and top surface of the shooting point (15), a plurality of cameras (14) are respectively arranged on each movable track (13) for flexibly collecting the video of the object to be measured, the single chip computer (2) includes an audio-visual prompt module for issuing an audio-visual prompt after receiving the position information of the measured person collected by the monitoring device (1), prompting the measured person's ankle to reach the designated camera video capture position, and the computer (3) is used to collect the image captured by the monitoring device (1) and perform image processing; it is characterized in that The method comprises the following steps: Step 1: photograph the subject's ankle dorsiflexion and plantar flexion movements, extract the position images of the first metatarsal and ankle, and perform image preprocessing; Step 2: Extract the dorsiflexion and plantar flexion angles of the subject's ankle joint; Step 2.1: Extract the position coordinates of the subject's first metatarsal and ankle joint; Starting from the first row of the black-and-white image preprocessed in step 1, the number of pixels with a pixel value of 0 in each row is calculated. When the number of pixels reaches the set first metatarsal pixel threshold, the row number at this time is determined as the ordinate y of the first metatarsal M , then take the average value of the horizontal coordinates of the pixels with a pixel value of 0 under the vertical coordinate of the first metatarsal bone. The average value at this time is the horizontal coordinate x of the first metatarsal bone. M After confirming the coordinates of the first metatarsal bone, continue to read and calculate downward in rows. When the number of pixels with a pixel value of 0 reaches the set ankle joint pixel threshold, the row number at this time is determined to be the ordinate y of the ankle joint. A , then take the average value of the horizontal coordinates of the pixels with a pixel value of 0 under the vertical coordinate of the ankle joint. The average value at this time is the horizontal coordinate x of the ankle joint. A ; Step 2.2: Extract the angle between the line connecting the first metatarsal bone and the ankle joint relative to the vertical direction, and determine whether it is a dorsiflexion angle or a plantar flexion angle; Perform atan transformation on the coordinates of the first metatarsal and ankle joint to obtain the angle between the line connecting the two and the vertical direction: According to the ankle placement direction of the subject in the image, with the sole on the left and the dorsum on the right, when θ>0, it means that the ankle joint is doing plantar flexion movement, and the plantar flexion angle is output When θ<0, it means the ankle joint is dorsiflexing and the dorsiflexion angle is output. When θ = 0, it means that the first metatarsal bone on the image is vertically above the ankle joint; Step 3: Determine the ankle flexibility of the subject by calculating the maximum dorsiflexion angle and plantar flexion angle of the subject; Step 4: photograph the inversion and eversion movements of the subject's ankle joint, extract the position image of the sole plane and perform image preprocessing; Step 5: Extract the inversion and eversion angles of the subject's ankle joint; Step 5.1: Extract the position coordinates of both sides of the subject's foot; Starting from the first row of the black-and-white image preprocessed in step 4, the number of pixels with a pixel value of 0 in each column is calculated. When the number of pixels reaches the set toe pixel threshold, the column number at this time is determined as the horizontal coordinate x of the toe. T , then take the average of the vertical coordinates of the pixel points with a pixel value of 0 under the horizontal coordinate of the toe. The average value at this time is the vertical coordinate y of the toe T ; The horizontal coordinate at the toe is (x T +5) column, from y T Start to retrieve pixel values ​​upwards, when B(x T +5,y T -i)=0, and B(x T +5,y T -(i+1))=255, we get the coordinate (x T +5,y T -i), where i = 1, 2, ...; at the same time from y T Start to retrieve pixel values ​​downwards, when B(x T +5,y T + k) = 0, and B(x T +5,y T +(k+1))=255, extract the coordinate (x T +5,y T +k), where k = 1, 2, ...; repeat in (x T +5n), where n = 1, 2, ..., to obtain the coordinates of the edge of the sole, and compare each coordinate with the previous coordinate to obtain the change in the ordinate of the edge of the sole, until the change in the ordinate exceeds the threshold we set, then stop the above search and coordinate extraction, and discard the coordinate; Extract the maximum horizontal coordinate for upward retrieval and its corresponding ordinate and the maximum horizontal coordinate for downward retrieval and its corresponding ordinate Calculate (x T ,y T ), The distance between two coordinate points Calculate (x T ,y T ), The distance between two coordinate points Compare the values ​​of M1 and M2 with the foot width threshold range we set. The position of the foot that falls within the threshold range is the position of the subject's foot. Finally, determine the position coordinates of both sides of the foot, recorded as (x T ,y T )、(x′ T , y′ T ); Step 5.2: Extract the angle between the sole of the foot and the vertical direction, and determine whether it is an inversion angle or a valgus angle; Perform atan transformation on the position coordinates of both sides of the sole of the foot to obtain the angle between the two lines relative to the vertical direction: According to the placement direction of the subject's ankle in the image, with the sole on the left and the dorsum on the right, when θ>0, if it is the left foot, it means that the ankle joint is inverted, and if it is the right foot, it means that the ankle joint is inverted; when θ<0, if it is the left foot, it means that the ankle joint is inverted, and if it is the right foot, it means that the ankle joint is inverted; Define a parameter D. When the measured foot is the left foot, D = 1, and when the measured foot is the right foot, D = -1. Then when D·θ>0, it means that the ankle joint is everting, and the output is the eversion angle. When D·θ<0, it means the ankle joint is inverted and the inversion angle is output. Step 6: Determine the ankle flexibility of the subject by calculating the maximum eversion angle and maximum inversion angle of the ankle joint of the subject.

2. The method for automatic ankle flexibility detection based on computer vision according to claim 1, characterized in that: The step 1 comprises the following specific steps: Step 1.1: Generate a sample image of the ankle side detection; The subject rotates the ankle joint to perform ankle dorsiflexion and plantar flexion movements, and the side of the subject's ankle is photographed to obtain the subject's ankle dorsiflexion and plantar flexion movement video. The frame rate F is selected, and the image stream in the video is extracted with 1 / F as the time interval. The mth image is represented by A1(m), and the sequence A1(1), A1(2), ..., A1(m) forms an ankle side detection sample for subsequent extraction and analysis of the first metatarsal and ankle joint positions. Step 1.2: Preprocess the ankle side detection sample image to obtain a preprocessed black and white image.

3. The method for automatic ankle flexibility detection based on computer vision according to claim 1, characterized in that: The step 3 comprises the following specific steps: After calculating the dorsiflexion and plantar flexion angles of the subject's ankle joint, the dorsiflexion angles α(m) and plantar flexion angles β(m) of all ankle joints are compared, where m represents the mth detection sample image; the maximum dorsiflexion angle α of the subject's ankle joint is obtained. max and maximum plantar flexion angle β max , if 15°≤α max ≤20°, the subject's ankle dorsiflexion flexibility is normal. max <15°, the subject’s ankle dorsiflexion flexibility is insufficient. max >20°, the subject's ankle dorsiflexion flexibility is found to be abnormal; If 40°≤β max ≤50°, the ankle plantar flexion flexibility of the subject is normal. max <40°, the subject's ankle plantar flexion flexibility is insufficient. max >50°, indicating that the subject's ankle plantar flexion flexibility is extraordinary.

4. The method for automatic ankle flexibility detection based on computer vision according to claim 2, characterized in that: The step 4 comprises the following specific steps: Step 4.1: Generate a sample image of the top surface of the toe; The subject performs ankle inversion and eversion movements, shoots the top surface of the subject's toes, obtains the dorsiflexion and plantar flexion movement video of the subject's ankle joint, selects the frame rate F, extracts the image stream in the video with 1 / F as the time interval, the mth image is represented by A2(m), the sequence A2(1), A2(2), ..., A2(m) forms the toe top surface detection sample, which is used for the subsequent extraction and analysis of the sole plane position; Step 4.2: Preprocess the sample image of the toe top surface detection to obtain a preprocessed black and white image.

5. The method for automatic ankle flexibility detection based on computer vision according to claim 1, characterized in that: The step 6 comprises the following specific steps: After calculating the inversion and eversion angles of the ankle joint, the eversion angle φ (m) and inversion angle φ (m) of all ankle joints are calculated. Compare, where m represents the mth detection sample image; get the maximum eversion angle φ of the ankle joint of the testee max and maximum varus angle If 20°≤φ max ≤40°, the subject's ankle eversion flexibility is normal. max ≤20°, the subject's ankle eversion flexibility is insufficient. max ≥40°, the subject's ankle eversion flexibility is considered abnormal; if The test subject's ankle inversion flexibility is normal. The test subjects' ankle inversion flexibility is insufficient. It was concluded that the subjects had extraordinary ankle inversion flexibility.

6. The method for automatic ankle flexibility detection based on computer vision according to claim 2, characterized in that: The step 1.2 includes the following specific steps: Step 1.2.1: grayscale the ankle side detection sample A1(m) to generate a grayscale image of the ankle side detection sample; Extract the values ​​of the three components R, G, and B of the pixel in the detection sample image A1(m), and perform weighted averaging according to different weights: gray(x, y) = 0.299*R(x, y) + 0.578*G(x, y) + 0.114*B(x, y), (x, y) represents the coordinates of any pixel point on the original image, R(x, y), G(x, y), and B(x, y) represent the red, green, and blue color component values ​​of the pixel point respectively, and finally generate the corresponding grayscale image; Step 1.2.2: Perform two-dimensional median filtering on the grayscale image of the ankle side detection sample; Set a b*b square template to roam in the grayscale image of the detection sample, and overlap the center of the template with each pixel position in the image in turn; read the grayscale value of each corresponding pixel under the template; sort these grayscale values ​​from small to large; extract the middle value of the grayscale value; assign this middle value to the pixel at the center position of the corresponding template as the output result; the output of the two-dimensional median filter is written as N(x, y) represents the neighborhood of the current pixel, that is, the area covered by the b*b template, (s, t)∈N(x, y), gray(s, t) represents the original grayscale value of each pixel in the neighborhood, and its output value is the median of the grayscale values ​​of each point in the current neighborhood, which is assigned to the current pixel and recorded as g median (x, y), for a median filter with a template size of b*b, its output value is the median of b*b grayscale values. The mutation area in the image with a size less than half of the template size is removed after filtering to process the random and isolated noise points in the sample image under test; Step 1.2.3: Perform grayscale transformation on the ankle side detection sample image after two-dimensional median filtering; Extract the pixel gray value g in the detection sample image after two-dimensional median filtering, and map the gray value g to the gray value r through the transformation function T, r = T (g) = c g a , c and a are the corresponding coefficients under the image condition, so that the original image is enhanced; Step 1.2.4: Binarize the ankle side detection sample image after grayscale transformation; The histogram of the detected sample image after grayscale transformation is defined as h(r k )=n k ; r k is the gray value of the kth level, n k is the gray level r in the image k The number of pixels, k = 0, 1, 2, ..., L-1, where L is the grayscale level; find a suitable threshold by enumeration, and binarize the detection sample image after grayscale transformation; when the grayscale value is greater than the threshold, let B(x, y) = 255, when the grayscale value is less than the threshold, let B(x, y) = 0, where B(x, y) represents the pixel value of a pixel on the image. After the binarization process, the image is obtained in black and white, which further highlights the characteristic information of the foot.

7. The method for automatic ankle flexibility detection based on computer vision according to claim 4, characterized in that: The pretreatment steps of step 4.2 are the same as those of step 1.2.

Images