An action recognition method based on TensorFlow object detection

Abstract

The present invention discloses an action recognition method based on TensorFlow object detection. The main steps are as follows: 1) Obtain a video stream containing human postures and target objects, and decompose it into several frame images; 2) Construct a convolutional neural network and train it to obtain a target object recognition model; 3) Use the target object recognition model to frame the target objects in each frame image and label the rectangle frames with category labels; 4) Based on the normalized coordinates (x min_nor , y min_nor ) of the upper left corner and the normalized coordinate values (x max_nor , y max_nor ) of the lower right corner of the rectangle frame, determine the coordinates of the target object; 5) Process the coordinates of the target object; 6) Establish an action recognition model; 7) Input the processed rectangle frame coordinates into the action recognition model to recognize the human posture. The present invention realizes the establishment of the action recognition model based on the method of extracting Euclidean distance and relative position from coordinate data, and quickly and accurately completes the action recognition by setting thresholds and analyzing multiple video frame data under timing conditions.

Description

Technical Field

[0001] The present invention relates to the field of action recognition, and specifically to an action recognition method based on TensorFlow object detection. Background Art

[0002] Action scoring is the main content of the comprehension test in the intelligent scale diagnosis of Alzheimer's disease. By requiring the subject to perform corresponding actions, according to the score, the subject's comprehension, memory, and execution ability can be measured, providing an important reference for the diagnosis of the condition.

[0003] The objects involved in the existing scale actions include cards, toothbrushes, mice, etc. The action recognition method combines object detection and image processing technologies, and the recognition accuracy is not high and the time consumption is long. There are three main reasons: First, it is difficult to detect and extract the coordinates of the card. The existing methods usually use image morphology processing on the first and last two frames of the video. The front and back of the card are painted with a single color, and the largest connected domain is found by using the color space and image opening and closing operations to achieve the positioning and coordinate extraction of the card, but it is very vulnerable to the interference of light and occlusion, and a series of operations take a long time; Second, it is difficult to distinguish the front and back of the card. Because the selection of video frames has a certain randomness, the cards in the two selected video frames may be the same side, and it is impossible to judge the action of turning the card; Third, when actually extracting the coordinates of the video image, due to factors such as occlusion and overlap, some video frames have missing or abnormal coordinate values, and it is very easy to make misjudgments when using single-frame data for recognition and judgment, resulting in a decrease in accuracy. Summary of the Invention

[0004] The purpose of the present invention is to provide an action recognition method based on TensorFlow object detection, which mainly includes the following steps:

[0005] 1) Obtain a video stream containing human postures and target objects, and decompose it into several frame images. Patterns are printed on both sides of the target object. The human posture is the action completed by the subject according to the AD scale instructions. The target object is a rectangular card. Different patterns are printed on the front and back of the rectangular card.

[0006] 2) Use TensorFlow and Faster R-CNN to construct a convolutional neural network, and train to obtain a target object recognition model. The convolutional neural network includes an input layer, an activation layer, a pooling layer, a convolutional layer conv, and an output layer.

[0007] The training dataset of the target object recognition model is the COCO dataset trained by the Faster-RCNN deep learning object detection algorithm.

[0008] 3) Use the target object recognition model to frame the target objects in each frame of the image and label the rectangle with the category. Determine the normalized coordinates (x min_nor , y min_nor ) of the upper left corner of the rectangle, the normalized coordinate values (x max_nor , y max_nor ) of the lower right corner, and the confidence level c. x min_nor , y min_nor , x max_nor , y max_nor , c ∈ [0, 1].

[0009] 4) Based on the normalized coordinates (x min_nor , y min_nor ) of the upper left corner of the rectangle and the normalized coordinate values (x max_nor , y max_nor ) of the lower right corner, determine the coordinates of the target object.

[0010] The main steps to extract the rectangle coordinates are as follows:

[0011] 4.1) Convert the normalized coordinates (x min_nor , y min_nor ) of the upper left corner and the normalized coordinate values (x max_nor , y max_nor ) of the lower right corner into the position coordinates (x min , y min ) of the upper left corner and (x max , y max ) of the lower right corner of the rectangle, that is:

[0012]

[0013] In the formula, img_width and img_height represent the image width and image height respectively.

[0014] 4.2) Fill in the missing coordinate data with 0.

[0015] 4.3) Determine the coordinates of the target object card(x min , y min ) and the lower right corner coordinates card(x max , y max ), that is

[0016]

[0017] In the formula, card_p(x min , y min ) and card_n(x min , y min ) represent the upper left corner coordinates of the objects printed on the front and back of the target object respectively; card_p(xmax , y max ), and card_n(x max , y max ) represent the lower right corner coordinates of the object printed on the front and back of the target object, respectively.

[0018] 5) Process the coordinates of the target object. The main steps are as follows:

[0019] 5.1) Replace the target object coordinates with the center coordinates (x, y) of the target object, i.e.:

[0020]

[0021] In the formula, (x, y) are the center coordinates of the target object.

[0022] 5.2) Use the piecewise linear interpolation function I n (t) to linearly interpolate and complete the missing data. Among them, the piecewise linear interpolation function I n (t) is a linear function in each interval [t i , t i+1 . The piecewise linear interpolation function I n (t) is as follows:

[0023]

[0024] In the formula, f i is a linear function. t is the segmentation point; i is the segmentation serial number; n is the total number of segments;

[0025] The interpolation basis function l i (t) is as follows:

[0026]

[0027] The piecewise linear interpolation function I n (t i ) in one segmentation interval is as follows:

[0028] I n (t i ) = f i (6)

[0029] 5.3) Interpolate and complete the missing data and perform median filtering.

[0030] 6) Establish an action recognition model, i.e.:

[0031]

[0032] In the formula, dis 12 (t) is the Euclidean distance between target object 1 and target object 2; x12 (t) is the relative position of target object 1 and target object 2; subscript 1 represents target object 1, and subscript 2 represents target object 2.

[0033] 7) Input the processed rectangular box coordinates into the action recognition model to recognize the human body posture.

[0034] The standard for recognizing the human body posture is: if the threshold and timing conditions are simultaneously met in N consecutive video frames, then the subject has completed the indicated action; otherwise, the subject has not completed the indicated action. The judgment function is as follows:

[0035]

[0036] In the formula, t flag is the time flag bit. k 12 is the human body posture recognition parameter; dis thr 、k thr 、x thr 、y thr respectively represent the Euclidean distance threshold, the human body posture recognition parameter threshold, the threshold, and the vertical coordinate threshold.

[0037] It should be noted that based on the Tensorflow framework, object detection is implemented, the rectangular box coordinates of the detected object are initially extracted, and the coordinates of the detected object are obtained through preprocessing such as centering, linear interpolation, and median filtering. Then, using information such as Euclidean distance, relative position, and timing conditions, object motion analysis is completed and human actions are recognized, effectively improving the speed and accuracy of action recognition, saving labor costs, and expanding the application scenarios of action recognition. It specifically involves content such as object detection, data processing, and action recognition models.

[0038] The technical effects of the present invention are beyond doubt. The present invention combines deep learning object detection methods, data interpolation and filtering processing, and the establishment of a mathematical model for action recognition, which can improve the speed and accuracy of action recognition and expand the application scenarios of action recognition.

[0039] Based on the object detection method provided by the Tensorflow object detection interface, the present invention extracts the rectangular box coordinates of the detected object, realizes the positioning and front-back distinction of the card, with high accuracy and good robustness; based on the data processing method of centering, linear interpolation, and median filtering, the present invention replaces the object with the center coordinates to simplify the analysis process, and interpolation filtering improves the accuracy of the coordinate data, ensuring the accuracy of action recognition; based on the method of extracting Euclidean distance and relative position from the coordinate data, the present invention realizes the establishment of an action recognition model, and quickly and accurately completes action recognition by setting thresholds and timing conditions to analyze multiple video frame data. Description of the Drawings

[0040] Figure 1 It is the structural diagram of the Faster RCNN deep learning object detection algorithm;

[0041] Figure 2 It is the object detection and coordinate extraction result;

[0042] Figure 3 It is the schematic diagram of the center coordinate position;

[0043] Figure 4 It is the original 116th frame video;

[0044] Figure 5 After being processed by the TensorFlow object detection interface;

[0045] Figure 6 It is the original x - coordinate data of the toothbrush;

[0046] Figure 7 It is the interpolated x - coordinate data of the toothbrush;

[0047] Figure 8 It is the filtered x - coordinate data of the toothbrush;

[0048] Figure 9 It is the distance change between the toothbrush and the card;

[0049] Figure 10 It is the relative position change between the toothbrush and the card;

[0050] Figure 11 It is the judgment process of Action 1;

[0051] Figure 12 It is the distance between the mouse and the card;

[0052] Figure 13 It is the relative position between the mouse and the card;

[0053] Figure 14 It is the judgment process of Action 2. Specific implementation mode

[0054] The present invention will be further described below in conjunction with embodiments, but it should not be understood that the above - mentioned subject scope of the present invention is limited to the following embodiments. Without departing from the above - mentioned technical idea of the present invention, various substitutions and changes made according to the common general technical knowledge and customary means in the art should be included within the protection scope of the present invention.

[0055] Embodiment 1:

[0056] An action recognition method based on TensorFlow object detection mainly includes the following steps:

[0057] 1) Obtain a video stream containing human postures and target objects, and decompose it into several frame images. Patterns are printed on both sides of the target object. The human posture is an action completed by the subject according to the instructions of the AD scale. Object detection can identify objects regardless of their forms of existence. Therefore, patterns that can identify objects are printed on the cards, and the positioning and coordinate extraction of the cards are indirectly achieved by detecting the printed patterns. To distinguish between the front and back, different objects can be printed on the front and back.

[0058] 2) Use TensorFlow (an open-source software library that uses data flow graphs and is used for numerical calculations) and the Faster-RCNN algorithm to construct a convolutional neural network, and train to obtain a target object recognition model. The convolutional neural network includes an input layer, a pooling layer, and a convolutional layer.

[0059] The TensorFlow object detection interface provides pre-trained standard dataset files, including the COCO dataset, the OpenImages dataset, the iNaturalist dataset, etc. The recognizable categories are rich. Each standard dataset also provides dataset files trained by different deep learning object detection algorithms, with different speeds and precisions, which can be selected according to one's own needs. Through testing multiple standard datasets, the COCO dataset trained under the Faster-RCNN deep learning object detection algorithm is finally selected. This dataset can recognize 90 categories, including common objects in life such as cars, apples, mice, etc. It has high precision, good stability, and its speed can also meet the requirements.

[0060] The convolutional neural network is the VGG16 model, which includes 13 conv (convolution) layers + 13 relu (activation) layers + 4 pooling layers.

[0061] The training dataset of the target object recognition model is the COCO dataset trained by the Faster-RCNN deep learning object detection algorithm.

[0062] 3) Use the target object recognition model to frame the target objects in each frame image and label the rectangular boxes with category labels. Determine the normalized coordinates (x min_nor , y min_nor ) of the upper left corner, the normalized coordinate values (x max_nor , y max_nor ) of the lower right corner, and the confidence level c. x min_nor , y min_nor , x max_nor , y max_nor , c ∈ [0, 1].

[0063] The main steps to use the Faster R-CNN algorithm to frame the target objects in each frame of the image are as follows:

[0064] 3.1) Input the picture into the convolutional network to generate the feature image FeatureMap.

[0065] 3.2) Input FeatureMap into the RPN network. First, perform a 3×3 convolutional operation, and then divide it into two convolutional branches to respectively judge the category (detect the target or background area) and predict the position of the rectangular box. Finally, multiple recommended candidate regions Proposal are extracted.

[0066] 3.3) Use ROIPooling to extract a unified fixed-size feature map for each Proposal. ROIPooling is a type of pooling. ROI (Region of interest) refers to the rectangular box.

[0067] 3.4) Input the unified-size feature map into the fully connected and softmax networks for classification and regression, and output the category to which the candidate region belongs and its exact position in the image. The algorithm structure is as Figure 1 shown.

[0068] 4) Based on the normalized coordinates (x min_nor , y min_nor ) of the upper left corner and the normalized coordinates (x max_nor , y max_nor ) of the lower right corner of the rectangular box, determine the coordinates of the target object.

[0069] The main steps to extract the coordinates of the rectangular box are as follows:

[0070] 4.1) Convert the normalized coordinates (x min_nor , y min_nor ) of the upper left corner and the normalized coordinates (x max_nor , y max_nor ) of the lower right corner into the position coordinate values of the upper left and lower right corners of the rectangular box, that is:

[0071]

[0072] In the formula, img_width and img_height respectively represent the image width and image height. The subscripts min and max indicate taking min or max in one calculation. The subscript min represents the upper left corner, the subscript max represents the upper right corner, x represents the abscissa, and y represents the ordinate.

[0073] 4.2) Due to factors such as occlusion and overlap, the phenomenon of missing coordinate data appears in some video frames. In order to ensure the consistency of the lengths of each data item, first fill these missing places with the outlier 0.

[0074] 4.3) Determine the coordinates of the target object card(x min,max ,y min,max ), that is:

[0075] card(x min,max ,y min,max ) = card_p(x min,max ,y min,max ) + card_n(x min,max ,y min,max ) (2)

[0076] In the formula, card_p(x min,max ,y min,max ) and card_n(x min,max ,y min,max ) respectively represent the coordinates of the objects printed on the front and back of the target object.

[0077] 5) Process the coordinates of the target object. The main steps are as follows:

[0078] 5.1) Replace the coordinates of the target object with the center coordinates (x, y) of the target object, that is:

[0079]

[0080] In the formula, (x, y) is the center coordinates of the target object.

[0081] 5.2) Use the piecewise linear interpolation function I n (t) to linearly interpolate and complete the missing data. Among them, the piecewise linear interpolation function I n (t) is a linear function in each interval [t i ,t i+1 . The piecewise linear interpolation function I n (t) is as follows:

[0082]

[0083] In the formula, I n (t) is the piecewise linear interpolation function, f i represents the actual value corresponding to t i ; l i (t) is the interpolation basis function, ensuring that the calculation of the inserted value is only related to the adjacent two nodes; t is the segmentation point; i is the segmentation serial number; n is the total number of segments; t i represents the i-th segmentation point.

[0084] The interpolation basis function l i (t) is as follows:

[0085]

[0086] A piecewise linear interpolation function I within a piecewise interval n (t i ) is as follows:

[0087] I n (t i ) = f i (6)

[0088] 5.3) Interpolate and complete the missing data, and perform median filtering.

[0089] 6) Establish an action recognition model, that is:

[0090]

[0091] In the formula, dis 12 (t) is the Euclidean distance between target object 1 and target object 2; x 12 (t) is the relative position between target object 1 and target object 2; subscript 1 represents target object 1, and subscript 2 represents target object 2.

[0092] 7) Input the processed rectangle box coordinates into the action recognition model to recognize the human pose.

[0093] The criteria for recognizing the human pose are as follows: If the threshold and timing conditions are simultaneously met in N consecutive video frames, the subject has completed the indicated action; otherwise, the subject has not completed the indicated action. The judgment function is as follows:

[0094]

[0095] In the formula, t flag is the time flag bit. k 12 is the human pose recognition parameter; dis thr , k thr , x thr , y thr represent the Euclidean distance threshold, the human pose recognition parameter threshold, the threshold, and the ordinate threshold respectively. The human pose recognition parameter k 12 is related to the video frame images and coordinate positions of target object 1 and target object 2. In practical applications, the parameters k 12 and k thr can be omitted. The F(*;#) function compares * and #, and takes 1 if the threshold condition is met, otherwise takes 0.

[0096] Example 2:

[0097] See Figures 2 to 14, The experiment using the action recognition method based on TensorFlow object detection to recognize Action 1 (put the toothbrush on the card and then take it back) and Action 2 (put the mouse on the other side of the card and then turn the card over) is mainly as follows:

[0098] 1) The main steps of object detection are as follows: 1.1) According to the objects included in the recognized actions, such as toothbrush, mouse, apple, banana, etc., select the coco dataset faster_rcnn_inception_v2_coco_2018_01_28 trained based on Faster-RCNN; bananas and apples are the objects printed on the front and back of the card.

[0099] 1.2) Use the TensorFlow object detection interface to detect and track the objects in the recognized action video, and record the object categories classes recognized in each frame of the video;

[0100] 1.3) Use the detection_graph.get_tensor_by_name function in the object detection interface to extract the normalized coordinates of the upper left and lower right corners of the rectangular frame of the selected object, and fill the missing coordinate data with 0;

[0101] 1.4) Extract the pixel width and height of each frame of the video, and multiply by the normalized coordinates to convert them into actual coordinates;

[0102] 1.5) The card coordinates can be obtained by adding the banana and apple coordinates;

[0103] 1.6) Save the recognized action video. The main results of object detection: Extract the 116th frame image of a recognized action video, measure its pixels as 1080×1920, and after being processed by the TensorFlow object detection interface, the target objects are displayed in the form of a rectangular frame + class name + confidence, such as Figure 4 and Figure 5 as shown.

[0104] 2) The main steps of coordinate data processing are as follows: 2.1) Centralize the rectangular frame coordinate data and represent the object in the form of the center coordinates (x, y) of the rectangular frame;

[0105] 2.2) Perform linear interpolation processing on the x and y coordinates of the toothbrush, mouse, and card respectively;

[0106] 2.3) Perform median filtering processing with a neighborhood of 11 on the x and y coordinates of the toothbrush, mouse, and card respectively.

[0107] The main results of coordinate data processing: After interpolation processing, the original x coordinate data of the toothbrush fills in the missing data, and after median filtering with a neighborhood of 11, the abnormal fluctuation data is eliminated, such asFigure 6 , Figure 7 , Figure 8 as shown.

[0108] 3) Main steps of Action 1:

[0109] 3.1) Calculate the Euclidean distance dis_toothbrush_card and relative position x_toothbrush_card between the toothbrush and the card;

[0110] 3.2) Set threshold conditions according to the trend of dis_toothbrush_card changing over time. When it is less than 200, it is regarded as completing "put the toothbrush on the card", and record the timing condition flag bit put_on_flag when this action is completed;

[0111] 3.3) Starting from put_on_flag as the timing starting point, when dis_toothbrush_card is greater than 400 and the x_toothbrush_card at the same time is of the same sign as the value at the initial moment, it is regarded as completing the action of "put the toothbrush back".

[0112] 4) Main steps of Action 2:

[0113] 4.1) According to the recorded classes, find the moment when the apple and the banana meet as the timing condition flag bit trans_flag for completing the action of "turn the card over";

[0114] 4.2) Calculate the Euclidean distance dis_mouse_card and relative position x_mouse_card between the mouse and the card;

[0115] 4.3) According to the change trends of x_mouse_card and dis_mouse_card, when x_mouse_card is greater than 0 and the corresponding dis_mouse_card is greater than 200, it is regarded as completing the action of "put the mouse on the other side of the card", and record the timing condition flag bit put_flag at this time;

[0116] 4.4) When put_flag < trans_flag, it is regarded as completing Action 2.

[0117] 5) Recognition results of Action 1 and Action 2: The changing trends of dis_toothbrush_card and x_toothbrush_card for the standard completion of the test of Action 1 are as Figure 9 , Figure 10 shown, and the judgment process is as Figure 11As shown, the changing trends of dis_mouse_card and x_mouse_card for the standard completion action 2 of the test are as follows Figure 12 , Figure 13 As shown, the judgment process is as follows Figure 14 As shown. To illustrate the action recognition effect of the present invention, the following 10 groups of experiments were designed according to possible situations to verify its speed and accuracy:

[0118]

[0119] According to the test results, the action recognition speed of the target detection method is about 1.914 frames / s, and all 10 groups of experiments tested were correctly recognized. To reflect the improvement effect of the present invention, the same experiments were carried out using the existing method: the action recognition speed of the existing method is about 1.657 frames / s. Among the 10 groups of experiments tested, there was 1 judgment error. In terms of speed, the target detection method was improved by 16%. For the recognition results, when the existing method uses image morphology processing to locate the card, it is extremely vulnerable to light interference, and the extracted coordinates will deviate from the center of the card, so the result reliability is not high. While the target detection method always locates the object at the center of the card, with high coordinate accuracy and more reliable results.

[0120] In summary, the present invention proposes an action recognition method based on TensorFlow object detection. The invention extracts the rectangular frame coordinates of the detected object based on the object detection method provided by the TensorFlow object detection interface, realizes the positioning and front-back distinction of the card, with high accuracy, good robustness, and rich recognizable categories; based on data processing methods of centralization, linear interpolation, and median filtering, the object is replaced with the central coordinates to simplify the analysis process, and interpolation filtering improves the accuracy of the coordinate data, ensuring the accuracy of action recognition; based on the method of extracting Euclidean distance and relative position from the coordinate data, the establishment of the action recognition model is realized, and by setting thresholds and timing conditions, the action recognition is quickly and accurately completed by analyzing multiple video frame data. The present invention improves the speed and accuracy of action recognition and quickly and accurately completes action recognition.

Claims

1. An action recognition method based on TensorFlow object detection, characterized in that It mainly includes the following steps: 1) Obtain a video stream containing human postures and target objects, and decompose it into several frame images; patterns are printed on both sides of the target object; 2) Use TensorFlow and the Faster R-CNN algorithm to build a convolutional neural network, and train to obtain a target object recognition model; 3) Use the target object recognition model to frame the target objects in each frame of the image, and label the rectangular frames with category labels; determine the normalized coordinates (x min_nor , y min_nor ) of the upper left corner of the rectangular frame, the normalized coordinate values (x max_nor , y max_nor ) of the lower right corner, and the confidence level c; x min_nor , y min_nor , x max_nor , y max_nor , c ∈ [0, 1]; 4) Determine the coordinates of the target object based on the normalized coordinates (x min_nor , y min_nor ) of the upper left corner of the rectangle and the normalized coordinate values (x max_nor , y max_nor ) of the lower right corner; 5) Process the coordinates of the target object; 6) Build an action recognition model; 7) Input the processed rectangular box coordinates into the action recognition model to recognize the human posture; The action recognition model is as follows: where dis 12 (t) is the Euclidean distance between target object 1 and target object 2; (x 12 (t), y 12 (t)) is the relative position between target object 1 and target object 2; subscript 1 represents target object 1, and subscript 2 represents target object 2; The criteria for recognizing the human posture are as follows: if the threshold and timing conditions are simultaneously met in N consecutive video frames, the subject has completed the indicated action; otherwise, the subject has not completed the indicated action; the judgment function is as follows: where t flag is the time flag bit; k 12 is the human body posture recognition parameter; dis thr , k thr , x thr , y thr represent the Euclidean distance threshold, the human body posture recognition parameter threshold, the threshold, and the ordinate threshold, respectively.

2. The action recognition method based on TensorFlow object detection according to claim 1, wherein The human posture is the action completed by the subject according to the instructions of the AD scale.

3. The action recognition method based on TensorFlow object detection according to claim 1, characterized in that The convolutional neural network includes an input layer, a pooling layer, and a convolutional layer.

4. The action recognition method based on TensorFlow object detection according to claim 1, wherein, The training dataset of the target object recognition model is the COCO dataset trained by the Faster-RCNN deep learning object detection algorithm.

5. The action recognition method based on TensorFlow object detection according to claim 1, characterized in that, The target object is a rectangular card; different patterns are printed on the front and back of the rectangular card.

6. The action recognition method based on TensorFlow object detection according to claim 1, characterized in that The main steps for extracting the rectangular box coordinates are as follows: 1) Convert the normalized coordinates (x min_nor , y min_nor ) at the upper left corner and the normalized coordinate values (x max_nor , y max_nor ) at the lower right corner into the coordinate values (x min , y min ) at the upper left corner of the rectangular box and the coordinate values (x max , y max ) at the lower right corner position, that is: In the formula, img_width and img_height represent the image width and image height respectively; 2) Fill the missing coordinate data with 0; 3) Determine the upper left corner coordinates card(x min , y min ) and the lower right corner coordinates card(x max , y max ), that is: wherein, card_p(x min , y min ) and card_n(x min , y min ) respectively represent the upper left coordinates of the object printed on the front and back of the target object; card_p(x max , y max ) and card_n(x max , y max ) respectively represent the lower right coordinates of the object printed on the front and back of the target object.

7. The action recognition method based on TensorFlow object detection according to claim 1, characterized in that The main steps for processing the coordinates of the target object are as follows: 1) Replace the target object coordinates with the center coordinates (x, y) of the target object, that is: In the formula, (x, y) are the center coordinates of the target object; 2) Use the piecewise linear interpolation function I n (t) to linearly interpolate and complete the missing data; among them, the piecewise linear interpolation function I n (t) is a linear function in each interval [t i , t i+1 , and the piecewise linear interpolation function I n (t) is as follows: where, I n (t) is a piecewise linear interpolation function, and f i represents the actual value corresponding to the i-th segmentation point t i ; l i (t) is an interpolation basis function, ensuring that the calculation of the inserted value is only related to two adjacent nodes; t is the segmentation point; i is the segmentation serial number; n is the total number of segments; Interpolation basis function l i (t) is as follows: A piecewise linear interpolation function I within a piecewise interval n (t i ) is as follows: I n (t i ) = f i (6) 3) Interpolate and complete the missing data, and perform median filtering.

Images