Skipping Rope Counting Method, Device, Medium and Electronic Device Based on Action Classification

Abstract

The present application relates to a skipping rope counting method, device, medium and electronic device based on action classification, and relates to the technical field of skipping rope counting. The method includes: obtaining a live skipping rope video of a target object, where the target object is a person who needs to count the number of skipping ropes; based on the live skipping rope video, determining video key frame features and 3D pose features related to the target object's skipping rope; based on the video key frame features and the 3D pose features, determining the current skipping rope state of the target object through a preset skipping rope action classification model; if the skipping rope state is starting to skip rope, counting the number of skipping ropes of the target object; if the skipping rope state is the end of skipping rope, when it is detected that the skipping rope state is starting to skip rope after the current time, recording that the skipping rope is temporarily interrupted once. The present application has the effect of improving the accuracy of skipping rope counting.

Description

Technical Field

[0001] This application relates to the technical field of skipping rope counting, and specifically relates to a skipping rope counting method, device, medium, and electronic device based on action classification. Background Art

[0002] Skipping rope is a sports game in which one or more people perform various jumping actions in a swinging rope. The main materials include cotton ropes, PVC (plastic) ropes, bamboo joint ropes, and steel ropes. Skipping rope counting refers to the act of counting the number of times of skipping rope during the skipping process. Skipping rope counting can help the skipper understand their exercise intensity and amount of exercise, so as to formulate a reasonable exercise plan and achieve a better exercise effect.

[0003] Currently, the commonly used method for skipping rope counting is: the method based on computer vision counting, that is, detecting the human key points of the skipper, and counting the skipping rope of the skipper based on the temporal characteristics of the human key points. In this way, once there are extra actions during the skipping rope counting or pseudo-skipping actions during the skipping rope interruption, inaccurate skipping rope counting problems such as overcounting or undercounting may occur, resulting in poor accuracy of skipping rope counting. Summary of the Invention

[0004] In order to improve the accuracy of skipping rope counting, this application provides a skipping rope counting method, device, medium, and electronic device based on action classification.

[0005] In the first aspect of this application, a skipping rope counting method based on action classification is provided, which specifically includes:

[0006] Obtain the on-site skipping rope video of the target object, where the target object is the person who needs to count the skipping rope;

[0007] Based on the on-site skipping rope video, determine the video key frame features and 3Dpose features related to the target object's skipping rope;

[0008] Based on the video key frame features and the 3D pose features, determine the current skipping rope state of the target object through a preset skipping rope action classification model;

[0009] If the skipping rope state is starting to skip rope, count the skipping rope of the target object;

[0010] If the skipping rope state is the end of skipping rope, when it is detected that the skipping rope state is starting to skip rope after the current time, record that the skipping rope is temporarily interrupted once.

[0011] By adopting the above technical solution, after obtaining the on-site skipping video, the video key frame features and 3D pose features related to the target object's skipping are extracted from the on-site skipping video. Through the video key frame features, the key frame detail information of the recorded skipping action can be better captured. Through the 3D pose features, the action posture of the target object can be comprehensively reflected. Further, based on the video key frame features and 3D pose features, the skipping action of the target object is accurately classified by the skipping action classification model, so as to more accurately determine the current skipping state of the target object. If the skipping state is starting to skip, then start counting the target object's skipping, thereby filtering the interference of non-skipping actions on counting to a certain extent, and improving the accuracy of skipping counting.

[0012] Optionally, before obtaining the on-site skipping video of the target object, it further includes:

[0013] Obtain skipping video sample data corresponding to different skipping action categories, where the skipping action categories include starting to skip, skipping, ending skipping, and not skipping;

[0014] For a single piece of the skipping video sample data, perform human body detection through a preset target detection algorithm, and after the human body detection is completed, perform pose detection through a preset pose detection algorithm to obtain the corresponding pose sequence data;

[0015] Based on the pose sequence data, determine the corresponding 3D pose feature sample, and for a single piece of the skipping video sample data, determine the corresponding skipping action video key frame;

[0016] Based on the skipping action video key frame, determine the corresponding key frame feature sample, and fuse the 3D pose feature sample and the key frame feature sample corresponding to the same piece of the skipping video sample data to obtain the fused feature;

[0017] Based on the fused features corresponding to each piece of the skipping video sample data, train a preset untrained action classification model to obtain a skipping action classification model.

[0018] By adopting the above technical solution, after obtaining the skipping video sample data of different skipping action categories, extract the corresponding 3D pose feature samples and key frame feature samples from each piece of the skipping video sample data respectively, and train the model by fusing the features of the two dimensions of the 3D pose feature samples and the key frame feature samples, so that the training effect of the model is better, the performance of the trained model is better, and further the skipping action classification model can more accurately identify and classify the skipping actions of the target object.

[0019] Optionally, the rope skipping counting for the target object specifically includes:

[0020] Based on the on-site rope skipping video, perform real-time rope skipping counting on the target object to obtain the number of rope skips, and detect whether there is a rope;

[0021] If so, when the rope skipping state is detected as the end of rope skipping after the current time, determine the number of rope skips in the interval from the current time to the end of rope skipping as the number of rope skips with a rope;

[0022] If not, when the rope skipping state is detected as the end of rope skipping after the current time, determine the number of rope skips in the interval from the current time to the end of rope skipping as the number of rope skips without a rope.

[0023] By adopting the above technical solution, during the process of rope skipping counting, if a rope is detected during the rope skipping process, it indicates that the target object is performing rope skipping with a rope before the end of rope skipping. Then, determine the number of rope skips counted in real time as the number of rope skips with a rope; otherwise, determine the number of rope skips counted in real time as the number of rope skips without a rope, thereby realizing accurate counting of the target object's rope skipping.

[0024] Optionally, the real-time rope skipping counting for the target object based on the on-site rope skipping video to obtain the number of rope skips specifically includes:

[0025] Based on the on-site rope skipping video, determine at least one human core key point corresponding to the target object;

[0026] Perform filtering processing on all the human core key points to obtain filtered key points;

[0027] Count the number of up and down reciprocating movements of the filtered key points to obtain the number of rope skips corresponding to the target object.

[0028] By adopting the above technical solution, perform filtering processing on the human core key points to remove the noise in the human core key points and improve the data quality. Finally, since a single up and down reciprocating movement reflects a single rope skipping process in which the target object jumps up and then falls from a high place, the number of up and down reciprocating movements of the filtered key points is determined as the number of rope skips, thereby counting the target object's rope skipping more accurately.

[0029] Optionally, the detection of whether there is a rope specifically includes:

[0030] When the up and down reciprocating movement of the filtered key points is detected, determine the occurrence period of a single up and down reciprocating movement in the on-site rope skipping video as the counting period;

[0031] Determine whether there is a first video frame and a second video frame in the skipping rope video segment corresponding to the counting period, where the first video frame is a video frame in which a rope is detected in the upper half of the target object, and the second video frame is a video frame in which a rope is detected in the lower half of the target object;

[0032] If so, determine that there is a rope; if not, determine that there is no rope.

[0033] By adopting the above technical solution, a single up-and-down reciprocating motion indicates that the target object performs a single skipping rope motion. If the rope appears in the upper half and the lower half of the target object respectively during the counting period corresponding to a single skipping rope motion, it means that there is a rope during the skipping rope process of the target object; otherwise, it means that there is no rope during the skipping rope process, thereby more accurately determining whether there is a rope during the skipping rope process of the target object.

[0034] Optionally, the method further includes:

[0035] When it is currently determined that the target object has a temporary interruption in skipping rope, obtain at least one actual skipping rope height and the corresponding actual wind speed of the target object from the start of skipping rope to the current time, where the actual wind speed is the wind speed when the skipping rope height of the target object is the actual skipping rope height;

[0036] Obtain the wind speed intervals where the historical wind speeds are located when historical rope-skippers have temporary interruptions in skipping rope, count the first occurrence times of each wind speed interval, and select the first number of wind speed intervals from each wind speed interval in descending order of the first occurrence times to be determined as key wind speed intervals;

[0037] Obtain the height intervals where the historical skipping rope heights are located when the historical wind speeds are in a single key wind speed interval and historical rope-skippers have temporary interruptions in skipping rope, count the second occurrence times of each height interval, select the second number of height intervals from each height interval in descending order of the second occurrence times, and determine them as the key height intervals corresponding to the single key wind speed interval;

[0038] Determine the first weight of each key wind speed interval and the second weight of the key height interval corresponding to each key wind speed interval. The first weight is the ratio of the first occurrence time of each key wind speed interval to the sum of the first occurrence times of all key wind speed intervals, and the second weight is the ratio of the second occurrence time of the single key height interval corresponding to the key wind speed interval to the sum of the second occurrence times of all corresponding key height intervals;

[0039] Based on the first weight, the second weight, the actual skipping rope height, and the corresponding actual wind speed, verify the current temporary interruption in skipping rope of the target object.

[0040] By adopting the above technical solution, the greater the number of first occurrences, the more likely the rope jumper will have a temporary interruption in rope jumping when the wind speed is in the corresponding wind speed interval, thereby determining the key wind speed interval; the greater the number of second occurrences, the more likely the rope jumper will have a temporary interruption in rope jumping when the rope jumper's rope jumping height is in the corresponding height interval in a single key wind speed interval, thereby determining the key height interval. Finally, combining the first weight and the second weight, the possibility of a temporary interruption in rope jumping is analyzed, thereby realizing the verification of the temporary interruption of rope jumping of the target object, thereby making the record of temporary interruption of rope jumping more accurate.

[0041] Optionally, the checking of whether the current target object has temporarily interrupted rope skipping based on the first weight, the second weight, the actual rope skipping height and the corresponding actual wind speed specifically includes:

[0042] The key height interval where the single actual rope skipping height is located is determined as the target height interval, and if there is a corresponding actual wind speed in the key wind speed interval, the corresponding key wind speed interval is determined as the target wind speed interval;

[0043] If the target height interval exists in each key height interval corresponding to the target wind speed interval, the corresponding target wind speed interval is determined as an important wind speed interval, and the product of the first weight of each important wind speed interval and the second weight of the corresponding target height interval is calculated;

[0044] When the current temporary interruption of rope skipping is the first interruption from the start of rope skipping to the current process, the products are summed to obtain the sum of the products. If the sum of the products exceeds a preset threshold, it is verified that the current target object has a temporary interruption of rope skipping.

[0045] By adopting the above technical solution, the product of the first weight of each important wind speed interval and the second weight of the corresponding target height interval is calculated. The larger the product, the greater the possibility of temporary interruption of rope skipping when the actual wind speed is in the important wind speed interval and the actual rope skipping height of the target object is in the corresponding target height interval. Finally, when the current temporary interruption of rope skipping is the first interruption from the start of rope skipping to the current process, the sum of the products is summed to obtain the sum of the products. If the sum of the products is greater than the preset threshold, it means that the overall possibility of temporary interruption of rope skipping is relatively high in the target object from the start of rope skipping to the current time interval, and there is a high probability of temporary interruption of rope skipping at present, then it is verified that the current target object does have a temporary interruption of rope skipping.

[0046] In a second aspect of the present application, a rope skipping counting device based on action classification is provided, which specifically comprises:

[0047] An information acquisition module, configured to acquire a live skipping video of a target object, where the target object is a person who needs to have their skipping counted;

[0048] A feature determination module, configured to determine video key frame features and 3D pose features related to the target object's skipping based on the live skipping video;

[0049] An action classification module, configured to determine the current skipping state of the target object based on the video key frame features and the 3D pose features through a preset skipping action classification model;

[0050] A skipping counting module, configured to count the number of skips of the target object if the skipping state is starting to skip;

[0051] An interruption recording module, configured to record one temporary skipping interruption if the skipping state is ending the skip and it is detected that the skipping state is starting to skip after the current time.

[0052] By adopting the above technical solution, the information acquisition module acquires the live skipping video of the target object, the feature determination module determines the video key frames and 3D pose features related to the target object's skipping. Then, the action classification module determines the current skipping state of the target object based on the video key frame features and the 3D pose features through a preset skipping action classification model. The skipping counting module counts the number of skips of the target object when the skipping state is starting to skip. Finally, the interruption recording module records one temporary skipping interruption when the skipping state is ending the skip and it is detected that the skipping state is starting to skip after the current time.

[0053] In a third aspect of the present application, there is provided a computer-readable storage medium, in which a computer program is stored. When the computer program is loaded and executed by a processor, the method steps described in any one of the first aspects are executed.

[0054] In a fourth aspect of the present application, there is provided an electronic device, specifically including:

[0055] A processor, a memory, and a computer program stored in the memory and capable of running on the processor. The processor is configured to load and execute the computer program stored in the memory so that the electronic device executes the method described in any one of the first aspects.

[0056] In summary, the present application includes at least one of the following beneficial technical effects:

[0057] After obtaining the on-site skipping rope video, video key frame features and 3D pose features related to the target object's skipping rope are extracted from the on-site skipping rope video. Through the video key frame features, the key frame detail information recording the skipping rope action can be better captured. Through the 3D pose features, the action posture of the target object can be comprehensively reflected. Further, based on these video key frame features and 3D pose features, the skipping rope action of the target object is accurately classified through a skipping rope action classification model, so as to more accurately determine the current skipping rope state of the target object. If the skipping rope state is starting to skip rope, then start counting the skipping rope of the target object, thereby filtering the interference of non-skipping rope actions on counting to a certain extent, and improving the accuracy of skipping rope counting. Description of the Drawings

[0058] Figure 1 is a schematic flowchart of a skipping rope counting method based on action classification provided by an embodiment of the present application;

[0059] Figure 2 is a schematic structural diagram of a skipping rope counting device based on action classification provided by an embodiment of the present application;

[0060] Figure 3 is a schematic structural diagram of another skipping rope counting device based on action classification provided by an embodiment of the present application.

[0061] Description of the reference numerals: 11, information acquisition module; 12, feature determination module; 13, action classification module; 14, skipping rope counting module; 15, interruption recording module; 16, model training module; 17, record verification module. Detailed Embodiments

[0062] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments.

[0063] In the description of the embodiments of the present application, words such as "exemplarily", "for example" or "for instance" are used to indicate examples, illustrations or explanations. Any embodiment or design solution described as "exemplarily", "for example" or "for instance" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplarily", "for example" or "for instance" is intended to present relevant concepts in a specific manner.

[0064] In the description of the embodiments of the present application, the term "and / or" is merely a description of the association relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, B exists alone, and both A and B exist simultaneously. Additionally, unless otherwise specified, the meaning of the term "plural" refers to two or more. For example, plural systems refer to two or more systems, and plural screen terminals refer to two or more screen terminals. Furthermore, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. The terms "include", "comprise", "have" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0065] See Figure 1 , the embodiments of the present application disclose a schematic flowchart of a skipping rope counting method based on action classification, which can be implemented depending on a computer program or run on a skipping rope counting device based on the von Neumann architecture. This computer program can be integrated into an application or run as an independent tool class application, and specifically includes:

[0066] S101: Obtain the on-site skipping rope video of the target object.

[0067] Specifically, the target object is the person who needs to count the skipping rope. The on-site skipping rope video is a video that records the skipping rope situation of the target object in the outdoor environment in real time. In other embodiments, the current action video can also be a video that records the skipping rope situation of the target object in the indoor environment. Additionally, the execution subject of the skipping rope counting method based on action classification disclosed in the embodiments of the present application is the server, and the server communicates wirelessly with the terminal and the camera. The terminal can be a personal computer or a smart phone. A client or a small program related to skipping rope counting is installed in the terminal, and the server can be the background server of the client or the small program, specifically an independent physical server or a server cluster composed of multiple physical servers.

[0068] Further, an implementation scenario of the skipping rope counting method based on action classification disclosed in this application is as follows: during the skipping rope training of students in school physical education classes or during the skipping rope subject in a physical education exam for candidates, when skipping rope counting is required, after the skipper is ready, the person sends a start counting instruction to the server through the client in the terminal. The server obtains the instruction and obtains the live video of the area where the target object is skipping rope in real time through the camera, obtaining the live skipping rope video, and the live skipping rope video only contains the target object. Corresponding cameras are set in different skipping rope areas. In other embodiments, in the live skipping rope video obtained by the camera for the target object, in addition to the target object, it may also contain other people skipping rope.

[0069] S102: Based on the live skipping rope video, determine the video key frame features and 3D pose features related to the target object's skipping rope.

[0070] Specifically, the video key frame refers to the frame with specific significance and function in the video. In the embodiments of this application, the video key frame is the frame of the key action reflecting the skipping rope state of the target object in the live skipping rope video. Exemplarily, the video key frame in the live skipping rope video can be a frame when the target object jumps to the highest point in the air in the video, or a frame at the moment when the target object takes off in the video. The video key frame features refer to the characteristics and attributes possessed by the video key frames in the live skipping rope video. After obtaining the live skipping rope video, the picture data of the video key frames related to the target object's skipping rope is obtained from the live skipping rope video through the preset Final Cut Pro X tool. Then, through the yolo series feature skeleton structures, such as yolov5 or yolov7, the video key frame features are extracted from the picture data.

[0071] Further, the 3D pose feature refers to the feature information describing the pose of the target object when skipping rope in the three-dimensional space. The 3D pose feature can capture the specific spatial positions of various parts of the target object's body when skipping rope, including the coordinates of parts such as the arms, legs, and head in the three-dimensional space, so as to present the skipping rope state of the target object more comprehensively and accurately. In the embodiments of this application, a feasible way to determine the 3D pose feature is as follows: detect the human body of the target object in the live skipping rope video through a preset target detection algorithm, and the target detection algorithm adopts the yolo series detection algorithms. Then, detect the pose of the target object in the live skipping rope video through a preset pose detection algorithm to obtain the pose data corresponding to this live skipping rope video. Finally, extract the heat map features from the pose data through the 3D convolution structure to obtain the 3D pose features.

[0072] S103: Based on the video key-frame features and 3D pose features, determine the current skipping state of the target object through a preset skipping action classification model.

[0073] Specifically, after the video key-frame features and 3D pose features are determined, input the video key-frame features and 3D pose features into the preset skipping action classification model to identify and classify the skipping actions of the target object in the on-site skipping video, and finally determine the current skipping state of the target object, so as to facilitate more accurate skipping counting for the target object. The skipping states include but are not limited to starting to skip, skipping, ending skipping, and not skipping. Starting to skip means that the target object enters the skipping movement process from not skipping, that is, the movement process from the state of not starting to skip to the state of swinging the rope with both hands and taking off. Skipping means the movement process of the target object's double-foot jump or single-foot and double-foot alternating jump. Ending skipping means that the target object enters the process of stopping skipping from skipping, mainly referring to the movement process from the state of swinging the rope with both hands and continuous jumping to stopping swinging the rope or stopping jumping. Not skipping means that the target object is not in the skipping action. Exemplarily, if it is determined through the skipping action classification model that the current skipping action classification of the target object is starting to skip, then the current skipping state of the target object is starting to skip. It should be noted that the skipping action classification model is a trained convolutional neural network model. In other embodiments, the skipping action classification model can also be a trained recurrent neural network model.

[0074] Further, before obtaining the on-site skipping video of the target object, during the process of determining the skipping action classification model, the specific process is as follows: Collect the historical skipping videos of students recorded by the camera. The historical skipping videos include videos of the entire process of single-person skipping. Then, use a video editing tool to clip out the skipping videos that only contain a single skipping action category from each historical skipping video, that is, the skipping video sample data. The skipping action categories include starting to skip, skipping, ending skipping, and not skipping. Among them, the video editing tool can use the Adobe Premiere Pro tool. In other embodiments, it can also be the Final Cut Pro X tool.

[0075] Further, for a single skipping rope video sample data, the human body target in the video is detected through a preset object detection algorithm, that is, the existence position and approximate area of the human body are located in the skipping rope video sample data. The object detection algorithm can be Faster R-CNN based on deep learning. In other embodiments, it can also be the yolo series detection algorithms. After the human body detection is completed, a pose detection is performed on the skipping rope video sample data through a preset pose detection algorithm, and finally the pose sequence data corresponding to the skipping rope video sample data is obtained, that is, a series of continuous human body pose information in the skipping rope video sample data, which includes the pose change of the human body over a period of time. Among them, the pose detection algorithm can be the hrnet detection algorithm. In other embodiments, it can also be the OpenPose algorithm.

[0076] Further, according to the pose sequence data, the corresponding 3D pose feature sample is determined. In the embodiment of the present application, a feasible determination method is: convert the pose sequence data into json file data, and through Gaussian mapping, construct heatmap data in the time series based on the json file data. Finally, extract heatmap features from the heatmap data through a 3D convolutional structure to obtain the 3D pose feature sample corresponding to the skipping rope video sample data. Then, for a single skipping rope video sample data, the key frames of the skipping rope action video are determined, that is, the video key frames in the skipping rope video sample data are determined. Then, based on the key frames of the skipping rope action video, the corresponding key frame feature samples are extracted. The specific determination method can refer to step S102 and will not be elaborated here.

[0077] Further, the 3D pose feature sample and the key frame feature sample corresponding to the same skipping rope video sample data are fused to obtain the fused feature. Thereby reducing the number of channels that the model needs to process, improving the quality of the sample data input into the model, reducing the calculation cost, and enhancing the performance of the model. Specifically, feature fusion can be performed through Channel Fusion. In other embodiments, feature fusion can also be performed through principal component analysis. Among them, Channel Fusion is a technology widely used in the fields of image processing and computer vision, which refers to the process of fusing image data from multiple sources or multiple channels.

[0078] Finally, the fused features corresponding to each skipping video sample data are input into a preset untrained action classification model for training. During the training process, cross-entropy loss function is used for constraint supervision until the model converges, and a skipping action classification model that can identify the skipping action category of a person based on 3D pose features and key frame features is obtained. Among them, the untrained action classification model is a convolutional neural network model. This process is a prior art and will not be elaborated here. It should be noted that compared with the conventional skipping action classification based only on 3D pose data during the skipping process, in this application, video key frames during the skipping process are also fused, which can capture the moment when a person starts or ends skipping more timely, making the accuracy of skipping action classification higher.

[0079] S104: If the skipping state is starting to skip, count the number of skips of the target object.

[0080] Specifically, if it is determined through the skipping action classification model that the current skipping action category of the target object is starting to skip and the skipping state is determined to be starting to skip, then start counting the number of skips of the target object. One implementable embodiment is as follows: Through a preset OpenPose algorithm, based on the on-site skipping video, various human key points of the target object are detected, including but not limited to the head, neck, shoulders, elbows, wrists, hips, knees, and ankles, etc. Then, at least one human core key point is selected from the various human key points. The human core key point refers to an important position point that plays a key role in describing the action posture of the target object's skipping. In the embodiment of this application, the human core key points include the key points of the target object's feet and hands. Further, filtering processing is performed on each human core key point to obtain filtered key points. Specifically, mean filtering processing is performed through a OneEuroFilter to remove the noise in the human core key points and improve the data quality.

[0081] Finally, count the number of up-and-down reciprocating movements of the filtered key points. Specifically, the OpenPose algorithm can observe the movement pattern of the filtered key points in the vertical direction according to the position change trajectory of the filtered key points in the consecutive frames of the on-site skipping video. If the filtered key points continuously make periodic position changes in the up-and-down direction within a period of time, it is determined that there is an up-and-down reciprocating movement, and the number of up-and-down reciprocating movements is counted to obtain the number of skips corresponding to the target object.

[0082] While counting the number of rope skipping of the target object, it is also necessary to detect whether there is a rope based on the on-site rope skipping video. If a rope is detected during the rope skipping process of the target object, it means that the rope skipping performed by the target object is rope skipping with a rope. Then, when the rope skipping state is detected as the end of rope skipping after the current time, the number of rope skipping within the interval from the current time to the end of rope skipping is determined as the number of rope skipping with a rope; conversely, if no rope is detected, it means that the rope skipping performed by the target object is rope skipping without a rope. Then, when the rope skipping state is detected as the end of rope skipping after the current time, the number of rope skipping within the interval from the current time to the end of rope skipping is determined as the number of rope skipping without a rope. Among them, for detecting whether there is a rope, a feasible detection method is: when it is detected that the filtered key point moves up and down reciprocally, the occurrence period of a single up-and-down reciprocal movement in the on-site rope skipping video is determined as the counting period, or the counting cycle. Then, rope detection is performed on each video frame of the rope skipping video segment corresponding to this counting period in the on-site rope skipping video. If there is a rope in the upper part of the target object in the video frame, then it is determined that there is a first video frame in the rope skipping video segment; if there is a rope in the lower part of the target object in the video frame, then it is determined that there is a second video frame in the rope skipping video segment. When both the first video frame and the second video frame exist in the rope skipping video segment, it is determined that there is a rope during the rope skipping process of the target object; conversely, it is determined that there is no rope during the rope skipping process of the target object.

[0083] S105: If the rope skipping state is the end of rope skipping, then when the rope skipping state is detected as starting rope skipping after the current time, record that the rope skipping is temporarily interrupted once.

[0084] Specifically, if the rope skipping state is the end of rope skipping, it means that the target object may have the rope touch the foot, resulting in a temporary interruption of rope skipping, or the rope skipping task of the target object has ended. Then, continue to classify the real-time rope skipping actions of the target object. If it is determined again that there is a rope skipping action category of starting rope skipping for the target object, that is, the rope skipping state is detected as starting rope skipping after the current time, it means that the target object has a temporary interruption of rope skipping. Then, record it as a temporary interruption of rope skipping once.

[0085] In other embodiments, when it is currently determined that the target object has a temporary interruption of rope skipping, it means that the target object may have made a mistake in rope skipping, such as the rope touching the foot, resulting in a temporary interruption of rope skipping, rather than the end of rope skipping. Then, obtain at least one actual rope skipping height and the corresponding actual wind speed of the target object from the start of rope skipping to the current time. Specifically, the actual wind speed is obtained through a preset wind speed sensor, and the distance of the target object's feet from the ground is measured through a preset distance sensor, so as to obtain the actual rope skipping height. Among them, the actual wind speed is the wind speed when the rope skipping height of the target object is the actual rope skipping height. Exemplarily, the actual rope skipping height A and the corresponding actual wind speed B can be understood as when the rope skipping height of the target object is A, the environmental wind speed is B.

[0086] Further, based on the historical records of skipping rope interruptions, obtain the wind speed intervals in which the historical wind speed was when the historical rope-skippers had temporary skipping rope interruptions. Count the first occurrence times of each wind speed interval. The larger the first occurrence times, the more likely the rope-skippers are to have temporary skipping rope interruptions when the wind speed is within the corresponding wind speed interval. Select the first number of wind speed intervals from each wind speed interval in descending order of the first occurrence times to determine the key wind speed intervals, that is, the wind speed intervals where temporary skipping rope interruptions are likely to occur. Among them, the historical records include, but are not limited to, the skipping rope height of the historical rope-skippers with temporary skipping rope interruptions and the corresponding wind speed, etc. The historical rope-skippers are rope-skippers with the same information as the target object such as height, weight, and physical health status.

[0087] Further, based on the above historical records, when the historical wind speed is within a single key wind speed interval, obtain the height intervals in which the historical skipping rope height was when the historical rope-skippers had temporary skipping rope interruptions. Count the second occurrence times of each height interval. The larger the second occurrence times, the more likely the rope-skippers are to have temporary skipping rope interruptions when the skipping rope height is within the corresponding height interval under a single key wind speed interval. Then, select the second number of height intervals from each height interval in descending order of the second occurrence times to determine the key height intervals corresponding to the key wind speed interval, that is, the height intervals where temporary skipping rope interruptions are likely to occur. It should be noted that the wind direction when the historical rope-skippers skip rope is the same as the wind direction when the target object skips rope, and both are the wind directions with the greatest impact on skipping rope.

[0088] Further, determine the first weight of each key wind speed interval and determine the second weight of the key height interval corresponding to each key wind speed interval. Among them, the first weight is the ratio of the first occurrence times of each key wind speed interval to the sum of the first occurrence times of all key wind speed intervals, and the second weight is the ratio of the second occurrence times of a single key height interval corresponding to the key wind speed interval to the sum of the second occurrence times of all corresponding key height intervals.

[0089] Finally, based on the first weight, the second weight, the actual rope skipping height and the corresponding actual wind speed, the temporary interruption of rope skipping of the current target object is verified, so that the result of the temporary interruption of rope skipping determined in step S105 is more accurate. A feasible implementation method is: the key height interval where the single actual rope skipping height is located is determined as the target height interval. If the actual wind speed corresponding to the single actual rope skipping height exists in the key wind speed interval, then the corresponding key wind speed interval is determined as the target wind speed interval. Further, if the above-mentioned target height interval exists in each key height interval corresponding to the target wind speed interval, then the corresponding target wind speed interval is determined as the important wind speed interval. Next, the product of the first weight of each important wind speed interval and the second weight of the corresponding target height interval is calculated. The larger the product, when the actual wind speed is in the important wind speed interval, when the actual rope skipping height of the target object is in the corresponding target height interval, the greater the possibility of a temporary interruption of rope skipping. Finally, when the current temporary interruption of rope skipping is the first interruption from the start of rope skipping to the current process, the products are summed up to obtain the sum of the products. If the sum of the products is greater than the preset threshold, it means that the overall possibility of a temporary interruption of rope skipping for the target object from the start of rope skipping to the current time interval is relatively high, and there is a high probability of a temporary interruption of rope skipping at present. In this way, it is verified that the current target object does have a temporary interruption of rope skipping, and at the same time, the accuracy of the rope skipping action classification model in action classification can be verified.

[0090] In one embodiment, when the current temporary interruption of rope skipping is not the first interruption from the start of rope skipping to the current process, the temporary interruption of rope skipping from the start of rope skipping to the current process is determined as a historical interruption, and the product exceeding a preset product threshold is selected from various products to be determined as the product to be analyzed, and if the actual wind speed and the actual rope skipping height corresponding to a single historical interruption are both in the important wind speed range and the target height range corresponding to the product to be analyzed, then it is determined that the record of the historical interruption is correct.

[0091] In one embodiment, if rope skipping counting is performed on a target object during physical exercise, the current rope skipping height and the corresponding current wind speed of the target object are obtained, the key wind speed interval where the current wind speed is located is determined as the reference wind speed interval. If the current rope skipping height exists in the key height interval corresponding to this reference wind speed interval, the corresponding key height interval is determined as the reference height interval. Calculate the first weight product of the first weight of this reference wind speed interval and the second weight of the corresponding reference height interval. If the first weight product is greater than the preset product threshold, it indicates that the target object continues to skip rope at the current rope skipping height and the possibility of a temporary interruption in rope skipping is relatively high. Then calculate the second weight product of the first weight of the reference wind speed interval and the second weights of the corresponding key height intervals, select the smallest second weight product from the second weight products, and when the smallest second weight product is less than the product threshold, determine the key height interval corresponding to the smallest second weight product as the recommended height interval. Finally, send a reminder to the target object to adjust the rope skipping height for this recommended height interval, thereby reducing the risk of temporary interruption in rope skipping, ensuring the continuity of rope skipping exercise to a certain extent, and improving the training effect.

[0092] In another embodiment, if the rope skipping area is indoors in summer, obtain the coverage area of the single air conditioner blowing indoors, and obtain the rope skipping heights of multiple rope skipping personnel in the coverage area. Determine the key wind speed interval where the single air conditioner blowing wind speed is located as the wind speed interval to be concerned. If the key height interval corresponding to the wind speed interval to be concerned contains the rope skipping height, then determine the corresponding key height interval as the height interval to be concerned. Calculate the third weight product of the first weight of the wind speed interval to be concerned and the second weights of the corresponding height intervals to be concerned, and sum up the third weight products to obtain the summation result. If the summation result is greater than the preset threshold, then send a reminder to reduce the air conditioner wind speed, thereby reducing the impact on the rope skipping of the rope skipping personnel.

[0093] The implementation principle of the rope skipping counting method based on action classification in the embodiments of this application is as follows: After obtaining the on-site rope skipping video, extract the video key frame features and 3D pose features related to the target object's rope skipping from the on-site rope skipping video. Through the video key frame features, the key frame detail information of the recorded rope skipping action can be better captured. Through the 3D pose features, the action posture of the target object can be comprehensively reflected. Further, based on these video key frame features and 3D pose features, accurately classify the target object's rope skipping action through the rope skipping action classification model, so as to more accurately determine the current rope skipping state of the target object. If the rope skipping state is starting to skip rope, then start counting the rope skipping of the target object, thereby filtering the interference of non-rope skipping actions on counting to a certain extent, and improving the accuracy of rope skipping counting.

[0094] The following is an embodiment of the device of the present application, which can be used to execute the method embodiment of the present application. For details not disclosed in the device embodiment of the present application, please refer to the method embodiment of the present application.

[0095] Please refer to Figure 2 , which is a schematic structural diagram of a skipping rope counting device based on action classification provided by an embodiment of the present application. The skipping rope counting device based on action classification can be implemented as all or part of the device through software, hardware, or a combination of both. The device includes an information acquisition module 11, a feature determination module 12, an action classification module 13, a skipping rope counting module 14, and an interruption recording module 15.

[0096] The information acquisition module 11 is configured to acquire the on-site skipping rope video of the target object, where the target object is the person who needs to count the skipping rope.

[0097] The feature determination module 12 is configured to determine the video key frame features and 3D pose features related to the target object's skipping rope based on the on-site skipping rope video.

[0098] The action classification module 13 is configured to determine the current skipping rope state of the target object through a preset skipping rope action classification model based on the video key frame features and 3D pose features.

[0099] The skipping rope counting module 14 is configured to count the skipping rope of the target object if the skipping rope state is starting to skip rope.

[0100] The interruption recording module 15 is configured to record one temporary interruption of the skipping rope if the skipping rope state is ending the skipping rope and the skipping rope state is detected as starting to skip rope after the current time.

[0101] Optionally, as Figure 3 shown, the device further includes a model training module 16, which is specifically configured to:

[0102] Acquire skipping rope video sample data corresponding to different skipping rope action categories, where the skipping rope action categories include starting to skip rope, skipping rope, ending the skipping rope, and not skipping rope;

[0103] For a single skipping rope video sample data, perform human detection through a preset object detection algorithm, and after the human detection is completed, perform pose detection through a preset pose detection algorithm to obtain the corresponding pose sequence data;

[0104] Based on the pose sequence data, determine the corresponding 3D pose feature samples, and for a single skipping rope video sample data, determine the corresponding skipping rope action video key frames;

[0105] Based on the key frames of the skipping rope action video, determine the corresponding key frame feature samples, and fuse the 3D pose feature samples and key frame feature samples corresponding to the same skipping rope video sample data to obtain the fused features;

[0106] Based on the fused features corresponding to each skipping rope video sample data, train the preset untrained action classification model to obtain the skipping rope action classification model.

[0107] Optionally, the skipping rope counting module 14 is specifically used for:

[0108] Based on the on-site skipping rope video, count the number of skipping ropes of the target object in real time to obtain the number of skipping ropes, and detect whether there is a rope;

[0109] If so, when the skipping rope state is detected as the end of skipping after the current time, determine the number of skipping ropes in the interval from the current time to the end of skipping as the number of skipping ropes with a rope;

[0110] If not, when the skipping rope state is detected as the end of skipping after the current time, determine the number of skipping ropes in the interval from the current time to the end of skipping as the number of skipping ropes without a rope.

[0111] Optionally, the skipping rope counting module 14 is specifically used for:

[0112] Based on the on-site skipping rope video, determine at least one human core key point corresponding to the target object;

[0113] Perform filtering processing on all human core key points to obtain filtered key points;

[0114] Count the number of up and down reciprocating movements of the filtered key points to obtain the number of skipping ropes corresponding to the target object.

[0115] Optionally, the skipping rope counting module 14 is specifically used for:

[0116] When it is detected that the filtered key points make up and down reciprocating movements, determine the occurrence period of a single up and down reciprocating movement in the on-site skipping rope video as the counting period;

[0117] Judge whether there are a first video frame and a second video frame in the skipping rope video segment corresponding to the counting period. The first video frame is the video frame in which the rope is detected in the upper part of the target object, and the second video frame is the video frame in which the rope is detected in the lower part of the target object;

[0118] If so, determine that there is a rope, if not, determine that there is no rope.

[0119] Optionally, the device further includes a recording verification module 17, which is specifically used for:

[0120] When it is currently determined that the target object has a temporary interruption in rope skipping, at least one actual rope skipping height and the corresponding actual wind speed of the target object from the start of rope skipping to the current are obtained, and the actual wind speed is the wind speed when the rope skipping height of the target object is the actual rope skipping height;

[0121] Obtain the wind speed interval in which the historical wind speed is located when the historical rope skipper has a temporary interruption in rope skipping, count the first occurrence times of each wind speed interval, and select the first number of wind speed intervals from each wind speed interval in the order from largest to smallest of the first occurrence times to determine as the key wind speed intervals;

[0122] Obtain the height interval in which the historical rope skipping height is located when the historical wind speed is in a single key wind speed interval and the historical rope skipper has a temporary interruption in rope skipping, count the second occurrence times of each height interval, and select the second number of height intervals from each height interval in the order from largest to smallest of the second occurrence times, and determine as the key height intervals corresponding to the single key wind speed interval;

[0123] Determine the first weight of each key wind speed interval and the second weight of the key height interval corresponding to each key wind speed interval. The first weight is the ratio of the first occurrence times of each key wind speed interval to the sum of the first occurrence times of all key wind speed intervals, and the second weight is the ratio of the second occurrence times of the single key height interval corresponding to the key wind speed interval to the sum of the second occurrence times of all corresponding key height intervals;

[0124] Based on the first weight, the second weight, the actual rope skipping height and the corresponding actual wind speed, verify the temporary interruption of rope skipping of the current target object.

[0125] Optionally, record the verification module 17, specifically used for:

[0126] Determine the key height interval where the single actual rope skipping height is located as the target height interval. If there is a corresponding actual wind speed in the key wind speed interval, determine the corresponding key wind speed interval as the target wind speed interval;

[0127] If there is a target height interval among the key height intervals corresponding to the target wind speed interval, determine the corresponding target wind speed interval as the important wind speed interval, and calculate the product of the first weight of each important wind speed interval and the second weight of the corresponding target height interval;

[0128] When the current temporary interruption in rope skipping is the first interruption in the process from the start of rope skipping to the current, sum up the products to obtain the sum of the products. If the sum of the products exceeds the preset threshold, verify the temporary interruption of rope skipping of the current target object.

[0129] It should be noted that when the skipping rope counting device based on action classification provided in the above embodiments executes the skipping rope counting method based on action classification, only the division of the above functional modules is used for illustration. In practical applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the skipping rope counting device based on action classification provided in the above embodiments and the embodiment of the skipping rope counting method based on action classification belong to the same concept. The implementation process is detailed in the method embodiment and will not be repeated here.

[0130] An embodiment of the present application also discloses a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the skipping rope counting method based on action classification in the above embodiments is adopted.

[0131] Among them, the computer program can be stored in a computer-readable medium. The computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some middleware form, etc. The computer-readable medium includes any entity or device that can carry the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the computer-readable medium includes but is not limited to the above components.

[0132] Among them, through this computer-readable storage medium, the skipping rope counting method based on action classification in the above embodiments is stored in the computer-readable storage medium, and is loaded and executed on the processor to facilitate the storage and application of the above method.

[0133] An embodiment of the present application also discloses an electronic device. A computer program is stored in the computer-readable storage medium. When the computer program is loaded and executed by the processor, the skipping rope counting method based on action classification described above is adopted.

[0134] Among them, the electronic device can be a desktop computer, a laptop computer or a cloud server and other electronic devices. The electronic device includes but is not limited to a processor and a memory. For example, the electronic device may also include input / output devices, network access devices, and a bus, etc.

[0135] Among them, the processor may adopt a central processing unit (CPU). Of course, according to the actual usage, other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. may also be adopted. The general-purpose processor may adopt a microprocessor or any conventional processor, etc. The present application does not make any restrictions on this.

[0136] Among them, the memory may be an internal storage unit of the electronic device. For example, the hard disk or memory of the electronic device, or it may also be an external storage device of the electronic device. For example, a plug-in hard disk, a smart media card (SMC), a secure digital card (SD), or a flash card (FC) equipped on the electronic device, etc. And the memory may also be a combination of the internal storage unit and the external storage device of the electronic device. The memory is used to store computer programs and other programs and data required by the electronic device. The memory may also be used to temporarily store the data that has been output or will be output. The present application does not make any restrictions on this.

[0137] Among them, through this electronic device, a skipping rope counting method based on action classification in the above embodiment is stored in the memory of the electronic device, and is loaded and executed on the processor of the electronic device for convenient use.

[0138] The above are only exemplary embodiments of the present disclosure and should not be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure still fall within the scope covered by the present disclosure. The present application aims to cover any variations, uses or adaptive changes of the present disclosure. These variations, uses or adaptive changes follow the general principles of the present disclosure and include common general knowledge or conventional technical means in the technical field not recorded in the present disclosure. The description and the embodiments are only regarded as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A skipping rope counting method based on action classification, characterized in that, The method includes: Obtaining a live skipping rope video of a target object, where the target object is a person who needs to have their skipping rope count; Based on the live skipping rope video, determining video key frame features and 3D pose features related to the target object's skipping rope; Based on the video key frame features and the 3D pose features, determining the current skipping rope state of the target object through a preset skipping rope action classification model; If the skipping rope state is starting to skip, then counting the number of times the target object skips; If the skipping rope state is the end of skipping, then when it is detected that the skipping rope state is starting to skip after the current time, recording that the skipping rope is temporarily interrupted once; When it is currently determined that the target object has a temporary interruption in skipping, obtaining at least one actual skipping height of the target object from the start of skipping to the current time and the corresponding actual wind speed, where the actual wind speed is the wind speed when the target object's skipping height is the actual skipping height; Obtaining the wind speed intervals in which the historical wind speed was located when historical skipping rope users had temporary interruptions in skipping, counting the first occurrence times of each of the wind speed intervals, and selecting, in descending order of the first occurrence times, the first number of wind speed intervals from each of the wind speed intervals to be determined as key wind speed intervals; Obtaining the height intervals in which the historical skipping heights were located when historical skipping rope users had temporary interruptions in skipping when the historical wind speed was within a single key wind speed interval, counting the second occurrence times of each of the height intervals, selecting, in descending order of the second occurrence times, the second number of height intervals from each of the height intervals, and determining them as the key height intervals corresponding to the single key wind speed interval; Determining the first weight of each key wind speed interval and determining the second weight of the key height interval corresponding to each key wind speed interval, where the first weight is the ratio of the first occurrence time of each key wind speed interval to the sum of the first occurrence times of all key wind speed intervals, and the second weight is the ratio of the second occurrence time of the single key height interval corresponding to the key wind speed interval to the sum of the second occurrence times of all corresponding key height intervals; Verify the temporary interruption of rope skipping of the current target object based on the first weight, the second weight, the actual rope skipping height, and the corresponding actual wind speed; wherein, if counting the rope skipping of the target object during physical exercise, obtain the current rope skipping height of the target object and the corresponding current wind speed, determine the key wind speed interval where the current wind speed is located as the reference wind speed interval. If the current rope skipping height exists in the key height interval corresponding to this reference wind speed interval, determine the corresponding key height interval as the reference height interval, calculate the first weight product of the first weight of this reference wind speed interval and the second weight of the corresponding reference height interval. If the first weight product is greater than the preset product threshold, then calculate the second weight product of the first weight of the reference wind speed interval and the second weight of each corresponding key height interval, select the minimum second weight product from each second weight product, and when the minimum second weight product is less than the product threshold, determine the key height interval corresponding to the minimum second weight product as the recommended height interval, and send a rope skipping height adjustment reminder to the target object for this recommended height interval.

2. The skipping rope counting method based on action classification according to claim 1, wherein Before obtaining the on-site rope skipping video of the target object, it further includes: Obtain rope skipping video sample data corresponding to different rope skipping action categories, where the rope skipping action categories include starting rope skipping, rope skipping, ending rope skipping, and not rope skipping; For a single piece of the rope skipping video sample data, perform human detection through a preset target detection algorithm, and after the human detection is completed, perform pose detection through a preset pose detection algorithm to obtain the corresponding pose sequence data; Based on the pose sequence data, determine the corresponding 3D pose feature sample, and for a single piece of the rope skipping video sample data, determine the corresponding key frame of the rope skipping action video; Based on the key frame of the rope skipping action video, determine the corresponding key frame feature sample, and fuse the 3D pose feature sample and the key frame feature sample corresponding to the same piece of the rope skipping video sample data to obtain the fused feature; Based on the fused features corresponding to each piece of the rope skipping video sample data, train a preset untrained action classification model to obtain a rope skipping action classification model.

3. The skipping rope counting method based on action classification according to claim 1, wherein, The counting of the rope skipping of the target object specifically includes: Based on the on-site rope skipping video, count the rope skipping of the target object in real time to obtain the number of rope skips, and detect whether there is a rope; If so, when the rope skipping state is detected as ending rope skipping after the current time, determine the number of rope skips in the interval from the current time to the end of rope skipping as the number of rope skips with a rope; If not, when the rope skipping state is detected as ending rope skipping after the current time, determine the number of rope skips in the interval from the current time to the end of rope skipping as the number of rope skips without a rope.

4. The skipping rope counting method based on action classification according to claim 3, characterized in that, The counting of the rope skipping of the target object in real time based on the on-site rope skipping video to obtain the number of rope skips specifically includes: Based on the on-site rope skipping video, determine at least one human body core key point corresponding to the target object; Perform filtering processing on all the human body core key points to obtain filtered key points; Count the number of up-and-down reciprocating motions of the filtered key points to obtain the number of rope skipping corresponding to the target object.

5. The skipping rope counting method based on action classification according to claim 4, wherein The detection of whether there is a rope specifically includes: When it is detected that the filtered key points make up-and-down reciprocating motions, determine the occurrence period of a single up-and-down reciprocating motion in the on-site rope skipping video as the counting period; Judge whether there are a first video frame and a second video frame in the rope skipping video segment corresponding to the counting period. The first video frame is a video frame in which a rope is detected in the upper part of the target object, and the second video frame is a video frame in which a rope is detected in the lower part of the target object; If so, determine that there is a rope; if not, determine that there is no rope.

6. The skipping rope counting method based on action classification according to claim 1, wherein, The verification of the temporary interruption of rope skipping by the current target object based on the first weight, the second weight, the actual rope skipping height, and the corresponding actual wind speed specifically includes: Determine the key height interval where the single actual rope skipping height is located as the target height interval. If there is a corresponding actual wind speed in the key wind speed interval, determine the corresponding key wind speed interval as the target wind speed interval; If the target height interval exists in the key height intervals corresponding to the target wind speed interval, determine the corresponding target wind speed interval as the important wind speed interval, and calculate the product of the first weight of each important wind speed interval and the second weight of the corresponding target height interval; When the current temporary interruption of rope skipping is the first interruption from the start of rope skipping to the current process, sum up the products to obtain the sum of the products. If the sum of the products exceeds the preset threshold, verify that the current target object has a temporary interruption of rope skipping.

7. A skipping rope counting device based on action classification, which is used to implement the skipping rope counting method based on action classification described in any one of claims 1 to 6, and is characterized in that, It includes: An information acquisition module (11) for acquiring an on-site rope skipping video of a target object, where the target object is a person who needs to count rope skipping; A feature determination module (12) for determining video key frame features and 3D pose features related to rope skipping of the target object based on the on-site rope skipping video; An action classification module (13) for determining the current rope skipping state of the target object through a preset rope skipping action classification model based on the video key frame features and the 3D pose features; A rope skipping counting module (14) for counting the rope skipping of the target object if the rope skipping state is the start of rope skipping; An interruption recording module (15) for recording a temporary interruption of rope skipping once when it is detected that the rope skipping state is the start of rope skipping after the current time if the rope skipping state is the end of rope skipping.

8. A computer-readable storage medium storing a computer program therein, characterized in that, When the computer program is loaded and executed by a processor, the method described in any one of claims 1-6 is adopted.

9. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that When the processor loads and executes the computer program, the method described in any one of claims 1-6 is adopted.

Images