A method for automatically determining the process of an athlete's long jump

Abstract

The application discloses a method for automatically judging the process of long jump of athletes, and comprises the following steps: marking a preparation area; acquiring a video frame of a long jump project sports area shot in real time, identifying and outputting a human body 3D model of all people in the sports area in each frame according to a human body 3D model generation algorithm; training an action classification model, including a take-off instant, a highest jump point and a landing instant; after detecting an athlete who raises hands, identifying and recording personnel in the preparation area, extracting a human body 3D model of all personnel in the preparation area in the video frame, searching for the athlete who raises hands among all personnel in the preparation area as a tracking object, and recording a video frame of tracking loss; if the number of the video frame of tracking loss exceeds a first threshold value, tracking failure is determined; if tracking is successful, the process of take-off, the highest jump point and landing of the tracked athlete is judged according to the position of the preparation area and in combination with the classification model. The athlete motion state recognition of the scheme provided by the application is more accurate.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a method for automatically judging the long jump process of an athlete, a computer storage medium and equipment. BACKGROUND

[0002] The intelligent detection of sports and competitive events is becoming increasingly mature. At present, the common visual monitoring technology for long jump, high jump and other events is used to obtain motion data through a camera and output the state of an athlete through human body posture detection.

[0003] For example, a technology is used to infer and predict the take-off frame and the landing frame through a time sequence action positioning model. The take-off state is located at the moment when the center of gravity is highest between the take-off and landing. According to the human body skeleton key point information in the take-off, take-off and landing states, it is determined whether the action is standard. For the take-off stage, it is determined whether the action is standard by analyzing whether the take-off angle meets the standard index. For the take-off stage, it is determined whether the action is standard by analyzing whether the take-off height and folding angle meet the requirements. For the landing stage, it is determined whether the action is standard by analyzing the landing angle and the center of gravity force line. However, the above technology often ignores the tracking state of the athlete in the entire process from entering to leaving the field, and the phenomenon of athlete capture failure may occur. Further, it is more difficult to recognize the entire process of an athlete in a complex environment. It is necessary to ensure that only the athlete is present in the sports field. In the presence of onlookers, it is easy to make a wrong judgment. SUMMARY

[0004] The present application provides a method for automatically judging the long jump process of an athlete.

[0005] The present application is solved by the following technical scheme: a method for automatically judging the long jump process of an athlete, characterized in that it comprises the following steps:

[0006] Labeling the preparation area: obtaining the collected image containing the preparation area and the motion area, labeling the four corner point coordinates of the preparation area in the image, and calculating the side line of the preparation area according to the corner point coordinates;

[0007] Obtaining the video frame of the long jump preparation area and the motion area shot in real time, identifying and outputting the human body 3D model of all people in the preparation area and the motion area in each frame according to the human body 3D model generation algorithm;

[0008] Training the action classification model, which includes the take-off moment, the highest point of jumping and the landing moment;

[0009] After detecting the athlete raising hands, personnel in the preparation area are identified and recorded, a human body 3D model of all personnel located in the preparation area in the video frame is extracted, an athlete raising hands in all personnel located in the preparation area is searched as a tracking object, and video frames of tracking loss and tracking success are recorded; if the number of video frames of tracking loss exceeds a first threshold value, it is determined that tracking fails, the motion state of the corresponding tracking object is emptied, and the process of athlete raising hands detection and tracking of the next athlete is started;

[0010] If tracking succeeds, the process of take-off, highest point of jumping and landing of the tracked athlete is judged according to the position of the preparation area and the human body 3D model data in combination with a classification model.

[0011] Specifically, the method for calculating the sidelines of the preparation area according to the corner point coordinates comprises the following steps: marking four corner points of the preparation area, and calculating the upper sideline, the right sideline, the lower sideline and the left sideline of the preparation area according to the four corner points.

[0012] Preferably, the method for identifying and recording personnel in the preparation area comprises the following steps: for all human body 3D models detected in each frame of image after detecting the athlete raising hands, the foot region corresponding to each human body 3D model is extracted,

[0013] The horizontal coordinates of all points in the foot region are brought into the upper sideline and the lower sideline, and the vertical coordinates based on the upper sideline and the vertical coordinates based on the lower sideline are calculated;

[0014] The vertical coordinates of all points in the foot region are brought into the left sideline and the right sideline, and the horizontal coordinates based on the left sideline and the horizontal coordinates based on the right sideline are calculated;

[0015] If the horizontal coordinates of all points in the foot region are between the horizontal coordinates based on the left sideline and the horizontal coordinates based on the right sideline, and the vertical coordinates of all points in the foot region are between the vertical coordinates based on the upper sideline and the vertical coordinates based on the lower sideline, it is determined that the personnel corresponding to the corresponding human body 3D model are located in the preparation area, the corresponding human body 3D model is recorded, otherwise, the corresponding human body 3D model is discarded.

[0016] Specifically, the method for extracting a human body 3D model of all personnel located in the preparation area in the video frame and searching for an athlete raising hands in all personnel located in the preparation area as a tracking object comprises the following steps:

[0017] All points in each group of human body 3D models are taken, the Euclidean distances between all points and all corresponding points of the human body 3D model of the athlete raising hands saved are calculated, and the average value of the Euclidean distances of all corresponding points is calculated;

[0018] The human body corresponding to the human body 3D model with the minimum average value is taken as a candidate tracking object;

[0019] If the smallest average value is less than or equal to the second threshold value, the candidate tracking object is determined as a tracking object, and the corresponding video frame is recorded as tracking success;

[0020] If the smallest average value is greater than the second threshold value, the corresponding video frame is determined as tracking loss, and is recorded as tracking failure.

[0021] Specifically, data of video frames of tracking loss and tracking success are recorded; if the number of video frames of tracking loss exceeds the first threshold value, a method for determining tracking failure comprises:

[0022] The video frames of tracking loss and tracking success are stored in a first list, and the storage content further comprises a video frame time;

[0023] When processing a current video frame, a proportion of tracking loss in a preset time in the past is calculated according to the data of the first list; if the proportion of video frames of tracking failure exceeds the first threshold value, tracking failure is determined.

[0024] Preferably, a method for determining that a tracked athlete has taken off comprises:

[0025] If the preparation area is on the left side of the video frame image, the longitudinal coordinates of all points in the region above the knee of the athlete are brought into a right straight line of the preparation area, to obtain first transverse coordinates of all points based on the right straight line; if the proportion of the transverse coordinates of all points in the region above the knee of the athlete that are greater than the first transverse coordinates is greater than a third threshold value, it is determined that the athlete has taken off;

[0026] If the preparation area is on the right side of the video frame image, the longitudinal coordinates of all points in the region above the knee of the athlete are brought into a left straight line of the preparation area, to obtain second transverse coordinates of all points based on the left straight line; if the proportion of the transverse coordinates of all points in the region above the knee of the athlete that are less than the second transverse coordinates is greater than a third threshold value, it is determined that the athlete has taken off.

[0027] The third threshold value is greater than or equal to one-half.

[0028] Preferably, a method for determining that a tracked athlete has reached a highest point comprises:

[0029] Each video frame image after the athlete takes off is input into a 3d model generation algorithm to generate a corresponding human body 3d model, and the generated human body 3d model is input into a trained classification model;

[0030] If the classification model determines that the athlete has not reached the highest point in the current video frame, the processing of the current frame is ended, and the processing of the next video frame is continued;

[0031] If the classification model determines that the athlete has reached the highest point in the current video frame, a subsequent other motion state determination process is entered.

[0032] Preferably, the method for judging the landing of the tracked athlete comprises:

[0033] After judging that the athlete has passed the highest point, the vertical coordinates of all points in the athlete's foot region in the current frame are obtained, the vertical coordinates of all points in the foot region in the previous frame are obtained, and the Euclidean distance of all corresponding points in the two frames and the distance average value are calculated.

[0034] If the distance average value is less than the fourth threshold value and greater than half of the fourth threshold value, it is judged that the athlete in the current frame has been in a landing state, and if not, it is judged that the athlete is in a falling state.

[0035] The video frames and the athlete state that have been judged to be in a landing state are recorded in a second list, and if the proportion of video frames judged to be in a landing state in a plurality of consecutive video frames exceeds a preset proportion, it is determined that the athlete is in a landing state, otherwise, it is judged that the athlete is not in a landing state.

[0036] Preferably, the method for judging the landing of the tracked athlete further comprises: inputting the video frames in the second list into a human body 3D model generation algorithm to generate corresponding human body 3D models, inputting the human body 3D models into a classification model, and if the classification model judges that the state is at the moment of landing, it is considered that the athlete in the video frames starting from the corresponding video frames and after that is in a landing state, otherwise, it is considered that the corresponding video frames are not in a landing state.

[0037] Preferably, the method for judging the take-off of the tracked athlete further comprises:

[0038] The video frames that have been identified as the take-off of the athlete are input into a human body 3D model generation algorithm to generate corresponding human body 3D models, the generated human body 3D models are input into a trained classification model, and if the classification model judges that the current video frame is at the moment of take-off, it is considered that the athlete has indeed taken off at the current video frame. Otherwise, it is considered that the athlete has not taken off at the current video frame.

[0039] The beneficial effects of the present application are:

[0040] Compared with the traditional long jump automatic recognition algorithm based on human body key point detection algorithm, the present application is more accurate and stable, because the human body 3D model can more accurately fit the whole human body than several human body key points, and the human body key point algorithm may be inaccurate and unstable due to missing key points caused by occlusion and other reasons, while the human body 3D model is less affected by these factors.

[0041] In addition, the present application further trains a classification algorithm based on the 3D human body model, further judges different stages of the whole process of long jump through the classification algorithm, and further improves the stability and accuracy of the algorithm through the combination of the rule algorithm based on the 3D human body model and the classification algorithm. DETAILED DESCRIPTION

[0042] The application will be further described in detail in connection with the following examples, which are intended to explain the application but not to limit the application to the following examples.

[0043] The application discloses a method for analyzing the whole process of long jump of a long jump athlete. The method can automatically determine whether the athlete leaves the preparation area, starts to jump, starts to fall, has landed, and accurately determine the key moments, so as to analyze the long jump process of the athlete, automatically determine the take-off speed, long jump height and long jump distance of the athlete, and provide sports suggestions for more targeted training of the athlete and facilitate recording of the whole process of the athlete.

[0044] The scheme needs some preparation work. Generally, each long jump item has a preparation area before starting the movement. The preparation area is a rectangular area with a length of about 0.5 meters and a width of about 0.5 meters. The long jump athlete stands in the area to prepare and jump. The movement area is a rectangular area with a length of about 3 meters and a width of about 1 meter. The above parameters are only used as a reference for the field. A camera is erected on the sports field and aimed at the preparation area and the movement area of the long jump item to shoot the whole long jump process of the athlete in the field.

[0045] An automatic method for determining the long jump process of an athlete, comprising the following steps:

[0046] Labeling the preparation area of the long jump item: obtaining the collected images containing the preparation area and the movement area, labeling the 4 corner point coordinates of the preparation area in the image, and calculating the side line of the preparation area according to the corner point coordinates;

[0047] Obtaining the video frames of the long jump item preparation area and the movement area shot in real time, identifying and outputting the human body 3D model of all people in the movement area in each frame according to the human body 3D model generation algorithm;

[0048] Training the action classification model, and the action classification includes the take-off moment, the highest jumping point and the landing moment;

[0049] After detecting the athlete who raises his hand, the personnel in the preparation area are identified and recorded, the human body 3D model of all personnel located in the preparation area in the video frames is extracted, the athlete who raises his hand among all personnel located in the preparation area is searched as a tracking object, and the video frames of tracking loss are recorded. If the number of video frames of tracking loss exceeds the first threshold value, it is determined that the tracking fails, the movement state of the corresponding tracking object is emptied, and the hand raising detection and tracking process of the next athlete is started;

[0050] If the tracking is successful, the preparation area position, the human body 3D model data and the classification model are combined to determine the take-off, the highest jumping point and the landing process of the tracked athlete.

[0051] Specifically, according to the corner point coordinate calculation method of the edge line of the preparation area, four corner points of the preparation area are marked, and the upper edge line, the right edge line, the lower edge line and the left edge line of the preparation area are calculated according to the four corner points.

[0052] Due to the randomness of the camera mounting angle, the preparation area may become an arbitrary quadrilateral on the camera shooting screen, so it is necessary to manually mark the area in advance for the judgment of multi-person interference. Of course, if allowed, an automatic marking algorithm for the preparation area can be developed to realize automatic marking.

[0053] A rectangular coordinate system is established with the upper left corner of the camera shooting screen as the origin, the horizontal right direction as the X-axis, and the vertical downward direction as the Y-axis. After the camera is erected, a camera shooting screen is extracted, and the 4 corner point coordinates of the preparation area quadrilateral in the screen are obtained by manual marking.

[0054] Suppose that the 4 corner point coordinates obtained in clockwise order from the left upper corner of the preparation area quadrilateral are (x1, y1), (x2, y2), (x3, y3) and (x4, y4). The marking number and order are not strictly required to be marked in clockwise order, and the purpose is to obtain the equations of the four straight lines.

[0055] The four straight lines calculated by the four corner points marked in the last step are the four edge lines of the preparation area.

[0056] The upper edge line equation of the preparation area is calculated by the two points (x1, y1) and (x2, y2), denoted as line up .

[0057] The right edge line equation of the preparation area is calculated by the two points (x2, y2) and (x3, y3), denoted as line right .

[0058] The lower edge line equation of the preparation area is calculated by the two points (x3, y3) and (x4, y4), denoted as line down .

[0059] The left edge line equation of the preparation area is calculated by the two points (x4, y4) and (x1, y1), denoted as line left .

[0060] The above equations are the equations of the line calculated by two points in mathematics, and are not described in detail.

[0061] Regarding obtaining video frames of the preparation area and the motion area of the long jump event in real time, a human body 3D model generation algorithm is used to identify and output the human body 3D model of all persons in the preparation area and the motion area in each frame, wherein the human body 3D model generation algorithm can use an existing algorithm, and more preferably, a 3D model generation algorithm is separately trained for the long jump motion scene by collecting and labeling human body 3D data, such as HMR, Spine, and Vide algorithms.

[0062] Regarding training of the action classification model, the action classification includes the method of takeoff instant, jump peak, and landing instant, and specifically includes the following steps.

[0063] A large number of pictures of the process of the long jump of the athletes are collected by using the camera, and the pictures of the takeoff instant, the jump peak, and the landing instant are manually labeled to divide into three categories.

[0064] A human body 3D model generation algorithm is used to process each picture to construct the human body 3D model corresponding to each picture, and a classification model of an artificial neural network is trained by using the constructed human body 3D model and the corresponding category.

[0065] The input of the classification model is the picture of the long jump, the human body 3D model corresponding to the picture is generated by the human body 3D model generation algorithm, and then the human body 3D model is input into the classification model based on the 3D human body model, and the category to which the athlete in the picture belongs in the motion process, i.e., the three categories of takeoff instant, jump peak, and landing instant, is output.

[0066] In a preferred scheme, the method for identifying and recording the personnel in the preparation area includes the following steps.

[0067] The horizontal coordinates of all points in the foot region are brought into the upper edge line and the lower edge line to calculate the vertical coordinates based on the upper edge line and the vertical coordinates based on the lower edge line.

[0068] The vertical coordinates of all points in the foot region are brought into the left edge line and the right edge line to calculate the horizontal coordinates based on the left edge line and the horizontal coordinates based on the right edge line.

[0069] If the horizontal coordinates of all points in the foot region are between the horizontal coordinates based on the left edge line and the horizontal coordinates based on the right edge line, and the vertical coordinates of all points in the foot region are between the vertical coordinates based on the upper edge line and the vertical coordinates based on the lower edge line, it is determined that the personnel corresponding to the human body 3D model is located in the preparation area, the corresponding human body 3D model is recorded, otherwise, the corresponding human body 3D model is discarded.

[0070] Specifically, the horizontal coordinates x of all points in the foot region are brought into the line f and the line up .down two straight lines, the intersection point y coordinate of the corresponding straight line passing through all points of the foot area and parallel to the Y axis and line up and line down up and y down ;

[0071] substitute the longitudinal coordinate y f of all points of the foot area into line left and line right two straight lines, the intersection point x coordinate of the corresponding straight line passing through all points of the foot area and parallel to the X axis and line up and line down left and x right .

[0072] If the horizontal coordinate x f of all points of the foot in the video frame satisfies the inequality x left <x f <x right , and the longitudinal coordinate y f of all points of the foot satisfies the inequality y up <y f <y down .

[0073] The horizontal coordinate x f and the longitudinal coordinate y f correspond to the person located in the preparation area, and the corresponding human body 3D model is recorded. If the human body 3D model corresponding to the person is not located in the preparation area through the above judgment, the human body 3D model is discarded.

[0074] According to the above method, the human body 3D model of all personnel located in the preparation area is calculated, and then the human body 3D model of all personnel located in the preparation area in the video frame is extracted, and the hand-raising athlete in all personnel located in the preparation area is searched as a tracking object, which specifically includes:

[0075] Take all points in each group of human body 3D models, calculate the Euclidean distance between all points and all corresponding points of the human body 3D model of the saved hand-raising athlete, and calculate the average value of the Euclidean distance of all corresponding points;

[0076] Take the human body corresponding to the human body 3D model with the smallest average value as the tracking object.

[0077] As a preferred scheme, the method of taking the human body corresponding to the human body 3D model with the smallest average value as the tracking object includes: taking the human body corresponding to the human body 3D model with the smallest average value as the candidate tracking object;

[0078] ​​If the minimum average value is less than or equal to the second threshold value, the candidate tracking object is determined as the tracking object, and the corresponding video frame is recorded as tracking success;

[0079] If the minimum average value is greater than the second threshold value, the corresponding video frame is determined as tracking loss, and is recorded as tracking failure.

[0080] The scheme can effectively prevent the problem of tracking error caused by the fact that although the 3D human model with the minimum Euclidean distance is calculated, the target athlete is not captured or leaves the preparation area.

[0081] As a preferred scheme, data of video frames of tracking loss and tracking success are recorded; if the number of video frames of tracking loss exceeds the first threshold value, the method for determining tracking failure comprises:

[0082] The video frames of tracking loss and tracking success are stored in a first list, and the storage content further comprises a video frame time;

[0083] When the current video frame is processed, the proportion of tracking loss in a preset time in the past is calculated according to the data of the first list; if the proportion of the number of video frames of tracking loss exceeds the first threshold value, tracking failure is determined. That is, the athlete currently tracked leaves the preparation area, the movement state is emptied, and the detection process of hand raising and identification of the next athlete is automatically started.

[0084] If the athlete does not leave the preparation area, subsequent judgment is continued.

[0085] The scheme can effectively prevent the scene of individual video frame tracking loss, and solves the special scene that the athlete does not really jump but leaves due to various reasons when tracking actual movement.

[0086] As a reference, the first threshold value can be selected as 80%, and the preset time in the past can be selected as two seconds, three seconds or the like.

[0087] After the tracking object is determined, the athlete performing the long jump is tracked in real time, and only the athlete performing the actual movement is processed at all times. Interference of other people is avoided, so that the accuracy of the algorithm is improved. The problem that other onlookers are misjudged as athletes and are interfered by the onlookers around, resulting in algorithm judgment error, is effectively solved. Therefore, only the athlete is required in the picture collected by the camera.

[0088] The method for determining that the tracked athlete jumps comprises:

[0089] If the preparation area is on the left side of the video frame image, the vertical coordinates of all points in the region above the knee of the athlete are brought into the right straight line of the preparation area, the first horizontal coordinates of all points based on the right straight line are obtained, and if the number of horizontal coordinates of all points in the region above the knee of the athlete that are greater than the first horizontal coordinates accounts for more than the third threshold, it is determined that the athlete has taken off;

[0090] If the preparation area is on the right side of the video frame image, the vertical coordinates of all points in the region above the knee of the athlete are brought into the left straight line of the preparation area, the second horizontal coordinates of all points based on the left straight line are obtained, and if the number of horizontal coordinates of all points in the region above the knee of the athlete that are less than the second horizontal coordinates accounts for more than the third threshold, it is determined that the athlete has taken off;

[0091] The third threshold is greater than or equal to one-half. If the preparation area is on the left side of the picture, line right This straight line is the starting line, the vertical coordinates y of all points in the region above the knee of the athlete are brought into line right , the intersection x-coordinate of the corresponding straight line passing through all points in the region above the knee of the athlete and parallel to the X-axis and line right is obtained, and if more than half of the points in the region above the knee have x-coordinates greater than the corresponding calculated x-coordinate, it is determined that the athlete has taken off.

[0092] Similarly, if the preparation area is on the right side of the picture, line left This straight line is the starting line, the vertical coordinates y of all points in the region above the knee of the athlete are brought into line left , the intersection x-coordinate of the corresponding straight line passing through all points in the region above the knee of the athlete and parallel to the X-axis and line left is obtained, and if more than half of the points in the region above the knee have x-coordinates less than the corresponding calculated x-coordinate, it is determined that the athlete has taken off. Thus, it can be basically determined whether the athlete has taken off, and a more accurate preferred solution is proposed below. It should be noted that the comparison of "greater than" and "less than" in the present solution is based on the comparison rule of establishing a rectangular coordinate system with the upper left corner of the picture taken by the camera as the origin, the horizontal right direction as the X-axis, and the vertical downward direction as the Y-axis. If the coordinate system is changed, the corresponding comparison rule needs to be adjusted accordingly.

[0093] The video frame that has been identified as the athlete taking off is input into a human body 3D model generation algorithm to generate a corresponding human body 3D model, and the generated human body 3D model is input into a trained classification model. If the classification model determines that the current video frame is the instant of taking off, it is considered that the current video frame indeed indicates that the athlete has taken off. Otherwise, it is considered that the current video frame indicates that the athlete has not taken off.

[0094] If it is judged that the athlete has not taken off, then the processing of the current frame is ended, and the processing of the next frame is continued.

[0095] If the athlete has taken off, then subsequent judgment is continued, such as the judgment of the highest point of the jump of the subsequent athlete, and of course in other application scenarios, it can also be other state judgment.

[0096] The algorithm directly uses whether the points above the knee of the 3D human body exceed the take-off line to judge the take-off, the rule is very simple, and compared with the existing method of using complex angle calculation to judge whether the athlete has taken off, the present scheme is more accurate and less prone to errors.

[0097] Further, a classification model based on a 3D model is preferably used to further classify whether the athlete has taken off, which can be further verified to ensure that the athlete has indeed taken off, and the accuracy is further improved.

[0098] The method for judging the highest point of the jump of the tracked athlete includes:

[0099] The video frame image of each frame after the athlete takes off is input into a 3D model generation algorithm to generate a corresponding human body 3D model, and the generated human body 3D model is input into a trained classification model.

[0100] If the classification model judges that the athlete in the current video frame has not reached the highest point, the processing of the current frame is ended, and the processing of the next video frame is continued.

[0101] If the classification model judges that the athlete in the current video frame has reached the highest point, then the subsequent other motion state judgment process is entered.

[0102] Compared with the algorithm based on human body key points in the prior art, the algorithm based on human body key points is prone to loss of key points, leading to judgment errors. The present application uses a classification algorithm based on a human body 3D model to judge whether the highest point of the long jump is reached, and the accuracy is higher.

[0103] When it is judged that the athlete has passed the highest point, it is further judged whether the athlete has landed. The method for judging the landing of the tracked athlete includes:

[0104] After judging that the athlete has passed the highest point, the longitudinal coordinates of all points of the athlete's foot region in the current frame are obtained, the longitudinal coordinates of all points of the foot region in the previous frame are obtained, and the Euclidean distance and the distance average value of all corresponding points in the two frames are calculated.

[0105] If the distance average value is less than the fourth threshold value and greater than half of the fourth threshold value, it is judged that the athlete in the current frame has landed, and if not, it is judged that the athlete is in a descending state.

[0106] The video frame determined to be in the landing state and the athlete state are recorded in a second list. If the proportion of video frames determined to be in the landing state in a plurality of continuous video frames exceeds a preset proportion, it is determined that the athlete is in the landing state, otherwise, it is determined that the athlete is not in the landing state.

[0107] By way of reference, the preset proportion can be 80%. Based on the above method, it can be determined whether the athlete lands. A more accurate preferred solution is proposed below.

[0108] As a preferred solution, the video frame in the second list is input into a human body 3D model generation algorithm to generate a corresponding human body 3D model. The human body 3D model is input into a classification model. If the classification model determines the state at the landing moment, it is considered that the athlete is in the landing state based on the video frame starting from the corresponding video frame and the video frame after the corresponding video frame, otherwise, it is considered that the corresponding video frame is not in the landing state.

[0109] Further, the method for determining whether the tracked athlete lands can also be processed only by using the classification model.

[0110] If it is determined that the athlete has not landed, the processing of the current frame is ended, and the processing of the next frame is continued.

[0111] If the athlete has landed, the algorithm is ended, and the automatic determination of the next person's jump process is performed.

[0112] Compared with the existing method of using only one foot key point to determine whether the athlete lands, the method is prone to errors due to key point detection loss. The average Euclidean distance of all points in the foot region of the human body 3D is used in the present application, which will not cause errors due to key point detection loss.

[0113] In addition, the classification model based on the 3D model is further used to further classify whether the athlete lands, which can ensure that the athlete really lands through further verification, and the accuracy is further improved.

[0114] In addition, the methods in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0115] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The software product is stored in a storage medium, and includes a number of instructions to make a device (which can be a single-chip microcomputer, a chip, etc.) or a processor execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0116] The computer device can be a desktop computer, a notebook computer, a palm computer, a cloud server and the like. The computer device can include, but is not limited to, a memory, a processor, and the like. It can be understood by those skilled in the art that the embodiments are only examples of the computer device, and do not constitute a limitation on the computer device, and can include more or fewer components, or combine certain components, or different components, for example, the computer device can also include an input device, a network access device, a bus and the like.

[0117] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0118] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any change or replacement within the technical scope disclosed in the present application should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method of automatically determining the progress of an athlete in a long jump, characterized by, The method comprises the following steps: The preparation area is labeled: the four corner points of the preparation area in the image are labeled, and the edge lines of the preparation area are calculated according to the corner point coordinates; Real-time video frames of the preparation area and the movement area of the long jump project are obtained, and the human body 3D model of all the people in each frame in the preparation area and the movement area is recognized and output according to the human body 3D model generation algorithm; The action classification model is trained, and the action classification includes the take-off instant, the highest point of the jump, and the landing instant; After detecting the athlete who raises his hand, the personnel in the preparation area are recognized and recorded, the human body 3D model of all the personnel in the preparation area in the video frames is extracted, the athlete who raises his hand among all the personnel in the preparation area is searched as a tracking object, and the video frames of tracking loss and tracking success are recorded; if the number of video frames of tracking loss exceeds the first threshold value, it is determined that the tracking fails, the movement state of the corresponding tracking object is cleared, and the hand raising detection and tracking process of the next athlete is started; If the tracking is successful, the process of the tracked athlete taking off, jumping to the highest point, and landing is determined according to the position of the preparation area and the human body 3D model data, and in combination with the classification model; The method for calculating the edge lines of the preparation area according to the corner point coordinates comprises the following steps: The four corner points of the preparation area are labeled, and the upper edge line, the right edge line, the lower edge line, and the left edge line of the preparation area are calculated according to the four corner points; The method for recognizing and recording the personnel in the preparation area comprises the following steps: For all the human body 3D models detected in each frame of image after detecting the athlete who raises his hand, the foot region corresponding to each human body 3D model is extracted, The horizontal coordinates of all the points in the foot region are brought into the upper edge line and the lower edge line to calculate the vertical coordinates based on the upper edge line and the vertical coordinates based on the lower edge line; The vertical coordinates of all the points in the foot region are brought into the left edge line and the right edge line to calculate the horizontal coordinates based on the left edge line and the horizontal coordinates based on the right edge line; If the horizontal coordinates of all the points in the foot region are between the horizontal coordinates based on the left edge line and the horizontal coordinates based on the right edge line, and the vertical coordinates of all the points in the foot region are between the vertical coordinates based on the upper edge line and the vertical coordinates based on the lower edge line, it is determined that the personnel corresponding to the corresponding human body 3D model is located in the preparation area, and the corresponding human body 3D model is recorded, otherwise, the corresponding human body 3D model is discarded; The method for extracting the human body 3D model of all the personnel in the preparation area in the video frames and searching for the athlete who raises his hand among all the personnel in the preparation area as a tracking object comprises the following steps: All the points in each group of human body 3D models are taken, the Euclidean distances between all the points and all the corresponding points of the human body 3D model of the saved athlete who raises his hand are calculated, and the average value of the Euclidean distances of all the corresponding points is calculated; The human body corresponding to the human body 3D model with the smallest average value is taken as the candidate tracking object; 2. The method of automatically determining the process of a jump of an athlete according to claim 1, characterized in that, If the smallest average value is less than or equal to the second threshold value, the candidate tracking object is determined as the tracking object, and the corresponding video frame is recorded as tracking success; If the smallest average value is greater than the second threshold value, the corresponding video frame is determined as tracking loss, and is recorded as tracking failure. The data of tracking loss and tracking success of the video frames are recorded; The method for determining tracking failure if the number of video frames of tracking loss exceeds the first threshold value comprises the following steps: The video frames of tracking loss and tracking success are stored in a first list, and the storage content further includes video frame time; When processing a current video frame, the proportion of tracking loss in a preset time in the past is calculated according to the first list data, and if the proportion of the video frames of tracking failure exceeds a first threshold value, it is determined that tracking fails.

3. The method of automatically determining the process of a jump of an athlete according to claim 1, wherein, The method for determining whether a tracked athlete has taken off comprises: If the preparation area is on the left side of the video frame image, the vertical coordinates of all points in the region above the athlete's knees are brought into the right straight line of the preparation area, the first horizontal coordinates of all points based on the right straight line are obtained, and if the proportion of the horizontal coordinates of all points in the region above the athlete's knees that are greater than the first horizontal coordinates is greater than a third threshold value, it is determined that the athlete has taken off; If the preparation area is on the right side of the video frame image, the vertical coordinates of all points in the region above the athlete's knees are brought into the left straight line of the preparation area, the second horizontal coordinates of all points based on the left straight line are obtained, and if the proportion of the horizontal coordinates of all points in the region above the athlete's knees that are less than the second horizontal coordinates is greater than the third threshold value, it is determined that the athlete has taken off. The third threshold value is greater than or equal to one-half.

4. The method of automatically determining the process of a jump of an athlete according to claim 1, wherein, The method for determining whether a tracked athlete has reached the highest point of a jump comprises: Each video frame image after the athlete takes off is input into a 3D model generation algorithm to generate a corresponding human body 3D model, and the generated human body 3D model is input into a trained classification model; If the classification model determines that the athlete has not reached the highest point in the current video frame, the processing of the current frame is ended, and the processing of the next video frame is continued; If the classification model determines that the athlete has reached the highest point in the current video frame, the subsequent other motion state determination process is entered.

5. The method for automatically determining the process of a long jump of an athlete according to claim 1, wherein, The method for determining whether a tracked athlete has landed comprises: After determining whether the athlete has passed the highest point, the vertical coordinates of all points in the athlete's foot region in the current frame are obtained, the vertical coordinates of all points in the foot region in the previous frame are obtained, the Euclidean distances of all corresponding points in the two frames are calculated, and the distance average value is calculated; If the distance average value is less than a fourth threshold value and greater than one-half of the fourth threshold value, it is determined that the athlete in the current frame has reached the landing state, and if not, it is determined that the athlete is in a descending state; The video frames and the athlete state of which it is determined that the athlete has reached the landing state are recorded in a second list, if the proportion of the video frames in which it is determined that the athlete has reached the landing state in a plurality of consecutive video frames exceeds a preset proportion, it is determined that the athlete has reached the landing state, otherwise, it is determined that the athlete has not reached the landing state.

6. The method of automatically determining the process of a jump of an athlete according to claim 5, wherein, The method for determining whether a tracked athlete has landed further comprises: inputting the video frames in the second list into a human body 3D model generation algorithm to generate a corresponding human body 3D model, and inputting the human body 3D model into a classification model, if the classification model determines the state at the landing moment, it is considered that the athlete in the video frames starting from the corresponding video frame and thereafter is in the landing state, otherwise, it is considered that the corresponding video frame is not in the landing state.

7. The method of automatically determining the process of a long jump of an athlete according to claim 3, wherein, The method for determining whether a tracked athlete has taken off further comprises: The video frame identified as the take-off of the athlete is input into a human body 3D model generation algorithm to generate a corresponding human body 3D model, and the generated human body 3D model is input into the trained classification model; if the classification model judges that the current video frame is the take-off moment, it is considered that the athlete has indeed taken off from the current frame, otherwise it is considered that the athlete has not taken off from the current frame.