Deep learning-based sports timing method, system, storage medium and terminal

Abstract

The embodiment of the application discloses a kind of based on deep learning's motion timing method, system, storage medium and terminal, the method includes: obtaining motion area image and marking motion area;Determine optimal human skeleton coordinate point according to the motion area image, the optimal human skeleton coordinate point includes shank skeleton coordinate point;Determine whether there is pre-emptive motion in the motion area according to the shank skeleton coordinate point;Determine that there is no pre-emptive motion, then determine the final motion time and final position in the motion area according to the shank skeleton coordinate point.Through obtaining optimal human skeleton coordinate point, determine the final motion time and final position in the motion area according to the contact of optimal human skeleton coordinate point and motion area, realize automatic timing, ensure the effectiveness and fairness of timing.

Description

Technical Field

[0001] This invention relates to the field of motion monitoring technology, and in particular to a motion timing method, system, storage medium, and terminal based on deep learning. Background Technology

[0002] With the development and progress of the times, people are paying more and more attention to their health and physical condition, and exercise has become a very common way to maintain health. To better monitor exercise results, many people choose exercise timers to assist in their workouts. Taking running timing as an example, traditional exercise timers use manual stopwatches, and staff begin timing when the starting gun fires. However, this method inevitably has errors because manual timing is easily affected by visual judgment and operational mistakes, and it requires multiple timers, making it labor-intensive. Therefore, this traditional timing method cannot meet the high precision requirements of modern competitions.

[0003] Existing technologies also use infrared and photoelectric sensors for timing, but these devices have some problems, such as complex device structures, difficulty in layout and installation, high costs, and the limitation of not being able to replay test results. Summary of the Invention

[0004] Therefore, it is necessary to propose a motion timing method, system, storage medium, and terminal based on deep learning to address the above problems.

[0005] A motion timing method based on deep learning, the method comprising:

[0006] Acquire images of the moving regions and label the moving regions.

[0007] The optimal human skeleton coordinates are determined based on the motion area image, and the optimal human skeleton coordinates include the lower leg skeleton coordinates.

[0008] The existence of preemptive movement within the movement area is determined based on the coordinate points of the lower leg bones.

[0009] If it is determined that there is no preemptive movement, then the final movement time and final position within the movement area are determined based on the coordinate points of the lower leg bones.

[0010] Specifically, determining the optimal human skeleton coordinate points based on the motion region image includes the lower leg skeleton coordinate points, and specifically includes:

[0011] The motion region image is divided into several grid units using a target detection algorithm.

[0012] Each of the grid cells outputs a+b+3c values, where a is the coordinate point of the human motion area, b is the grid confidence level, and 3c is the coordinate point of the human skeleton.

[0013] Iterate through the grid confidence scores corresponding to a number of grid cells to determine the optimal grid confidence score when the grid confidence score is greater than a threshold.

[0014] Obtain the optimal human skeleton coordinates corresponding to the optimal grid confidence.

[0015] Specifically, determining whether preemptive movement exists within the movement area based on the coordinate points of the lower leg bone and the coordinate points of the rectangular region frame includes:

[0016] The preemptive movement time is determined based on the contact between the lower leg bone coordinates and the starting baseline within the movement area.

[0017] Determine the comparison relationship between the preemptive movement time and the preset movement time.

[0018] When the preemptive movement time is less than the preset movement time, it is determined that preemptive movement exists.

[0019] If the preemptive movement time is greater than the preset movement time, it is determined that there is no preemptive movement.

[0020] Specifically, determining the existence of preemptive movement when the preemptive movement time is less than the preset movement time includes:

[0021] The coordinate points of the lower leg bones are formed by connecting the coordinate points of the knee and the ankle.

[0022] When preemptive movement is determined, the preemptive movement position of the ankle coordinate point within the movement area is determined using the point polygon test function, and the preemptive movement time and the preemptive movement position are returned.

[0023] Wherein, if it is determined that there is no preemptive movement, then the final movement time and final position within the movement area are determined based on the coordinates of the optimal image rectangular region and the coordinates of the lower leg bone, specifically including:

[0024] Determine the contact between the starting baseline within the movement area and the coordinate point of the lower leg bone, and obtain the starting movement time at this point.

[0025] Determine the contact between the termination baseline within the movement area and the coordinate point of the lower leg bone, and obtain the termination time at this point.

[0026] The final movement time is determined based on the difference between the termination time and the start time.

[0027] The final movement position of the ankle coordinate point within the movement area is determined using the point polygon test function, and the final movement time and final movement position are returned.

[0028] Specifically, determining the contact between the starting baseline within the motion area and the coordinate point of the lower leg bone requires simultaneously satisfying the following conditions: and Where A is the starting coordinate of the starting baseline, and B is the ending coordinate of the starting baseline. Let C be the vector representation of the starting baseline, and B be the starting coordinates of the lower leg bone coordinate points. This is the vector representation of the coordinate points of the lower leg bones.

[0029] Specifically, determining the contact between the termination baseline within the motion area and the coordinate point of the lower leg bone requires simultaneously satisfying the following conditions: and Where E is the starting coordinate of the termination baseline, and F is the ending coordinate of the termination baseline. This is the vector representation of the starting baseline.

[0030] A deep learning-based motion timing system, the system comprising:

[0031] The motion region image acquisition module is used to acquire motion region images and mark motion regions.

[0032] The coordinate point determination module is used to determine the optimal human skeleton coordinate points based on the motion area image, wherein the optimal human skeleton coordinate points include the lower leg skeleton coordinate points.

[0033] The preemptive movement determination module is used to determine whether preemptive movement exists within the movement area based on the coordinate points of the lower leg bones.

[0034] The final movement time and final position determination module is used to determine the final movement time and final position within the movement area based on the coordinate points of the lower leg bones if there is no preemptive movement.

[0035] A storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the method described above.

[0036] A terminal includes a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method described above.

[0037] The embodiments of the present invention have the following beneficial effects:

[0038] The motion timing method based on deep learning provided by this invention obtains the optimal human skeleton coordinate points and determines the final motion time and final position within the motion area based on the contact between the optimal human skeleton coordinate points and the motion area, thereby achieving automatic timing and ensuring the effectiveness and fairness of timing. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] in:

[0041] Figure 1 This is a flowchart illustrating an embodiment of the motion timing method based on deep learning provided by the present invention;

[0042] Figure 2 This is a schematic diagram of the structure of an embodiment of the motion region image provided by the present invention;

[0043] Figure 3 This is a schematic diagram showing the vector representation of the coordinate points of the left lower leg bones and the starting baseline.

[0044] Figure 4 This is a schematic diagram showing the vector representation of the coordinate points of the left lower leg bones and the starting baseline.

[0045] Figure 5 This is a flowchart illustrating another embodiment of the motion timing method based on deep learning provided by the present invention;

[0046] Figure 6 This is a schematic diagram of an embodiment of the deep learning-based motion timing system provided by the present invention;

[0047] Figure 7 This is a schematic diagram of the structure of an embodiment of the terminal provided by the present invention;

[0048] Figure 8 This is a schematic diagram of an embodiment of the storage medium provided by the present invention. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0050] like Figure 1 As shown, Figure 1 This is a flowchart illustrating an embodiment of the deep learning-based motion timing method provided by the present invention. A deep learning-based motion timing method includes:

[0051] S101: Acquire the motion region image and label the motion region.

[0052] In one implementation scenario, two camera devices are respectively mounted on the sides of the starting and ending baselines within the motion area to capture images of the motion area and then mark these images. Specifically, for example... Figure 2 As shown, Figure 2 This is a schematic diagram of an embodiment of the motion area image provided by the present invention. The motion area image of the track is acquired, and the starting and ending baselines of the track in the image are marked. Twelve points are marked on the track in the image. The area formed by connecting points 1, 2, 3, and 4, and the area formed by connecting points 7, 8, 9, and 10, represent the first track. The area formed by connecting points 3, 4, 5, and 6, and the area formed by connecting points 9, 10, 11, and 12, represent the second track. Figure 2 Provides a motion area image with two runways.

[0053] S102: Determine the optimal human skeleton coordinate points based on the motion area image. The optimal human skeleton coordinate points include the lower leg skeleton coordinate points.

[0054] In a specific implementation scenario, the YOLOv8s_pose keypoint detection algorithm divides the motion image region into several grid cells. Each grid cell outputs 4 + 1 + 17 × 3 values, where 4 represents the coordinates of the athlete, 1 represents the grid confidence score indicating the presence of the athlete, and 17 × 3 represents the coordinates of the human skeleton. Specifically, 17 represents the human skeleton keypoint, and 3 represents the coordinates (x, y, p). Here, x is the horizontal coordinate, y is the vertical coordinate, and p is the probability value of belonging to that keypoint. The grid confidence scores of several grid cells are iterated. When the grid confidence score is greater than a threshold, the optimal grid confidence score is obtained, and then the human skeleton coordinates corresponding to that optimal grid confidence score are obtained.

[0055] It should be noted that the human skeletal coordinate points include the lower leg bone coordinate points, brow bone key points, ear bone key points, nasal bone key points, shoulder bone key points, elbow bone key points, wrist bone key points, and hip bone key points. Among them, the lower leg bone coordinate points are formed by connecting the knee coordinate points and the ankle coordinate points.

[0056] S103: Determine whether there is preemptive movement within the movement area based on the coordinate points of the lower leg bones.

[0057] In one implementation scenario, the timing of preemptive movement is determined based on the contact status between the lower leg bone coordinate points and the starting baseline within the movement area. Specifically, the lower leg bone coordinate points include the left and right lower leg bone coordinate points. Taking the left lower leg bone coordinate point as an example, it is determined whether the lower leg bone coordinate point is in contact with the starting baseline.

[0058] The starting coordinates of the initial baseline are A(L_X1, L_Y1), and the ending coordinates are B(L_X2, L_Y2). The starting coordinates of the left lower leg bone coordinate points are C(X1, Y1) and D(X2, Y2). Using point A as the reference point, connect points A and C, as well as points A and D. The vector representation of the starting baseline forms a set of vector representations: like Figure 3 As shown, Figure 3 This is a vector representation of the coordinates of the left lower leg bones relative to the starting baseline. arrive It rotates counterclockwise, and arrive It rotates clockwise, therefore, as shown in the following equation:

[0059]

[0060] Points C and D are located on either side of the starting baseline AB.

[0061] Furthermore, using C as the reference point, construct a vector group: This is a vector representation of the coordinate points of the lower leg bones. For example... Figure 4 As shown, Figure 4 This is a vector representation of the coordinates of the left lower leg bones relative to the starting baseline. arrive It rotates counterclockwise, and arrive It rotates clockwise, therefore, as shown in the following equation:

[0062]

[0063] That is, points A and B are distributed on both sides of the vector representation of the coordinate points of the left lower leg bone.

[0064] Combining formulas (1) and (2), it can be seen that the coordinate point of the left lower leg bone contacts the starting baseline. The contact time at this moment is recorded as the preemptive movement time. When the preemptive movement time is less than the preset movement time, it is determined that preemptive movement exists.

[0065] Furthermore, the coordinate points of the lower leg bones are formed by connecting the knee coordinate points and the ankle coordinate points. When it is determined that there is preemptive movement, the preemptive movement position of the ankle coordinate point within the movement area is determined according to the point polygon test function, and the preemptive movement time and preemptive movement position are returned.

[0066] S104: If it is determined that there is no preemptive movement, then the final movement time and final position within the movement area are determined based on the optimal lower leg bone coordinates.

[0067] In a specific implementation scenario, if the preemptive movement time is greater than the preset movement time, it is determined that there is no preemptive movement. The contact time between the starting baseline and the lower leg bone coordinate point within the movement area is determined by formulas (1) and (2), and recorded as the starting movement time. Further, the contact time between the ending baseline and the lower leg bone coordinate point within the movement area is determined.

[0068] Specifically, the starting coordinates of the termination baseline are E(L_X3, L_Y3), and the ending coordinates of the termination baseline are F(L_X4, L_Y4). Using C as the reference point, a set of vectors is formed by connecting points C and E, as well as C and F: arrive It rotates clockwise, and arrive It rotates counterclockwise, therefore, as shown in the following equation:

[0069]

[0070] That is, points E and F are distributed on both sides of the vector representation of the coordinate points of the left lower leg bone.

[0071] Furthermore, using E as the reference point, construct a vector group: The vector representation of the terminating baseline. arrive It rotates counterclockwise, and arrive It rotates clockwise, therefore, as shown in the following equation:

[0072]

[0073] That is, points C and D are located on either side of the vector representation of the termination baseline.

[0074] When both formulas (3) and (4) are satisfied, the coordinate point of the left lower leg bone contacts the termination baseline. The contact time at this time is recorded as the termination time. The final movement time is determined based on the difference between the termination time and the starting time.

[0075] Furthermore, the final movement position of the ankle coordinate point within the movement area is determined based on the point polygon test function, and the final movement time and final movement position are returned.

[0076] As described above, the motion timing method based on deep learning provided by this invention obtains the optimal human skeleton coordinate points and determines the final motion time and final position within the motion area based on the contact between the optimal human skeleton coordinate points and the motion area, thereby achieving automatic timing and ensuring the effectiveness and fairness of timing.

[0077] like Figure 5 As shown, Figure 5 This is a flowchart illustrating another embodiment of the deep learning-based motion timing method provided by the present invention. A deep learning-based motion timing method includes:

[0078] S201: Acquire the motion region image and label the motion region.

[0079] S202: Determine the optimal human skeleton coordinates based on the motion area image. The optimal human skeleton coordinates include the lower leg skeleton coordinates.

[0080] S203: Determine the preemptive movement time based on the contact between the lower leg bone coordinate points and the starting baseline within the movement area.

[0081] S204: Determine the comparison relationship between the advance exercise time and the preset exercise time.

[0082] S2041: When the preemptive movement time is less than the preset movement time, it is determined that preemptive movement exists.

[0083] S205: Determine the preemptive movement position of the ankle coordinate point within the movement area based on the point polygon test function, and return the preemptive movement time and preemptive movement position.

[0084] It should be noted that the coordinate points of the lower leg bones are formed by connecting the coordinate points of the knee and the ankle.

[0085] S2042: When the preemptive movement time is greater than the preset movement time, it is determined that there is no preemptive movement.

[0086] S206: Determine the contact between the starting baseline and the lower leg bone coordinate point within the motion area, and obtain the starting motion time at this point.

[0087] S207: Determine the contact between the termination baseline and the lower leg bone coordinate point within the motion area, and obtain the termination time at this point.

[0088] S208: Determine the final movement time based on the difference between the termination time and the start time.

[0089] In one implementation scenario, the deep learning-based motion timing method provided by this invention can be used for basketball round-trip timing. Two camera devices are respectively set up on the sides of the starting point and the round-trip point within the motion area to capture images of the basketball court's motion area. These images are then marked and calibrated. Specifically, the tester starts from the starting point (starting baseline), dribbles the ball to a designated location, shoots, dribbles back to the round-trip point, and then dribbles back to the designated point to shoot again. After the shot is made, the tester dribbles to the ending baseline, ends the timing, calculates the total time, and then calculates the score. The contact time between the lower leg bone coordinate point and the starting point is taken as the starting motion time, and timing begins at this point. The contact time between the lower leg bone coordinate point and the ending baseline is taken as the ending motion time, and timing ends at this point. It is also necessary to record the contact time with the round-trip point and the number of contacts between the lower leg bone coordinate point and the round-trip point. Generally, the number of contacts is set to 3. After 3 contacts, the contact between the lower leg bone coordinate point and the ending baseline is determined, and then the final ending motion time is returned. The final score is determined by the difference between the ending motion time and the starting motion time. The steps for determining the final score are shown in S201-S207. It should be noted that steps S201-S207 are... Figure 1 The implementation scenarios shown have been discussed in detail and will not be repeated here.

[0090] In another implementation scenario, the deep learning-based motion timing method provided by this invention can be used for timing shuttle runs. It requires the tester to make multiple back-and-forth runs, and two camera devices are set up on the sides of the starting point and the back-and-forth point inside the motion area to collect images of the motion area of ​​the track. The motion area is then calibrated in these images. Specifically, the tester starts from the starting baseline (starting point) within the exercise area and records the starting time t1. After reaching the round-trip line within the exercise area, the first round-trip time t2 is recorded. Then, the tester runs back to the starting point to complete the first round-trip run and records the time of contact with the starting point, which is recorded as the starting time. Then, the second round-trip run begins. When the coordinate point of the lower leg bone contacts the round-trip line again, the current system time t4 is recorded. Then, the tester runs back towards the starting point and contacts the starting point again, recording the current system time t5. At this point, the second round-trip run is completed. This process is repeated to obtain the contact times for the next three round-trip runs as t6, t7, t8, t9, t10, and t11, respectively. When (number of contact times - 1) / 2 equals the number of turns, the timing stops. The last contact time with the starting point (ending baseline) is the ending time, which is t11. The tester is then notified that the test has been completed. The final total round-trip time is t = t11 - t1, which is the ending time - the starting time. The steps for determining the final score are shown in S201-S207. It should be noted that steps S201-S207 are... Figure 1 The implementation scenarios shown have been discussed in detail and will not be repeated here.

[0091] S209: Determine the final movement position of the ankle coordinate point within the movement area based on the point polygon test function, and return the final movement time and final movement position.

[0092] In one implementation scenario, the coordinate points of the lower leg bones are formed by connecting the knee coordinate points and the ankle coordinate points. The track area where the ankle coordinates are located is determined by a point polygon test function.

[0093] Specifically, in combination Figure 2 In the case of two running track areas in the exercise area, the first running track area and the ankle coordinates to be tested are input into the point polygon test function. When the function returns -1, the ankle coordinates are not in the first running track area, so the first running track area and the ankle coordinates to be tested are input again for judgment; when the function returns 1, the ankle coordinates are in the first running track area; when the function returns 0, the ankle coordinates are in the first running track area.

[0094] As described above, the deep learning-based motion timing method provided by this invention uses the yolov8s_pose key point detection algorithm based on deep learning to detect the optimal human skeleton key points of the tester in one go. Automatic timing is achieved by the contact between the lower leg bone key points and the starting and ending baselines, which meets the high precision requirements of modern competitions. Moreover, it only requires two artificial intelligence cameras to be placed in a suitable position to acquire images of the motion area, simplifying the equipment structure and deployment.

[0095] like Figure 6 As shown, Figure 6 This is a schematic diagram of an embodiment of the deep learning-based sports timing system provided by the present invention. A deep learning-based sports timing system 10, system 10 includes:

[0096] The motion region image acquisition module 11 is used to acquire motion region images and mark motion regions.

[0097] The coordinate point determination module 12 is used to determine the optimal human skeleton coordinate points based on the motion area image. The optimal human skeleton coordinate points include the lower leg skeleton coordinate points.

[0098] The preemptive movement determination module 13 is used to determine whether preemptive movement exists within the movement area based on the coordinate points of the lower leg bones.

[0099] The final movement time and final position determination module 14 is used to determine the final movement time and final position within the movement area based on the coordinate points of the lower leg bones if there is no preemptive movement.

[0100] In a specific implementation scenario, in the motion area image acquisition module 11, two camera devices are respectively mounted on the sides of the starting and ending baselines within the motion area to acquire motion area images, which are then marked. In the coordinate point determination module 12, the motion area image is divided into several grid cells using a target detection algorithm. The grid confidence scores of several grid cells are traversed, and the optimal grid confidence score, which is greater than a threshold, is determined. The optimal human skeleton coordinate points corresponding to the optimal grid confidence score are then obtained. In the preemptive movement determination module 13, the preemptive movement time is determined based on the contact between the lower leg bone coordinate points and the starting baseline within the motion area. The preemptive movement time is compared with a preset movement time. If the preemptive movement time is less than the preset movement time, preemptive movement is confirmed; if the preemptive movement time is greater than the preset movement time, preemptive movement is confirmed to be absent. In the final movement time and final position determination module 14, the starting baseline and the lower leg bone coordinate point in the movement area are determined to be in contact, and the starting movement time is obtained at this time. The ending baseline and the lower leg bone coordinate point in the movement area are determined to be in contact, and the ending movement time is obtained at this time. The final movement time is determined based on the difference between the ending movement time and the starting movement time. The final movement position of the ankle coordinate point in the movement area is determined based on the point polygon test function, and the final movement time and final movement position are returned.

[0101] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of an embodiment of the terminal provided by the present invention. The terminal 20 includes a memory 21 and a processor 22. The memory 21 stores a computer program, and the processor 22 executes the computer program during operation to achieve, for example, Figure 1 and Figure 5 The method shown.

[0102] The specific technical details of the motion timing method based on deep learning implemented by the aforementioned terminal 20 when executing the computer program have been discussed in detail in the above method steps, and therefore will not be repeated here.

[0103] Please see Figure 8 , Figure 8 This is a schematic diagram of a structure of an embodiment of the storage medium provided by the present invention. The storage medium 30 stores at least one computer program 31, which is executed by the processor 22 to perform the following... Figure 1 and Figure 5 The method shown is detailed above and will not be repeated here. In one embodiment, the storage medium 30 can be a storage chip, hard disk, portable hard disk, USB flash drive, optical disk, or other read / write storage device, or even a server, etc.

[0104] The foregoing has described specific embodiments of this specification; other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than those shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily have to follow the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0105] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and non-volatile computer-readable storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0106] The apparatus, device, non-volatile computer-readable storage medium and method provided in the embodiments of this specification are corresponding. Therefore, the apparatus, device and non-volatile computer storage medium also have similar beneficial technical effects as the corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the corresponding apparatus, device and non-volatile computer storage medium will not be repeated here.

[0107] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0108] For ease of description, the above apparatus is described by dividing it into various functional units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components. Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, the embodiments of this specification can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0109] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0110] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0111] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0112] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0113] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0114] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0115] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0116] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0117] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0118] The above description discloses only preferred embodiments of the present invention and should not be construed as limiting the scope of the present invention. Therefore, equivalent variations made in accordance with the claims of the present invention are still within the scope of the present invention.

Claims

1. A motion timing method based on deep learning, characterized in that, The method includes: Acquire images of the moving regions and label the moving regions; The optimal human skeleton coordinate points are determined based on the motion area image, and the optimal human skeleton coordinate points include the lower leg skeleton coordinate points. Determining whether preemptive movement exists within the movement area based on the lower leg bone coordinate points specifically includes: determining the preemptive movement time based on the contact between the lower leg bone coordinate points and the starting baseline within the movement area; and determining the comparison relationship between the preemptive movement time and the preset movement time. When the preemptive movement time is less than the preset movement time, it is determined that preemptive movement exists. Specifically, the coordinate points of the lower leg bones are formed by connecting the knee coordinate points and the ankle coordinate points. When preemptive movement is determined, the preemptive movement position of the ankle coordinate point within the movement area is determined according to the point polygon test function, and the preemptive movement time and the preemptive movement position are returned. When the preemptive movement time is greater than the preset movement time, it is determined that there is no preemptive movement; If no preemptive movement is determined, the final movement time and final position within the movement area are determined based on the lower leg bone coordinates. Specifically, this includes: determining when the starting baseline within the movement area contacts the lower leg bone coordinates and obtaining the starting movement time; determining when the ending baseline within the movement area contacts the lower leg bone coordinates and obtaining the ending movement time; determining the final movement time based on the difference between the ending movement time and the starting movement time; and determining the final movement position of the ankle coordinates within the movement area using a point polygon test function and returning the final movement time and final movement position. Specifically, when there are two running lane areas in the movement area, the first running lane area and the ankle coordinates to be tested are input into the point polygon test function. When the function returns -1, the ankle coordinates are not within the first running lane area, so the first running lane area and the ankle coordinates to be tested are input again for judgment; when the function returns 1, the ankle coordinates are within the first running lane area; when the function returns 0, the ankle coordinates are on the first running lane area.

2. The motion timing method based on deep learning according to claim 1, characterized in that, The step of determining the optimal human skeleton coordinate points based on the motion area image, wherein the optimal human skeleton coordinate points include the lower leg skeleton coordinate points, specifically includes: The motion region image is divided into several grid units using a target detection algorithm; Each of the grid cells outputs a+b+3c values, where a is the number of coordinate points in the human motion area, b is the number of grid confidence scores, and 3c is the number of coordinate points in the human skeleton. Iterate through the grid confidence scores corresponding to a number of grid cells to determine the optimal grid confidence score when the grid confidence score is greater than a threshold. Obtain the optimal human skeleton coordinates corresponding to the optimal grid confidence.

3. The motion timing method based on deep learning according to claim 1, characterized in that, The determination of the contact between the starting baseline within the motion area and the coordinate point of the lower leg bone requires that simultaneously satisfy the following conditions: and Where A is the starting coordinate of the starting baseline, and B is the ending coordinate of the starting baseline. Let C be the vector representation of the starting baseline, and D be the starting coordinates of the lower leg bone coordinate points. This is the vector representation of the coordinate points of the lower leg bones.

4. The motion timing method based on deep learning according to claim 3, characterized in that, The determination of the contact between the termination baseline within the motion area and the coordinate point of the lower leg bone requires that simultaneously satisfy the following conditions: and Where E is the starting coordinate of the termination baseline, and F is the ending coordinate of the termination baseline. This is the vector representation of the starting baseline.

5. A motion timing system based on deep learning, characterized in that, The system includes: The motion region image acquisition module is used to acquire motion region images and mark motion regions; The coordinate point determination module is used to determine the optimal human skeleton coordinate points based on the motion area image, wherein the optimal human skeleton coordinate points include the lower leg bone coordinate points. The preemptive movement determination module is used to determine whether preemptive movement exists within the movement area based on the lower leg bone coordinate points. Specifically, it includes: determining the preemptive movement time based on the contact between the lower leg bone coordinate points and the starting baseline within the movement area; determining the comparison between the preemptive movement time and a preset movement time; determining that no preemptive movement exists when the preemptive movement time is greater than the preset movement time; and determining that preemptive movement exists when the preemptive movement time is less than the preset movement time. Specifically, the lower leg bone coordinate points are formed by connecting knee and ankle coordinate points; when preemptive movement is determined, the preemptive movement position of the ankle coordinate point within the movement area is determined using a point polygon test function, and the preemptive movement time and position are returned. The final movement time and final position determination module is used to determine the final movement time and final position within the movement area based on the lower leg bone coordinates if no preemptive movement is detected. Specifically, this includes: determining when the starting baseline within the movement area contacts the lower leg bone coordinates and obtaining the starting movement time; determining when the ending baseline within the movement area contacts the lower leg bone coordinates and obtaining the ending movement time; determining the final movement time based on the difference between the ending movement time and the starting movement time; and determining the final movement position of the ankle coordinates within the movement area using a point polygon test function and returning the final movement time and final movement position. Specifically, when there are two running track areas in the movement area, the first running track area and the ankle coordinates to be tested are input into the point polygon test function. If the function returns -1, the ankle coordinates are not within the first running track area, and the first running track area and the ankle coordinates to be tested are input again for judgment. If the function returns 1, the ankle coordinates are within the first running track area; if the function returns 0, the ankle coordinates are within the first running track area.

6. A storage medium, characterized in that, The device stores a computer program that, when executed by a processor, causes the processor to perform the steps of the method as described in any one of claims 1 to 4.

7. A terminal, characterized in that, It includes a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method as described in any one of claims 1 to 4.

Images