Method, device and equipment for obtaining long jump distance and computer readable storage medium

Abstract

The application discloses a long-jump distance acquisition method, device and equipment and a computer readable storage medium. The method comprises the following steps: acquiring an image frame and a depth image of the image frame; identifying a human target and a corresponding category in the image frame based on a target detection module and a target tracking module; determining the position of the human target of each category in the image frame according to the depth image; calculating the moving distance of the human target according to the position; if the moving distance of the human target in the current image frame is greater than a preset value, regarding the human target as a long-jump target and regarding the previous image frame as a take-off frame; if the moving distance of the long-jump target in the previous image frame is greater than the moving distance of the long-jump target in the next image frame after the take-off frame, regarding the previous image frame as a stop frame; and calculating the long-jump distance according to the take-off frame and the stop frame. Thus, the application can give an objective and accurate long-jump distance calculation result, avoids the generation of cheating behaviors and also avoids the observation error of the naked eye.

Description

Technical Field

[0001] This invention relates to the field of target detection, and more particularly to a method, apparatus, device, and computer-readable storage medium for obtaining long jump distance. Background Technology

[0002] In long jump, the distance is usually determined manually. However, it is understandable that human verification inevitably involves observational errors and the possibility of cheating, which leads to a lower reliability of manually verified long jump distances. Summary of the Invention

[0003] In view of this, the present invention provides a method, apparatus, device and computer-readable storage medium for obtaining long jump distance, so as to improve the low reliability of long jump distance based on human verification.

[0004] In a first aspect, embodiments of the present invention provide a method for obtaining long jump distance, comprising:

[0005] Real-time acquisition of image frames and depth images of those image frames;

[0006] Using a pre-set target detection module and target tracking module, human targets of each category in each image frame are identified;

[0007] Based on the depth image of each image frame, determine the location of the human target of each category in each image frame;

[0008] Based on the position of the human target of each category in each image frame, calculate the distance moved by the human target of each category in each image frame relative to the previous image frame;

[0009] If, in the current image frame, there exists a category of human targets whose moving distance is greater than a preset value, then the category of human targets whose moving distance is greater than the preset value is taken as the long jump target, and the previous image frame of the current image frame is taken as the starting frame of the long jump target.

[0010] For the image frames following the take-off frame, if the distance the long jump target moves in the preceding image frame is greater than the distance it moves in the following image frame, then the preceding image frame is taken as the stop frame of the long jump target.

[0011] The long jump distance of the long jump target is obtained based on the position of the long jump target in the corresponding take-off frame and the corresponding stop frame.

[0012] Optionally, in one feasible embodiment of the present invention, obtaining the long jump distance of the long jump target based on the position of the long jump target in the corresponding take-off frame and the corresponding stop frame includes:

[0013] If a preset take-off position exists, the jump distance of the long jump target is calculated based on the preset take-off position and the position of the long jump target in the corresponding stop frame.

[0014] If the preset take-off position does not exist, the difference between the position of the long jump target in the corresponding take-off frame and the corresponding stop frame is calculated to obtain the long jump distance of the long jump target.

[0015] Optionally, in one feasible embodiment of the present invention, the step of taking a human target of a certain category whose moving distance is greater than a preset value in the current image frame as a long jump target and taking the previous image frame of the current image frame as the starting frame of the long jump target includes:

[0016] If, in the current image frame, there exists a category of human targets whose moving distance is greater than a preset value, and the position of the human targets of that category is within a preset starting position range, then the human targets of that category whose moving distance is greater than the preset value are taken as long jump targets, and the previous image frame of the current image frame is taken as the starting frame of the long jump targets.

[0017] Optionally, in one feasible embodiment of the present invention, the step of identifying human targets of each category in each image frame using a preset target detection module and target tracking module includes:

[0018] Each image frame is input to a preset target detection module to identify each category of target and the bounding box of each target in each image frame;

[0019] Based on the preset nonmaximum suppression module, the bounding boxes of the same target in each image frame are merged into one bounding box;

[0020] Based on the bounding box of the human target classified as human, the image frame corresponding to the human target is cropped to obtain the human image of the human target.

[0021] The human image of each image frame is input into a preset target tracking module to identify human targets of each category in each image frame.

[0022] Optionally, in one feasible embodiment of the present invention, the step of inputting the human image of each image frame to a preset target tracking module to identify the human target of each category in each image frame includes:

[0023] Based on the preset target tracking module, feature extraction is performed on the region image corresponding to the bounding box of the human target to obtain the feature label of each human target;

[0024] Use the feature label of each human target in the first image frame of all image frames as the target label;

[0025] For each image frame after the first image frame, the feature label with the highest cosine similarity to the target label among the feature labels of each human target in the image frame is taken as the similar label of the target label;

[0026] The human targets corresponding to the target label and the human targets corresponding to similar labels of the target label are set to the same category to obtain the human targets of each category in each image frame.

[0027] Optionally, in one feasible embodiment of the present invention, the size of the image frame is consistent with the size of the depth image of the image frame, and the pixel information of the pixels in the depth image includes depth;

[0028] Determining the location of each category of human target in each image frame based on the depth image of each image frame includes:

[0029] For each image frame, based on the bounding box of each human target in the image frame, determine the region depth image of each human target in the depth image of the image frame;

[0030] Based on a first preset formula, a first average depth value of the depth image of the human target is calculated according to the depth of each pixel in the depth image of the human target region, wherein the first preset formula includes:

[0031]

[0032] In the formula, d mean Let d represent the first depth mean of the human target, n represent the number of pixels included in the depth image of the human target region, and d represent the depth mean of the human target region. i This represents the depth of the i-th pixel in the depth image of the human target region;

[0033] Based on a second preset formula, a pixel segmentation threshold for the region depth image of each human target is calculated, wherein the second preset formula includes:

[0034]

[0035] In the formula, σ represents the pixel segmentation threshold of the region depth image, and d j This represents the depth of the j-th pixel in the depth image of the human target region;

[0036] For each region depth image of the human target, a second depth of the region depth image is calculated based on the first preset formula and the pixels in the region depth image whose difference from the mean depth of the region depth image is less than or equal to the pixel segmentation threshold of the region depth image. The second depth is then used as the position of the human target corresponding to the region depth image.

[0037] Optionally, in one feasible embodiment of the present invention, the step of identifying human targets of each category in each image frame using a preset target detection module and target tracking module includes:

[0038] The image frame is scaled and normalized to obtain a preprocessed image frame;

[0039] Based on the preset target detection module and target tracking module, each category of human target is identified in each preprocessed image frame.

[0040] Secondly, embodiments of the present invention provide a device for calculating long jump distance, comprising:

[0041] The acquisition module is used to acquire image frames and depth images of the image frames in real time;

[0042] The recognition module is used to identify human targets of each category in each image frame by utilizing a preset target detection module and target tracking module;

[0043] The determination module is used to determine the location of each category of human target in each image frame based on the depth image of each image frame;

[0044] The distance calculation module is used to calculate the movement distance of each category of human target in each image frame relative to the previous image frame, based on the position of each category of human target in each image frame.

[0045] The jump determination module is used to determine if, in the current image frame, there is a human target of a certain category whose moving distance is greater than a preset value, the human target of the category whose moving distance is greater than the preset value is taken as the long jump target, and the previous image frame of the current image frame is taken as the jump frame of the long jump target.

[0046] The stop determination module is used to determine the stop frame of the long jump target if, for the image frame following the take-off frame, the distance the long jump target moves in the preceding image frame is greater than the distance it moves in the following image frame.

[0047] The calculation module is used to obtain the long jump distance of the long jump target based on the position of the long jump target in the corresponding take-off frame and the corresponding stop frame.

[0048] Thirdly, embodiments of the present invention provide a computer device, including a memory and a processor, wherein the memory stores a computer program, and the computer program executes a long jump distance acquisition method as disclosed in any of the first aspects when running on the processor.

[0049] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when run on a processor, executes a method for obtaining long jump distance as disclosed in any of the first aspects.

[0050] In the long jump distance acquisition method provided by this invention, a computer device acquires image frames and depth images of the image frames in real time based on a preset camera device. Then, based on a preset target detection module and target tracking module, it identifies each human target and its category in each image frame. Next, based on the depth image of each image frame, it determines the position of each category of human target in the image frame, i.e., the position of each human target relative to the camera device. Subsequently, based on the position of each category of human target in each image frame, it calculates the movement distance of each category of human target relative to the previous image frame, i.e., the difference between the position of the human target in the current frame and its position in the previous frame is used as the movement distance. Next, if in the current image frame, there exists a category of human target whose moving distance is greater than a preset value, it indicates that the human target jumped in the previous frame. Therefore, the category of human target whose moving distance is greater than the preset value is taken as the long jump target, and the image frame preceding the current image frame is taken as the starting frame of the long jump target. After the starting frame, if the moving distance of the long jump target in the preceding image frame is greater than the moving distance in the following image frame, it indicates that the long jump target landed in the preceding image frame. Therefore, the preceding image frame is taken as the stopping frame of the long jump target. Finally, based on the position of the long jump target in the corresponding starting frame and the corresponding stopping frame, the long jump distance of the long jump target is obtained.

[0051] Based on this, the embodiments of the present invention realize real-time detection of the position of the long jumper based on the target detection module and the target tracking module; at the same time, combined with the corresponding movement distance of the long jumper at the take-off and landing, accurate detection of take-off and landing is realized; furthermore, the long jump distance calculation based on computer equipment can provide objective and accurate long jump distance calculation results, avoiding cheating and avoiding observation errors that may occur in visual observation. Attached Figure Description

[0052] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope of protection of the present invention. In the various drawings, similar components are numbered similarly.

[0053] Figure 1 A flowchart illustrating the first method for obtaining long jump distance provided by an embodiment of the present invention is shown;

[0054] Figure 2 A flowchart illustrating the second method for obtaining long jump distance provided in an embodiment of the present invention is shown;

[0055] Figure 3 A schematic diagram of the process of S130 provided in an embodiment of the present invention is shown;

[0056] Figure 4 A schematic diagram of the structure of the long jump distance calculation device provided in an embodiment of the present invention is shown. Detailed Implementation

[0057] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0058] The components of the embodiments of the invention described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0059] In the following, the terms “comprising,” “having,” and their cognates, which may be used in various embodiments of the invention, are intended only to indicate a particular feature, number, step, operation, element, component, or combination thereof, and should not be construed as excluding, firstly, the presence of one or more other features, numbers, steps, operations, elements, components, or combinations thereof, or adding the possibility of one or more features, numbers, steps, operations, elements, components, or combinations thereof.

[0060] Furthermore, the terms "first," "second," and "third" are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.

[0061] Unless otherwise specified, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which the various embodiments of the invention pertain. Terms (such as those defined in commonly used dictionaries) shall be interpreted as having the same meaning as in their contextual meaning in the relevant technical field and shall not be interpreted as having an idealized or overly formal meaning, unless clearly defined in the various embodiments of the invention.

[0062] Example 1

[0063] Reference Figure 1 The diagram illustrates a flowchart of a first method for obtaining long jump distance provided by an embodiment of the present invention. The method for obtaining long jump distance provided by an embodiment of the present invention includes:

[0064] S110, acquire the image frame and the depth image of the image frame in real time.

[0065] It is understood that the computer device in this embodiment of the invention will use the shooting module / device / component to capture images containing long jump athletes in real time and obtain the corresponding depth images.

[0066] It is easy to understand that, in order to ensure accurate identification of the start and end of the long jump athlete's long jump action, the embodiments of the present invention enable the shooting module to capture images in real time at a shooting mode of multiple frames per second, that is, to ensure accurate acquisition of the long jump action by shooting frame by frame.

[0067] It is also understood that the method of acquiring the depth image of the image can be set according to the actual situation. Exemplarily, in one feasible embodiment of the present invention, the computer device acquires image frames and depth images based on a first camera and a second camera that capture images at the same frequency. The first camera is used to capture image frames, and the second camera is used to capture depth images simultaneously with the first camera, thereby obtaining the depth image of the image frame.

[0068] In a preferred embodiment of the present invention, the computer device completes the image frame and the depth image of the image frame based on the RealSense camera.

[0069] S120, using a preset target detection module and target tracking module, identify human targets of each category in each image frame.

[0070] That is, when the computer device obtains an image frame, it inputs the image frame into a preset target detection module and target tracking module, so that the target detection module can detect the human body in the image frame and the target tracking module can identify the position of the human body in each image frame.

[0071] It is understood that the target detection module and the target tracking module in the embodiments of the present invention can be implemented by selecting the corresponding algorithm or model according to the actual situation. For example, in one feasible method provided in the embodiments of the present invention, the target detection module includes YOLOv5s.

[0072] It is also understandable that object detection tasks refer to identifying different types of objects / targets in an image. For example, when an image includes multiple pedestrians, multiple trees, and multiple vehicles, the object detection task needs to identify and label the people, trees, and vehicles in the image, but it does not distinguish between different pedestrians, different trees, and different vehicles.

[0073] In target tracking tasks, it is necessary to identify the location of targets of the same category in different images. For example, if multiple images all contain pedestrian targets A, B, and C, the target tracking task needs to identify which category (A, B, or C) each pedestrian in each image belongs to.

[0074] S130, determine the location of each category of human target in each image frame based on the depth image of each image frame.

[0075] Understandably, the pixel information of each pixel in a depth image will include depth, which represents the distance between the object corresponding to the pixel and the camera.

[0076] It is easy to understand that in the embodiments of the present invention, "the position of the human target" refers to the position of the human target relative to the camera.

[0077] It is also understood that, in this embodiment of the invention, after the computer device uses the target detection module and the target tracking module to determine the pixel position of each type of human target in the image frame, it will use the depth image of the image frame to determine the depth of the pixels at the same / corresponding pixel positions in the depth image, and then determine the distance of each human target relative to the camera.

[0078] Optionally, in one feasible approach, the computer device calculates the average depth based on the depth of each pixel in the depth image of the human target, and uses the average depth as the distance of the human target relative to the camera.

[0079] Furthermore, it is understood that the term "camera" mentioned in the embodiments of the present invention refers to a module / device / device used for capturing images in the field of target detection / recognition, and is not intended to limit the specific category of the capturing module / device / device in the embodiments of the present invention.

[0080] S140, based on the position of the human target of each category in each image frame, calculate the moving distance of the human target of each category in each image frame relative to the previous image frame.

[0081] In other words, when determining the position of each category of human target in each image frame, the computer device will use the positional changes of each category of human target in the preceding and following frames to determine the moving distance of each category of human target in the following image frame.

[0082] It is easy to understand that the first image frame does not have a corresponding previous image frame. Therefore, in this embodiment of the invention, the movement distance of each category of human target in the first image frame is 0.

[0083] Furthermore, after the computer device uses the target detection module, the target tracking module, and the depth image of the image frame to determine the position of each category of human target in each image frame, for each image frame after the first image frame, the computer device will use the position of each category of human target in the current image frame and the position in the previous image frame to calculate the change in position, that is, calculate the moving distance of each category of human target relative to the previous image frame.

[0084] S150, if in the current image frame, there exists a category of human targets whose moving distance is greater than a preset value, then the category of human targets whose moving distance is greater than the preset value is taken as the long jump target, and the previous image frame of the current image frame is taken as the starting frame of the long jump target.

[0085] In other words, as the computer device continuously determines the moving distance of each category of human target in each image frame based on the acquired image frames, if there is a category of human target larger than a preset value in the last acquired image frame, that is, in the latest acquired image frame, it means that the human target of this category jumped in the previous image frame, and therefore the moving distance of the human target of this category in the current image frame is greater than the preset value.

[0086] Furthermore, after determining that the human target has moved a distance greater than a preset value in the current frame / latest frame, the computer device uses this type of human target as the long jump target and the previous image frame before the current frame as the starting frame for the long jump target.

[0087] Furthermore, it is understood that the preset values ​​in the embodiments of the present invention are set according to actual conditions, and therefore are not limited here.

[0088] S160, for the image frame after the take-off frame, if the distance the long jump target moves in the previous image frame is greater than the distance it moves in the next image frame, then the previous image frame is taken as the stop frame of the long jump target.

[0089] It is understandable that after a long jumper jumps and lands, the athlete may remain stationary or return to a standing position and walk / jog slowly. Therefore, the athlete's speed after landing should be less than the speed during the jump.

[0090] Based on this, after determining the take-off frame, this embodiment of the invention will determine whether the moving distance of the long jumper, i.e. the long jump target, in any image frame after the take-off frame is less than the moving distance of the long jump target in the previous image frame.

[0091] If not, it means that the long jump target is still in the air and the long jump process has not yet ended.

[0092] If so, it means that the long jump target landed in the previous image frame, so the previous image frame is taken as the stop frame of the long jump target.

[0093] S170, based on the position of the long jump target in the corresponding take-off frame and the corresponding stop frame, the long jump distance of the long jump target is obtained.

[0094] It is understandable that the position of the long jump target in the corresponding take-off frame and the position in the corresponding stop frame represent the distance of the long jump target relative to the camera when taking off and the distance of the long jump target relative to the camera when landing, respectively. Therefore, by using the position of the long jump target in the take-off frame and the stop frame, the change in distance of the long jump target relative to the camera can be calculated, that is, the long jump distance of the long jump target.

[0095] It is also understandable that if the camera is located in the direction of the long jumper's movement, the distance between the long jumper and the camera will decrease after the long jump ends. Therefore, in this case, the long jump distance is calculated as: the position of the long jump target at the start frame minus the position of the long jump target at the stop frame.

[0096] When the camera is located in the negative direction of the long jumper's movement, the long jump distance is calculated as follows: the position of the long jump target at the stop frame minus the position of the long jump target at the take-off frame.

[0097] In one feasible approach, the long jump distance is calculated as the absolute value of the difference between the position of the long jump target in the stop frame and the position of the long jump target in the take-off frame.

[0098] Furthermore, it is understood that if the long jumper takes off at or before the takeoff line / takeoff point, the long jump distance should be the difference between the position of the long jump target in the stop frame and the position of the takeoff line. Therefore, in one feasible embodiment of the present invention, S170 includes:

[0099] If a preset take-off position exists, the jump distance of the long jump target is calculated based on the preset take-off position and the position of the long jump target in the corresponding stop frame.

[0100] If the preset take-off position does not exist, the difference between the position of the long jump target in the corresponding take-off frame and the corresponding stop frame is calculated to obtain the long jump distance of the long jump target.

[0101] In other words, under this feasible method, the computer device in this embodiment of the invention will determine whether it has a pre-stored position of the take-off point / take-off line, i.e., a preset take-off position, before calculating the long jump distance.

[0102] If a preset take-off position is stored, it means that the long jump distance is calculated as the difference between the preset take-off position and the long jumper's landing position.

[0103] If no preset start position is stored, it means that the long jump distance is calculated as the difference between the position of the target in the corresponding start frame and the corresponding stop frame.

[0104] Therefore, the embodiments of the present invention can reasonably and accurately calculate the long jump distance for any type of long jump sport, overcoming the calculation differences under different application scenarios.

[0105] In the long jump distance acquisition method provided by this invention, a computer device acquires image frames and depth images of the image frames in real time based on a preset camera device. Then, based on a preset target detection module and target tracking module, it identifies each human target and its category in each image frame. Next, based on the depth image of each image frame, it determines the position of each category of human target in the image frame, i.e., the position of each human target relative to the camera device. Subsequently, based on the position of each category of human target in each image frame, it calculates the movement distance of each category of human target relative to the previous image frame, i.e., the difference between the position of the human target in the current frame and its position in the previous frame is used as the movement distance. Next, if in the current image frame, there exists a category of human target whose moving distance is greater than a preset value, it indicates that the human target jumped in the previous frame. Therefore, the category of human target whose moving distance is greater than the preset value is taken as the long jump target, and the image frame preceding the current image frame is taken as the starting frame of the long jump target. After the starting frame, if the moving distance of the long jump target in the preceding image frame is greater than the moving distance in the following image frame, it indicates that the long jump target landed in the preceding image frame. Therefore, the preceding image frame is taken as the stopping frame of the long jump target. Finally, the long jump distance of the long jump target is calculated based on the position of the long jump target in the corresponding starting frame and the corresponding stopping frame.

[0106] Based on this, the embodiments of the present invention realize real-time detection of the position of the long jumper based on the target detection module and the target tracking module; at the same time, combined with the corresponding movement distance of the long jumper at the take-off and landing, accurate detection of take-off and landing is realized; furthermore, the long jump distance calculation based on computer equipment can provide objective and accurate long jump distance calculation results, avoiding cheating and avoiding observation errors that may occur in visual observation.

[0107] Optionally, in one feasible embodiment of the present invention, step S150 includes:

[0108] If, in the current image frame, there exists a category of human targets whose moving distance is greater than a preset value, and the position of the human targets of that category is within a preset starting position range, then the human targets of that category whose moving distance is greater than the preset value are taken as long jump targets, and the previous image frame of the current image frame is taken as the starting frame of the long jump targets.

[0109] It is understandable that when a long jumper needs a run-up to increase the long jump distance, the distance the long jumper moves in the take-off frame and the frame before take-off may be very close and both greater than the preset value. This can cause the computer to mistakenly take the frame before take-off as the take-off frame and output an incorrect long jump distance calculation result.

[0110] In response to this situation, the present invention controls the computer device to determine whether the moving distance of each type of human target in the image frame exceeds a preset value after obtaining the position and moving distance of each type of human target in real time, and to determine whether the position of each type of human target is within a preset starting position range. If the moving distance exceeds the preset value and the position is within the preset starting position range, it indicates that the human target of this type is a long jump target.

[0111] It should be noted that, in the embodiments of the present invention, the take-off position interval represents the area in front of the take-off line. In one example, since the distance between the take-off line and the camera represents the take-off line position in the embodiments of the present invention, and the distance between the long jumper and the camera continuously shortens as the long jumper performs the long jump, the take-off position interval is an interval composed of a first distance value and a second distance value, wherein the first distance value represents the take-off line position, and the second distance value is greater than the first distance value.

[0112] Based on this feasible approach, the embodiments of the present invention only consider human targets that have moved a distance exceeding a preset value within a preset take-off position range as jump targets, thereby avoiding the situation where the computer device identifies the long jumper's approach run as a take-off, thus ensuring the reliability of the long jump distance output by the computer device.

[0113] Optionally, in one feasible embodiment of the present invention, please refer to the following: Figure 2 The diagram illustrates a flowchart of S120 provided in an embodiment of the present invention. S120 in this feasible manner includes:

[0114] S121, each of the image frames is input to a preset target detection module to identify each class of target and the bounding box of each target in each image frame;

[0115] S122, based on the preset non-maximum suppression module, merge the bounding boxes of the same target in each image frame into one bounding box;

[0116] S123, based on the bounding box of the human target classified as human body, crop the image frame corresponding to the human target to obtain the human image of the human target;

[0117] S124, input the human image of each image frame to the preset target tracking module to identify the human target of each category in each image frame.

[0118] It is understandable that in object detection tasks, various factors such as improper anchor settings and / or improper detection range settings may cause the object detection model to generate multiple corresponding bounding boxes for a single object in an image.

[0119] To address this issue, this embodiment of the invention applies Non-Maximum Suppression (NMS) to the bounding boxes output by the target tracking module / module, processing multiple bounding boxes corresponding to a single target, i.e., merging multiple bounding boxes into one. This avoids the situation where subsequent target tracking tasks will produce erroneous tracking results when the same target corresponds to multiple bounding boxes.

[0120] In a preferred embodiment, the computer device determines the largest common area corresponding to multiple bounding boxes, and uses the largest common area as the bounding box of the target.

[0121] Furthermore, since this embodiment of the invention is for the detection of long jump athletes, that is, for the detection of human targets, after the computer completes the merging of bounding boxes, it deletes the bounding boxes of targets whose type is not "human" in each target in each image frame, so that in the subsequent cropping process, only the images corresponding to targets whose type is "human", that is, the images corresponding to human targets, are cropped.

[0122] Optionally, in a preferred embodiment of the present invention, since the long jumper is always located near the center of each image frame, after the computer device maps the bounding box of the human target to the image frame, it checks whether the center point of the bounding box in the image frame is located within a preset area of ​​the image frame. If yes, the human image is cropped based on the bounding box of the human target; otherwise, the human target is skipped. The preset area includes the center point of the image frame. Based on this, accurate identification of the long jumper is ensured.

[0123] Optionally, in one feasible embodiment of the present invention, step S124 includes:

[0124] Based on the preset target tracking module, feature extraction is performed on the region image corresponding to the bounding box of the human target to obtain the feature label of each human target;

[0125] Use the feature label of each human target in the first image frame of all image frames as the target label;

[0126] For each image frame after the first image frame, the feature label with the highest cosine similarity to the target label among the feature labels of each human target in the image frame is taken as the similar label of the target label;

[0127] The human targets corresponding to the target label and the human targets corresponding to similar labels of the target label are set to the same category to obtain the human targets of each category in each image frame.

[0128] In other words, in this feasible method, the present invention will use Reid to complete target tracking. That is, after determining the human target of each category in the first image frame, for each image frame after the first image frame, the computer device only detects whether the human target in the current image frame is the same as or similar to the human target in the first image frame. If so, the human target in the current image frame will be regarded as the human target in the first image frame.

[0129] Specifically, in this feasible approach, the computer device extracts features from the region image of each human target in each image frame and uses the extraction results as the feature label, or Reid, for each human target. Simultaneously, if the current image frame is the first image frame, the feature label of each human target in the first image frame is used as the target label.

[0130] Then, for each image frame after the first image frame, the feature label with the highest cosine similarity to the target label among the feature labels of each human target in the image frame is taken as the similar label of the target label.

[0131] Finally, the human target corresponding to each target label and the human target corresponding to the feature label of the target label are set to the same category, that is, the human target corresponding to the feature label of the target label is consistent with the human target corresponding to the target label.

[0132] Based on this, in this feasible method, the computer device only needs to detect whether the human target is similar, without needing to identify the specific category of the human target. Therefore, it can complete target tracking with less resources, realizing the lightweighting of long jump distance calculation.

[0133] Optionally, in one feasible embodiment of the present invention, please refer to the following: Figure 3 The diagram shows a flowchart of S130 provided in an embodiment of the present invention. In this feasible method, the size of the image frame is consistent with the size of the depth image of the image frame, and the pixel information of the pixels of the depth image includes depth.

[0134] Furthermore, S130 includes:

[0135] S131, for each of the image frames, determine the region depth image of each human target in the depth image of the image frame based on the bounding box of each human target in the image frame;

[0136] S132, based on a first preset formula, calculate the first average depth of the region depth image of the human target according to the depth of each pixel in the region depth image of the human target, wherein the first preset formula includes:

[0137]

[0138] In the formula, d mean Let d represent the first depth mean of the human target, n represent the number of pixels included in the depth image of the human target region, and d represent the depth mean of the human target region. i This represents the depth of the i-th pixel in the depth image of the human target region;

[0139] S134, based on the second preset formula, calculate the pixel segmentation threshold of the region depth image of each human target, wherein the second preset formula includes:

[0140]

[0141] In the formula, σ represents the pixel segmentation threshold of the region depth image, and d j This represents the depth of the j-th pixel in the depth image of the human target region;

[0142] S135, for each region depth image of the human target, according to the first preset formula and the pixels in the region depth image whose difference from the mean depth of the region depth image is less than or equal to the pixel segmentation threshold of the region depth image, calculate the second depth of the region depth image, and use the second depth as the position of the human target corresponding to the region depth image.

[0143] It should be understood that in this feasible approach, in addition to using the depth mean as the position of the human target relative to the camera, the computer device will also remove information of irrelevant pixels in the depth image of the region corresponding to the human target in order to improve the accurate identification of the human target's position.

[0144] Specifically, since the image frame and the depth image of the image frame have the same size, the pixels in the image frame correspond one-to-one with the pixels in the depth image of the image frame. Then, the computer device determines the position of the human target in the depth image of the image frame based on the bounding box output by the target detection model, that is, based on the position of the human target in the image frame, and thus obtains the region corresponding to the human target in the depth image, that is, the region depth image.

[0145] Next, based on the depth of each pixel in the region depth image, the average depth of all pixels in the region depth image is calculated.

[0146] Subsequently, according to the second preset formula, the pixel segmentation threshold is calculated, and the pixel segmentation threshold is used to determine the pixels in the region depth image whose depth difference from the mean depth is less than or equal to the pixel segmentation threshold. The mean depth is then calculated again to obtain the second mean depth, which is then used as the position of the human target relative to the camera.

[0147] It is understood that the embodiments of the present invention cluster the pixels in the region depth image based on the pixel segmentation threshold, so that pixels that are not related to the long jump target are removed, thereby ensuring that the effective pixel depth is used when the position of the long jump target is determined.

[0148] Optionally, in one feasible embodiment of the present invention, step S120 includes:

[0149] The image frame is scaled and normalized to obtain a preprocessed image frame;

[0150] Based on the preset target detection module and target tracking module, each category of human target is identified in each preprocessed image frame.

[0151] In other words, this feasible method preprocesses the input images of the target detection module to process input images of the same or different sizes into images of the same size, where the pixel value of each pixel is within a preset range. This enables the target detection module to perform target detection based on input images of the same size and similar data size, thereby avoiding the decrease in target detection accuracy due to inconsistent output image size and / or differences in units.

[0152] Preferably, in one feasible method provided by the embodiments of the present invention, the preprocessed image frame needs to have the same pixel distribution as the data of the sample during the training of the target detection module, and the same size as the sample, so as to improve the image detection accuracy.

[0153] Furthermore, it is understandable that the images input to the target tracking module can also undergo corresponding preprocessing operations to ensure that the input images and sample images are similar / consistent in image size and / or data dimensions, thereby guaranteeing the accuracy of target tracking.

[0154] Example 2

[0155] Corresponding to the method for obtaining long jump distance provided in Embodiment 1 of the present invention, Embodiment 2 of the present invention also provides a device for calculating long jump distance, referring to... Figure 4 This diagram illustrates the structure of a method and apparatus for obtaining long jump distance provided in an embodiment of the present invention. The long jump distance calculation apparatus 200 provided in an embodiment of the present invention includes:

[0156] The acquisition module 210 is used to acquire image frames and depth images of the image frames in real time;

[0157] The recognition module 220 is used to identify human targets of each category in each image frame by utilizing a preset target detection module and target tracking module;

[0158] The determining module 230 is used to determine the position of each category of human target in each image frame based on the depth image of each image frame;

[0159] The distance calculation module 240 is used to calculate the movement distance of each category of human target in each image frame relative to the previous image frame based on the position of each category of human target in each image frame.

[0160] The jump determination module 250 is used to determine if, in the current image frame, there is a human target of a certain category whose moving distance is greater than a preset value, the human target of the category whose moving distance is greater than a preset value is taken as the long jump target, and the previous image frame of the current image frame is taken as the jump frame of the long jump target.

[0161] The stop determination module 260 is used to determine the stop frame of the long jump target if, for the image frame following the take-off frame, the distance the long jump target moves in the preceding image frame is greater than the distance it moves in the following image frame.

[0162] The calculation module 270 calculates the long jump distance of the long jump target based on the position of the long jump target in the corresponding take-off frame and the corresponding stop frame.

[0163] Optionally, in one feasible embodiment of the present invention, the computing module includes:

[0164] The first long jump distance calculation submodule is used to calculate the long jump distance of the long jump target based on the preset start position and the position of the long jump target in the corresponding stop frame if a preset start position exists.

[0165] The second long jump distance calculation submodule is used to calculate the difference between the position of the long jump target in the corresponding start frame and the corresponding stop frame if the preset start position does not exist, so as to obtain the long jump distance of the long jump target.

[0166] Optionally, in one feasible embodiment of the present invention, the identification module and the jump determination module are further configured to, if in the current image frame, there exists a category of human target whose moving distance is greater than a preset value, and the position of the human target of the category is within a preset jump position range, then take the human target of the category whose moving distance is greater than the preset value as the long jump target, and take the previous image frame of the current image frame as the jump frame of the long jump target.

[0167] Optionally, in one feasible embodiment of the present invention, the identification module includes:

[0168] The target detection submodule is used to input each image frame into a preset target detection module to identify each category of target and the bounding box of each target in each image frame;

[0169] The non-maximum suppression submodule is used to merge the bounding boxes of the same target in each image frame into a single bounding box based on a preset non-maximum suppression module.

[0170] The cropping submodule is used to crop the image frame corresponding to the human target based on the bounding box of the human target classified as human body, so as to obtain the human image of the human target.

[0171] The target tracking submodule is used to input the human image of each image frame into a preset target tracking module to identify human targets of each category in each image frame.

[0172] Optionally, in one feasible embodiment of the present invention, the target tracking submodule includes:

[0173] The feature extraction unit is used to extract features from the region image corresponding to the bounding box of the human target based on the preset target tracking module, so as to obtain the feature label of each human target.

[0174] The target label determination unit is used to take the feature label of each human target in the first image frame of all image frames as the target label;

[0175] The similarity label determination unit is used to, for each image frame after the first image frame, take the feature label with the highest cosine similarity to the target label among the feature labels of each human target in the image frame as the similar label of the target label;

[0176] The category setting unit is used to set the human targets corresponding to the target label and the human targets corresponding to similar labels of the target label to the same category, so as to obtain the human targets of each category in each image frame.

[0177] Optionally, in one feasible embodiment of the present invention, the size of the image frame is consistent with the size of the depth image of the image frame, and the pixel information of the pixels in the depth image includes depth;

[0178] Furthermore, the determining module includes:

[0179] The image acquisition submodule is used to determine the region depth image of each human target in the depth image of each image frame based on the bounding box of each human target in the image frame;

[0180] The first mean calculation submodule is used to calculate the first mean depth of the region depth image of the human target based on a first preset formula, according to the depth of each pixel in the region depth image of the human target. The first preset formula includes:

[0181]

[0182] In the formula, d mean Let d represent the first depth mean of the human target, n represent the number of pixels included in the depth image of the human target region, and d represent the depth mean of the human target region. i This represents the depth of the i-th pixel in the depth image of the human target region;

[0183] The threshold calculation submodule is used to calculate the pixel segmentation threshold of the region depth image of each human target based on a second preset formula, wherein the second preset formula includes:

[0184]

[0185] In the formula, σ represents the pixel segmentation threshold of the region depth image, and d j This represents the depth of the j-th pixel in the depth image of the human target region;

[0186] The second mean calculation submodule is used to calculate the second depth of the region depth image for each human target region depth image according to the first preset formula and the pixels in the region depth image whose difference from the mean depth of the region depth image is less than or equal to the pixel segmentation threshold of the region depth image, and use the second depth as the position of the human target corresponding to the region depth image.

[0187] Optionally, in one feasible embodiment of the present invention, the identification module includes:

[0188] The preprocessing submodule is used to scale and normalize the image frame to obtain a preprocessed image frame;

[0189] The input submodule is used to identify human targets of each category in each preprocessed image frame according to the preset target detection module and target tracking module.

[0190] The long jump distance calculation device 200 provided in this application embodiment can implement each process of the long jump distance acquisition method corresponding to Embodiment 1, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0191] This invention also provides a computer device, including a memory and a processor. The memory stores a computer program, and the computer program executes the method for obtaining the long jump distance as described in Embodiment 1 when it runs on the processor.

[0192] This invention also provides a computer-readable storage medium storing a computer program, which executes the method for obtaining the long jump distance as described in Embodiment 1 when run on a processor.

[0193] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that, as an alternative implementation, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0194] In addition, the functional modules or units in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0195] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a smartphone, personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0196] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for obtaining long jump distance, characterized in that, include: Real-time acquisition of image frames and depth images of those image frames; Using a pre-set target detection module and target tracking module, human targets of each category in each image frame are identified; Based on the depth image of each image frame, determine the location of the human target of each category in each image frame; Based on the position of the human target of each category in each image frame, calculate the distance moved by the human target of each category in each image frame relative to the previous image frame; If, in the current image frame, there exists a category of human targets whose moving distance is greater than a preset value, and the position of the human targets of that category is within a preset starting position range, then the human targets of that category whose moving distance is greater than the preset value are taken as long jump targets, and the previous image frame of the current image frame is taken as the starting frame of the long jump targets. For the image frames following the take-off frame, if the distance the long jump target moves in the preceding image frame is greater than the distance it moves in the following image frame, then the preceding image frame is taken as the stop frame of the long jump target. The long jump distance of the long jump target is obtained based on the position of the long jump target in the corresponding take-off frame and the corresponding stop frame; The step of identifying human targets of each category in each image frame using a preset target detection module and target tracking module includes: Each image frame is input to a preset target detection module to identify each category of target and the bounding box of each target in each image frame; Based on the preset nonmaximum suppression module, the bounding boxes of the same target in each image frame are merged into one bounding box; Based on the bounding box of the human target classified as human, the image frame corresponding to the human target is cropped to obtain the human image of the human target. The human image of each image frame is input into a preset target tracking module to identify human targets of each category in each image frame; Determining the location of each category of human target in each image frame based on the depth image of each image frame includes: For each image frame, based on the bounding box of each human target in the image frame, determine the region depth image of each human target in the depth image of the image frame; Based on a first preset formula, a first average depth value of the depth image of the human target is calculated according to the depth of each pixel in the depth image of the human target region, wherein the first preset formula includes: In the formula, d mean This represents the first average depth of the human target. n This indicates the number of pixels included in the depth image of the human target region. d i The region depth image representing the human target is the first i The depth of each pixel; Based on a second preset formula, a pixel segmentation threshold for the region depth image of each human target is calculated, wherein the second preset formula includes: In the formula, σ This represents the pixel segmentation threshold of the region depth image. d j The region depth image representing the human target is the first j The depth of each pixel; For each region depth image of the human target, a second depth of the region depth image is calculated based on the first preset formula and the pixels in the region depth image whose difference from the mean depth of the region depth image is less than or equal to the pixel segmentation threshold of the region depth image. The second depth is then used as the position of the human target corresponding to the region depth image.

2. The method for obtaining long jump distance according to claim 1, characterized in that, The step of obtaining the long jump distance of the long jump target based on the position of the long jump target in the corresponding take-off frame and the corresponding stop frame includes: If a preset take-off position exists, the jump distance of the long jump target is calculated based on the preset take-off position and the position of the long jump target in the corresponding stop frame. If the preset take-off position does not exist, the difference between the position of the long jump target in the corresponding take-off frame and the corresponding stop frame is calculated to obtain the long jump distance of the long jump target.

3. The method for obtaining long jump distance according to claim 1, characterized in that, The step of inputting the human image of each image frame into a preset target tracking module to identify human targets of each category in each image frame includes: Based on the preset target tracking module, feature extraction is performed on the region image corresponding to the bounding box of the human target to obtain the feature label of each human target; Use the feature label of each human target in the first image frame of all image frames as the target label; For each image frame after the first image frame, the feature label with the highest cosine similarity to the target label among the feature labels of each human target in the image frame is taken as the similar label of the target label; The human targets corresponding to the target label and the human targets corresponding to similar labels of the target label are set to the same category to obtain the human targets of each category in each image frame.

4. The method for obtaining long jump distance according to claim 1, characterized in that, The step of identifying human targets of each category in each image frame using a preset target detection module and target tracking module includes: The image frame is scaled and normalized to obtain a preprocessed image frame; Based on the preset target detection module and target tracking module, each category of human target is identified in each preprocessed image frame.

5. A device for calculating long jump distance, characterized in that, include: The acquisition module is used to acquire image frames and depth images of the image frames in real time; The recognition module is used to identify human targets of each category in each image frame by utilizing a preset target detection module and target tracking module; The determination module is used to determine the location of each category of human target in each image frame based on the depth image of each image frame; The distance calculation module is used to calculate the movement distance of each category of human target in each image frame relative to the previous image frame, based on the position of each category of human target in each image frame. The jump determination module is used to determine if, in the current image frame, there is a type of human target whose moving distance is greater than a preset value, and the position of the human target of the type is within a preset jump position range, then the human target whose moving distance is greater than the preset value is taken as the long jump target, and the previous image frame of the current image frame is taken as the jump frame of the long jump target. The stop determination module is used to determine the stop frame of the long jump target if, for the image frame following the take-off frame, the distance the long jump target moves in the preceding image frame is greater than the distance it moves in the following image frame. The calculation module is used to obtain the long jump distance of the long jump target based on the position of the long jump target in the corresponding take-off frame and the corresponding stop frame; The step of identifying human targets of each category in each image frame using a preset target detection module and target tracking module includes: Each image frame is input to a preset target detection module to identify each category of target and the bounding box of each target in each image frame; Based on the preset nonmaximum suppression module, the bounding boxes of the same target in each image frame are merged into one bounding box; Based on the bounding box of the human target classified as human, the image frame corresponding to the human target is cropped to obtain the human image of the human target. The human image of each image frame is input into a preset target tracking module to identify human targets of each category in each image frame; Determining the location of each category of human target in each image frame based on the depth image of each image frame includes: For each image frame, based on the bounding box of each human target in the image frame, determine the region depth image of each human target in the depth image of the image frame; Based on a first preset formula, a first average depth value of the depth image of the human target is calculated according to the depth of each pixel in the depth image of the human target region, wherein the first preset formula includes: In the formula, d mean This represents the first average depth of the human target. n This indicates the number of pixels included in the depth image of the human target region. d i The region depth image representing the human target is the first i The depth of each pixel; Based on a second preset formula, a pixel segmentation threshold for the region depth image of each human target is calculated, wherein the second preset formula includes: In the formula, σ This represents the pixel segmentation threshold of the region depth image. d j The region depth image representing the human target is the first j The depth of each pixel; For each region depth image of the human target, a second depth of the region depth image is calculated based on the first preset formula and the pixels in the region depth image whose difference from the mean depth of the region depth image is less than or equal to the pixel segmentation threshold of the region depth image. The second depth is then used as the position of the human target corresponding to the region depth image.

6. A computer device, characterized in that, The device includes a memory and a processor, wherein the memory stores a computer program that, when executed on the processor, performs the method for obtaining the long jump distance as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed on a processor, performs the method for obtaining the long jump distance as described in any one of claims 1-4.

Images