Goal detection method and related equipment thereof

Abstract

The invention relates to a goal detection method and related equipment thereof. The method comprises the following steps: obtaining candidate image frames in a video shot by a camera; calculating a first intersection-to-union ratio of a basketball bounding box and a basket bounding box in the candidate image frame and a second intersection-to-union ratio of the basketball bounding box and a net bounding box; under the condition that the first intersection-to-union ratio and the second intersection-to-union ratio meet preset conditions, the relative distance between the basketball and the basket in the candidate image frame is determined, and the relative distance is the difference value between the first distance between the basketball and the camera and the second distance between the basket and the camera in the candidate image frame; and under the condition that the target image frame with the relative distance smaller than the corresponding distance threshold exists in the candidate image frames, determining that the goal exists. According to the method and the device, the technical problem of low goal detection accuracy caused by misjudgment of goal due to lack of depth information in the image in the video shot by the monocular camera in a way of detecting the goal by using the video shot by the monocular camera is solved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of goal detection, and in particular to a goal detection method and related equipment. Background Art

[0002] In the automatic editing of basketball videos, it is necessary to obtain the goal frames in the video stream and then extract the goal clips. Extracting goal frames from the video stream is generally done by analyzing the distance between the basketball and the basket or net in the image frame by frame. Because the video image is a 2D space, there will be false detections in the direction where the basketball and the basket or net overlap. Summary of the invention

[0003] According to one aspect of an embodiment of the present disclosure, a goal detection method is provided, comprising: obtaining candidate image frames in a video captured by a camera, wherein the candidate image frames are image frames in the video including a basketball bounding box, a basket frame bounding box, and a net bounding box; calculating a first intersection-and-union ratio between the basketball bounding box and the basket frame bounding box, and a second intersection-and-union ratio between the basketball bounding box and the net bounding box in the candidate image frames; determining a relative distance between the basketball and the basket frame in the candidate image frames when the first intersection-and-union ratio and the second intersection-and-union ratio satisfy a preset condition, wherein the relative distance is a difference between a first distance from the basketball to the camera and a second distance from the basket frame to the camera in the candidate image frames; determining that a goal exists when there is a target image frame in the candidate image frames in which the relative distance is less than a corresponding distance threshold.

[0004] According to another aspect of an embodiment of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor implements the above method when executing the computer program.

[0005] According to another aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and the computer program implements the above method when executed by a processor.

[0006] According to another aspect of the embodiments of the present disclosure, a computer program product is provided, including a computer program / instruction, and when the computer program / instruction is executed by a processor, the above method is implemented.

[0007] The goal detection method proposed in the embodiment of the present disclosure uses the method of judging the relative distance between the basketball and the basket to further judge whether there is a goal frame in the basketball video frame, thereby reducing the situation of misjudging goals in scenes with overlapping perspectives. In particular, for basketball videos where the image lacks depth information, it is easy to misjudge goals by using the intersection-over-union judgment method alone. The method described in the embodiment of the present disclosure can effectively improve the judgment of goal frames in basketball videos that lack depth information.

[0008] The details of one or more embodiments of the present disclosure are set forth in the following drawings and description to make other features, objects, and advantages of the present disclosure more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other embodiments can be obtained based on these drawings without creative work.

[0010] Figure 1A is a schematic diagram of a basketball, a basket frame, and a net having the same y-axis distance as a monocular camera for detecting a goal according to the prior art; Figure 1B It is a schematic diagram of a basketball, a basket frame, and a net with different y-axis distances using a monocular camera to detect a goal according to the prior art; Figure 2 is a flow chart of a goal detection method according to an embodiment of the present disclosure; Figure 3 is a flowchart of goal frame acquisition according to an embodiment of the present disclosure; Figure 4A is a schematic diagram of a relative position of a basketball bounding box and a basket hoop bounding box according to an embodiment of the present disclosure; Figure 4B is a schematic diagram of another relative position of a basketball bounding box and a basket hoop bounding box according to an embodiment of the present disclosure; Figure 4C is a schematic diagram of another relative position of a basketball bounding box and a basket hoop bounding box according to an embodiment of the present disclosure; Figure 4D is a schematic diagram of another relative position of a basketball bounding box and a basket hoop bounding box according to an embodiment of the present disclosure; Figure 4E is a schematic diagram of another relative position of a basketball bounding box and a basket hoop bounding box according to an embodiment of the present disclosure; Figure 4F is a schematic diagram of another relative position of a basketball bounding box and a basket hoop bounding box according to an embodiment of the present disclosure; Figure 5 is a flowchart of detailed steps of goal detection according to an embodiment of the present disclosure; Fig. 6A is a schematic diagram of a relative distance calculation principle according to an embodiment of the present disclosure; Figure 6B is a schematic diagram of the actual relative distance according to an embodiment of the present disclosure; Fig. 7A is a schematic diagram of a distance threshold between a basketball and a basket according to an embodiment of the present disclosure; Figure 7B is a schematic diagram of another distance threshold between a basketball and a basket according to an embodiment of the present disclosure; Figure 7C is a schematic diagram of another distance threshold between a basketball and a basket according to an embodiment of the present disclosure; Figure 8 Schematic diagram of the structure of the electronic device of this embodiment. DETAILED DESCRIPTION

[0011] Embodiments of the present embodiment will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present embodiment are shown in the accompanying drawings, it should be understood that the present embodiment can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein, which are instead provided for a more thorough and complete understanding of the present embodiment. It should be understood that the drawings and embodiments of the present embodiment are only for exemplary purposes and are not intended to limit the scope of protection of the present embodiment.

[0012] In the related art, videos recorded with a monocular camera lack image depth information. For example, when extracting goal frames from basketball videos recorded with a monocular camera, due to the lack of image depth information, it may appear that the basketball, the basket, and the net overlap in the image, but in fact the basketball, the basket, and the net are at different depths in the image, which leads to misjudgment of goal frames and low accuracy of goal detection.

[0013] It should be noted that a monocular camera usually refers to a camera system with only one lens. It uses one lens to capture the environment and convert it into a two-dimensional image, but the two-dimensional image captured by a monocular camera often lacks depth information. A binocular camera usually refers to a camera system consisting of two parallel lenses. By calculating the parallax of the two lenses for the same object, the distance of the object from the camera system can be calculated. However, binocular cameras also have certain defects. For example, additional lenses need to be added, especially when used in portable electronic devices such as mobile phones or wearable devices, which increases costs and makes structural design difficult.

[0014] For example, see Appendix Figure 1A and Figure 1B As shown in the figure, when a monocular camera is shooting a goal video, the basketball can also be photographed by facing the basket and the net at a certain angle when the basketball enters the shooting area. However, since the monocular camera shoots two-dimensional images, the same or very similar photos may be taken in different situations.

[0015] like Figure 1A As shown in the figure, the y-axis distances of the basketball, the basket, and the net relative to the monocular camera are the same or very close, and the basketball falls into the basket. At this time, the image captured by the monocular camera can show that the basketball falls into the basket, which is consistent with the actual situation. Figure 1B As shown in the figure, the y-axis distances between the basketball and the basket and net relative to the monocular camera are quite different, and the basketball did not fall into the basket. At this time, the image captured by the monocular camera still shows that the basketball fell into the basket, which is inconsistent with the actual situation.

[0016] This is because the picture taken by a monocular camera only includes information on the x-axis and z-axis, but very little information on the y-axis. Therefore, the method of detecting goals using videos taken by a monocular camera in the related art is prone to misjudgment of goals due to the lack of depth information in the images in the videos taken by the monocular camera, resulting in a technical problem of low goal detection accuracy.

[0017] In order to solve this problem, a relevant solution is provided in the embodiments of the present disclosure, which is described in detail below.

[0018] Figure 2 is a flow chart of a goal detection method according to an embodiment of the present disclosure. Figure 2 As shown, the method comprises the following steps: Step S201, obtaining a candidate image frame in a video captured by a camera, wherein the candidate image frame is an image frame in the video including a basketball boundary frame, a basket frame boundary frame, and a net boundary frame; Step S202, calculating a first intersection-and-union ratio between the basketball bounding box and the basket frame bounding box, and a second intersection-and-union ratio between the basketball bounding box and the net bounding box in the candidate image frame; Step S203, when the first intersection-and-union ratio and the second intersection-and-union ratio meet a preset condition, determining a relative distance between the basketball and the basket in the candidate image frame, wherein the relative distance is a difference between a first distance from the basketball to the camera and a second distance from the basket to the camera in the candidate image frame; Step S204: when there is a target image frame whose relative distance is less than the corresponding distance threshold in the candidate image frames, it is determined that there is a goal.

[0019] The goal detection method provided in this embodiment characterizes whether there may be a goal-scoring behavior in the candidate image frame by calculating the first intersection-and-union ratio between the basketball bounding box and the basket frame bounding box, and the second intersection-and-union ratio between the basketball bounding box and the net bounding box in the candidate image frame when the camera captures a video related to basketball sports.

[0020] When the first intersection-and-union ratio and the second intersection-and-union ratio meet the preset conditions, it is determined that there may be a goal-scoring behavior in the candidate image frame. Then the difference between the first distance from the basketball to the camera and the second distance from the basket to the camera in the candidate image frame is determined, and the difference is used to determine whether the basketball and the basket are close enough in the depth direction of the shooting space represented by the candidate image frame to indicate whether there is a goal. The depth direction can be the y-axis direction mentioned above. That is, when there is a target image frame in the candidate image frame where the relative distance is less than the corresponding distance threshold, it is determined that there is a goal-scoring behavior.

[0021] It can be understood that the intersection-and-union ratio represents the ratio of the intersection and union of two image detection frames. When there is no intersection between the two image detection frames, the intersection-and-union ratio is 0, and when the two image detection frames are the same size and completely overlap, the intersection-and-union ratio is 1. The first intersection-and-union ratio represents the ratio of the intersection and union of the basketball bounding box and the basket bounding box, and the second intersection-and-union ratio represents the ratio of the intersection and union of the basketball bounding box and the net bounding box.

[0022] The above distance threshold indicates the maximum distance between the basketball and the basket in the depth direction of the corresponding shooting space when the basketball falls into the basket. When the distance between the basketball and the basket exceeds the distance threshold, it is considered that the basketball cannot fall into the basket, that is, there is no goal-scoring behavior in the candidate image frame.

[0023] Based on the method provided by the embodiment of the present disclosure, it is achieved that the difference between the first distance from the basketball to the camera and the second distance from the basket to the camera is used to accurately determine whether the basketball is in the basket, and then accurately detect whether there is a goal, thereby improving the technical effect of the accuracy of goal detection. This solves the technical problem that the method of using a video of a monocular camera to detect a goal in the related art is prone to misjudgment due to the lack of depth information, resulting in a low accuracy rate of goal detection.

[0024] It should be noted that the method described in the present disclosure solves the technical problem that the basketball image taken by a monocular camera may cause the judgment of the goal frame to be wrong due to the absence of depth information. However, the method described in the present disclosure can still be applied to, for example, the extraction of goal frames of basketball images / videos taken by a binocular camera. In other words, whether there is sufficient depth information in the video or image does not affect the use of the method described in the present disclosure. Those skilled in the art can apply the exemplary method described in the present disclosure to other suitable videos or images to extract goal frames without any creative work.

[0025] The execution subject of any example step described in the present disclosure may be an electronic device including a camera (e.g., a monocular camera), such as a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc.

[0026] By processing the video data collected by the camera, the first intersection-and-union ratio between the basketball bounding box and the basket frame bounding box in the candidate image frame, and the second intersection-and-union ratio between the basketball bounding box and the net bounding box are calculated. To determine whether there is a possibility of a goal in the video image. In the case of a possibility of a goal, the relative distance between the basketball and the basket frame in the candidate image frame is calculated. The example disclosed in this disclosure uses the difference between the first distance from the basketball to the camera and the second distance from the basket frame to the camera as the relative distance. Whether there is a goal is determined based on whether the relative distance is less than a distance threshold.

[0027] The camera may be a monocular camera, and the image captured by the monocular camera lacks depth information. The video acquired by the camera may be a video related to basketball. The candidate image frame may be an image frame including a basketball bounding box, a basket bounding box, and a net bounding box in the basketball-related video.

[0028] It is understandable that not every image frame in a basketball-related video includes a basketball, a basket hoop, and a net. Some image frames may not have any of the above three, while other image frames may include one or two of the above three. However, image frames related to goal-scoring will definitely include a basketball, a basket hoop, and a net.

[0029] Therefore, the candidate image frame can be determined based on whether the basketball bounding box, the basket frame bounding box and the net bounding box exist simultaneously in each image frame, and whether the candidate image frame may be a goal frame can be judged based on the first intersection-and-union ratio of the basketball bounding box and the basket frame bounding box and the second intersection-and-union ratio of the basketball bounding box and the net bounding box.

[0030] The increase of the first intersection-and-union ratio indicates that the overlap between the basketball bounding box and the basket frame bounding box increases, and the decrease of the first intersection-and-union ratio indicates that the overlap between the basketball bounding box and the basket frame bounding box decreases. The increase of the second intersection-and-union ratio indicates that the overlap between the basketball bounding box and the net bounding box increases, and the decrease of the second intersection-and-union ratio indicates that the overlap between the basketball bounding box and the net bounding box decreases. When a goal is scored, the basketball gradually falls into the basket, then falls into the net, and gradually moves away from the basket and the net. After completely moving away from the basket, it continues to gradually move away from the net until it leaves the net.

[0031] In the above-mentioned goal-scoring process, the first intersection-in-union ratio first gradually increases to a maximum value, and then gradually decreases to zero. The second intersection-in-union ratio also first gradually increases to a maximum value, and then gradually decreases to zero, lagging behind the change of the first intersection-in-union ratio. However, due to the lack of depth information, the changes in the first intersection-in-union ratio and the second intersection-in-union ratio can only characterize whether there is a possibility of scoring a goal to a certain extent.

[0032] That is, the change of the first intersection-and-union ratio can explain the change of the relative position of the basketball and the basket, and the second intersection-and-union ratio can explain the change of the relative position of the basketball and the net. According to whether the change trend of the first intersection-and-union ratio and the second intersection-and-union ratio conforms to the change of the relative position of the basketball and the basket net in real situation, it is judged whether there is a possibility of goal.

[0033] Similar to the change of the first intersection-and-union ratio, the change trend of the second intersection-and-union ratio between the bounding box of the basketball and the bounding box of the net can be used to determine whether a goal is likely to be scored, based on whether the change trend of the second intersection-and-union ratio between the bounding box of the basketball and the bounding box of the net is consistent with the change of the relative position of the basketball and the net in the real situation. Since the basketball first enters the basket and then the net when a goal is scored, it is also possible to determine whether the change of the intersection-and-union ratio of the basketball relative to the basket and the net is consistent with the situation by combining whether the change trend of the second intersection-and-union ratio is later than the change trend of the first intersection-and-union ratio, and then determine whether a goal is likely to be scored.

[0034] When the first intersection-and-union ratio and the second intersection-and-union ratio meet the preset conditions, it can be determined that there is a possibility of scoring a goal. The preset conditions can be set according to actual conditions, and can be one or more of the conditions of whether the first intersection-and-union ratio and the second intersection-and-union ratio meet the actual conditions.

[0035] If the first intersection and union ratio and the second intersection and union ratio do not meet the preset conditions, there is no possibility of a goal, and the output is directly that there is no goal. This can more quickly eliminate the situation where there is no goal.

[0036] When it is determined that a goal is possible, the relative distance between the basketball and the basket in the candidate image frame is determined, and the relative distance is the difference between the first distance from the basketball to the camera and the second distance from the basket to the camera in the candidate image frame. Figure 6B As shown, the distance between the center of mass of the basketball and the center of the basket can be used to represent the relative distance, and the distance needs to be less than the distance threshold to be counted as a goal.

[0037] The method for selecting the distance threshold will be described in detail later.

[0038] Through the above steps, it is possible to accurately determine whether there is a goal in the video captured by the camera. This solves the technical problem that the method of detecting goals using videos captured by a monocular camera in the related art easily leads to misjudgment of goals due to the lack of depth information in the images in the videos captured by the monocular camera, resulting in low accuracy of goal detection.

[0039] As an optional embodiment, determining whether the first intersection-and-union ratio and the second intersection-and-union ratio meet preset conditions includes: when the maximum value of the first intersection-and-union ratio reaches the corresponding first threshold, the second intersection-and-union ratio is greater than zero and increases from zero to the maximum value, and the maximum value of the second intersection-and-union ratio reaches the corresponding second threshold, determining that the preset conditions are met.

[0040] In order to ensure accurate exclusion of the situation where there is no goal frame, this embodiment takes into account the three situations of whether the first intersection-and-union ratio and the second intersection-and-union ratio are consistent with the actual situation. That is, if the change trend of the first intersection-and-union ratio is consistent with the change of the relative position of the basketball and the basket frame in the case of a real goal, and the change trend of the second intersection-and-union ratio is consistent with the change of the relative position of the basketball and the basket net in the case of a real goal, and the change trend of the second intersection-and-union ratio is later than the change trend of the first intersection-and-union ratio, it is determined that there is a possibility of a goal, otherwise, there is no possibility of a goal.

[0041] That is, when the changing trend of the first intersection-and-union ratio is from zero to a maximum value and then gradually decreases to zero, the maximum value of the first intersection-and-union ratio reaches the corresponding first threshold value, indicating that the first intersection-and-union ratio is in line with the actual situation. And when the maximum value of the first intersection-and-union ratio reaches the corresponding first threshold value, the second intersection-and-union ratio is greater than zero, indicating that the timing of the change of the second intersection-and-union ratio lags behind the timing of the change of the first intersection-and-union ratio, that is, in the goal-scoring behavior, the basketball first falls into the basket, and then falls from the basket into the net. And, the changing trend of the second intersection-and-union ratio is from zero to a maximum value and then gradually decreases to zero. At this time, the maximum value of the second intersection-and-union ratio reaches the corresponding second threshold value, indicating that the second intersection-and-union ratio is in line with the actual situation. When the above conditions are met at the same time, it can be determined that the preset conditions are met.

[0042] As an optional embodiment, after determining that there is a goal, the method further includes: acquiring an image frame at a moment when the second intersection-over-union ratio reaches a maximum value as a goal frame, wherein the goal frame is used to display the goal.

[0043] After determining the goal, the image frame at the moment when the second intersection-and-union ratio reaches the maximum value can be obtained as the goal frame. The maximum value of the second intersection-and-union ratio indicates that the basketball has completely fallen into the net, which is more effective and intuitive in expressing the goal action.

[0044] As an optional embodiment, determining the relative distance between a basketball and a basket frame in a candidate image frame includes: performing image segmentation on the candidate image frame to obtain a basketball pixel area and a basket frame pixel area; determining a first diameter of the basketball in the candidate image frame based on the basketball pixel area, and determining a second diameter of the basket frame in the candidate image frame based on the basket frame pixel area; obtaining a first distance based on camera focal length information, the first diameter, and a real basketball diameter, and obtaining a second distance based on the focal length information, the second diameter, and a real basket frame diameter; and determining a difference based on the first distance and the second distance.

[0045] The above-mentioned image segmentation of the candidate image frame to obtain the basketball pixel area and the basket frame pixel area can use the image segmentation model to segment the basketball pixel area from the ROI area of ​​the basketball boundary box and convert it into a basketball mask, that is, the basketball pixel area. The pixel area of ​​the basket frame is segmented from the ROI area of ​​the basket frame boundary box and converted into a basket frame mask, that is, the basket frame pixel area.

[0046] Specifically, as an optional embodiment, the candidate image frame is segmented to obtain a basketball pixel area and a basket frame pixel area, including: detecting the basketball bounding box and the basket frame bounding box in the candidate image frame through a detection model; determining and segmenting the area of ​​interest of the basketball bounding box and the area of ​​interest of the basket frame bounding box; based on the area of ​​interest of the basketball bounding box and the basket frame bounding box, detecting and segmenting the basketball pixel area and the basket frame pixel area in the candidate image frame through a segmentation model.

[0047] The above segmentation model is used to segment the target in the image. Common image segmentation models include Mask R-CNN, Instance-sensitive FCN, FCIS, YOLCAT, SOLO, lendMask, BlendMask, etc. In the same way of transfer learning, the model is trained again using its own data set so that the model can segment the basketball pixel area and the basket pixel area in the image. In actual implementation, in order to speed up the segmentation speed of the segmentation model, the image can be segmented on the bounding box ROI (Region Of Interest) sub-image based on the target detection model.

[0048] like Fig. 6A and Figure 6B As shown, the diameter d1 of the basketball and the diameter d2 of the basket on the actual plane are both known, and the focal length of the camera is known. In order to calculate the first distance s1 from the basketball to the camera and the second distance s2 from the basket to the camera, it is necessary to first calculate the diameter d1' of the basketball on the pixel plane, which is the first diameter, and the diameter d2' of the basket, which is the second diameter.

[0049] The above-mentioned determination of the first diameter of the basketball in the candidate image frame based on the basketball pixel area and the determination of the second diameter of the basket frame in the candidate image frame based on the basket frame pixel area are both the sizes of the basketball and the basket frame on the pixel plane, which can be obtained by calculating the pixels in the pixel area. The details are as follows: As an optional embodiment, a first diameter of a basketball in a candidate image frame is determined based on a basketball pixel area, and a second diameter of a basket frame in a candidate image frame is determined based on a basket frame pixel area, including: converting a basketball pixel area into a basketball mask, and converting a basket frame pixel area into a basket frame mask; performing morphological processing on the basketball mask and the basket frame mask, and obtaining first contour information of the basketball and second contour information of the basket frame based on the processing results; extracting and generating a first circumscribed rectangle based on an outermost first contour of the basketball, and generating a second circumscribed rectangle based on an innermost second contour information of the basket frame; determining a first diameter of the basketball according to a size of the first circumscribed rectangle, and determining a second diameter of the basket frame according to a size of the second circumscribed rectangle.

[0050] The above conversion of the basketball pixel area into the basketball mask and the basket pixel area into the basket mask can directly output the mask corresponding to the pixel area through the above segmentation model. The above steps are only to illustrate the logical order between the steps. In implementation, two or even more steps may be directly obtained in one step.

[0051] The above-mentioned morphological processing of the basketball mask and the basket mask can first dilate the mask to remove the scattered black spots in the mask. Then perform a reverse corrosion process to restore the size of the mask to that before dilation, thereby improving the morphological performance of the mask. Then perform filtering. For example, median filtering, mean filtering, Gaussian filtering, etc. Through filtering, the burrs on the edge of the mask can be removed, which facilitates the subsequent extraction of contour information.

[0052] Based on the processing results of the basketball mask and the basket frame mask, first contour information of the basketball and second contour information of the basket frame are obtained respectively.

[0053] Since the basketball is an impenetrable sphere, the first circumscribed rectangle is extracted and generated based on the outermost first contour of the basketball, and the first diameter of the basketball is determined according to the size of the first circumscribed rectangle. The basket is hollow and can be penetrated, so the second circumscribed rectangle is generated based on the innermost second contour information of the basket, and the second diameter of the basket is determined according to the size of the second circumscribed rectangle. Thus, the first diameter of the basketball on the pixel plane and the second diameter of the basket are accurately calculated based on the actual situation.

[0054] As an optional embodiment, obtaining the first distance based on the focal length information of the camera, the first diameter, and the actual basketball diameter, and obtaining the second distance based on the focal length information, the second diameter, and the actual basketball frame diameter includes: calculating the first distance and the second distance based on the following formula: , where is the first distance or the second distance, is the focal length of the camera, is the actual diameter of the basketball or the actual diameter of the basket, The first diameter or the second diameter.

[0055] Principle Fig. 6A As shown, it is known that the length of a line segment in the pixel coordinate system is (unit is pixel), the length in the actual coordinate system is , the focal length of the device camera is .

[0056] Depend on It can be seen that: . (Use appropriate units of measurement such as mm for calculations).

[0057] In this embodiment, if Figure 6B As shown, the diameter of the basketball in the actual plane is , the diameter of the basketball on the pixel plane is , the focal length of the device is f, the above three values ​​are all known, and the calculation formula of s is used to obtain the first distance between the basketball and the camera in the actual plane. .

[0058] Similarly: The inner diameter of the basket in the actual plane is , the diameter of the basketball on the pixel plane is , the focal length of the device is f, the above three values ​​are all known, and the calculation formula of s is used to obtain the second distance between the basket and the camera in the actual plane .

[0059] The distance threshold can be the sum of the radius of the basketball and the radius of the basket. FIG. 7A to FIG. 7C shown. Fig. 7A This is the case where the center positions of the basketball and the basket are the same, in which case their relative distance is 0. Figure 7B As shown in , when the outside of the basketball contacts the inner circle of the basket, the relative distance is the inner radius of the basket minus the radius of the basketball. Figure 7C As shown, when the outer side of the basketball contacts the outer ring of the basket, the relative distance at this time is the inner radius of the basket plus the radius of the basketball.

[0060] Considering the limit case of basketball goal Figure 7C As shown, the center of the basketball and the basket is more than Figure 7C If the distance shown is less than 0.000, it is very likely that the goal has not been scored. Therefore, the distance threshold of this embodiment is selected as Figure 7C In the case shown, the distance threshold can be the sum of the radius of the basketball and the radius of the basket, thereby improving the accuracy of goal detection.

[0061] It should be noted that this embodiment also provides an optional implementation, which is described in detail below.

[0062] This embodiment proposes a basketball goal detection method based on a monocular device. The focal length of the device is obtained by calibrating the internal parameters of the monocular device. When using the device to detect the goal frame, firstly, a judgment is made based on the overlap between the basketball boundary box and the basket frame boundary box and the net boundary box, and then a judgment is made based on the relative distance between the basketball and the basket frame. Through the constraints of the second step, the misjudgment of the goal in the scene of overlapping perspectives can be reduced.

[0063] Figure 3 is a flowchart of obtaining a goal frame according to an embodiment of the present disclosure, such as Figure 3 As shown, the overall steps of this implementation are as follows: Step 1: First, calibrate the parameters of the monocular camera device to obtain the focal length of the monocular camera .

[0064] Step 2: Load the pre-set or trained image detection model and segmentation model.

[0065] Step 3: Obtain video frame images in real time through a monocular camera, and detect the basketball bounding box, the basket frame bounding box, and the net bounding box frame by frame to obtain candidate image frames including the basketball bounding box, the basket frame bounding box, and the net bounding box.

[0066] Step 4: First, check the overlap between the basketball bounding box, the basket frame bounding box, and the net bounding box. If the overlap meets the requirement, proceed to step 5, otherwise return to step 3 to continue to the next frame.

[0067] Step 5: Calculate the relative distance between the basketball and the basket. If the relative distance meets the requirement, it is considered that the goal frame can be detected. Otherwise, it is considered that there is no goal frame. Then return to step 3 to continue to the next frame.

[0068] Specifically, in the above step 1, when the device camera internal parameter calibration obtains the focal length of the device, it can be obtained by, for example, calling a system function, or obtaining the camera focal length value according to the specific structure of the camera. For example, when the camera is a mobile phone with a camera, the current focal length value is determined based on the preset relationship between the position of the voice coil motor in the mobile phone and the focal length value.

[0069] In the above step 2, the image detection model and segmentation model are as follows: Image detection models are used to detect objects in images and obtain the bounding box and category of the object. Commonly used image detection models include: R-CNN, Fast R-CNN, Faster R-CNN, Mask R-CNN, and YOLOV. You can use transfer learning to train the image detection model again using your own dataset so that the image detection model can detect objects such as the basketball bounding box, the basket hoop bounding box, and the net bounding box in the image.

[0070] The segmentation model is used to segment the target in the image. Common segmentation models include Mask R-CNN, Instance-sensitive FCN, FCIS, YOLCAT, SOLO, lendMask, BlendMask, etc. In the same transfer learning method, the model is trained again using its own data set so that the segmentation model can segment the basketball area and the basket area in the image, so as to calculate the relative distance between the basketball and the basket.

[0071] In actual implementation, in order to speed up the segmentation speed of the segmentation model, the image can be segmented on the bounding box ROI sub-image based on the image detection model to obtain the basketball area and the basket area.

[0072] In step 4 above, bounding boxes and bounding box overlap detection are involved, as follows: First, the candidate image frame sequence needs to be analyzed. The relative distance change trend between the basketball and the basket and net in the candidate image frame can be used for judgment, or the intersection-over-union (IOU) change trend between the basketball bounding box and the basket bounding box and the net bounding box can be used for judgment. Compared with the two methods, the IOU change trend of the intersection-over-union (IOU) of the bounding box has a higher accuracy rate. Therefore, the scheme of this embodiment is explained by the bounding box overlap detection (i.e., IOU change trend).

[0073] During the goal-scoring process, the first IOU change trend of the basketball bounding box and the basket bounding box in the candidate image frame, as well as the second IOU change trend of the basketball bounding box and the net bounding box, are shown in Figure 2. 4A to 4F As shown, the relative positions of the basketball and the basket net change in chronological order: 1. First, the basketball bounding box and the basket bounding box do not overlap, and the first intersection-and-union ratio (IOU) is 0. Figure 4A shown.

[0074] 2. Gradually, the basketball bounding box and the basket bounding box overlap, and the first intersection-union ratio IOU>0, such as Figure 4B As shown, the moment when the first intersection and IOU>0 can be recorded here .

[0075] 3. As the basketball initially falls into the basket, the first intersection-over-union ratio (IOU) between the basketball bounding box and the basket bounding box gradually increases. The x value of the left boundary of the basket bounding box is less than the x value of the left boundary of the basketball bounding box, and the x value of the right boundary of the basket bounding box is greater than the x value of the right boundary of the basketball bounding box, that is, the left and right boundaries of the basketball fall into the left and right boundaries of the basket. Figure 4C In this embodiment, the x-axis increases from left to right.

[0076] The upper boundary y value of the basket frame is greater than the upper boundary y value of the basketball frame, and the lower boundary y value of the basket frame is greater than the lower boundary y value of the basketball frame. That is, the upper and lower boundaries of the basketball are higher than the upper and lower boundaries of the basket frame, and there is a trend that the basketball will fall completely into the upper and lower boundaries of the basket frame. Figure 4C In this embodiment, the y-axis increases from top to bottom.

[0077] 4. As the basketball gradually falls and passes through the basket, the first intersection-over-union (IOU) of the basketball bounding box and the basket bounding box gradually increases and reaches a peak value. Here, a first threshold can be set for the first IOU to determine the goal frame. That is, only when the peak value of the first IOU is greater than the first threshold, it is considered that there may be a goal frame in this candidate image frame sequence. The value range of the first threshold can be defined as .

[0078] After the first intersection-and-union ratio reaches its peak, it gradually decreases, and satisfies the x value of the left boundary of the basket frame bounding box is less than the x value of the left boundary of the basketball frame bounding box, and the x value of the right boundary of the basket frame bounding box is greater than the x value of the right boundary of the basketball frame bounding box, that is, the left and right borders of the basketball are within the left and right borders of the basket frame. The y value of the upper boundary of the basket frame bounding box is less than the y value of the upper boundary of the basketball frame bounding box, and the y value of the lower boundary of the basket frame bounding box is less than the y value of the lower boundary of the basketball frame bounding box, that is, the upper and lower boundaries of the basketball are lower than the upper and lower boundaries of the basket frame, and there is a trend of gradually moving away from the upper and lower boundaries of the basket frame, such as Figure 4D and Figure 4E That is, the action of the basketball falling from the basket.

[0079] 5. After the basketball passes through the basket, it continues to fall until the IOU of the basketball's bounding box and the basket's bounding box becomes 0 again. Here we can record the moment when the IOU of the first IOU = 0 , the y value of the lower boundary of the basket bounding box is less than the y value of the upper boundary of the basketball bounding box, such as Figure 4F shown.

[0080] During the goal-scoring process, the second intersection-over-union (IOU) change trend of the basketball bounding box and the net bounding box is as follows: The second intersection-over-union (IOU) of the basketball bounding box and the net bounding box also changes from 0 to a peak value during the goal-scoring process. Similarly, a second threshold can be set for the second IOU to determine the goal frame. That is, only when the second IOU peak value is greater than the second threshold, it is considered that there may be a goal frame in the candidate image frame sequence. The value range of the second threshold can be defined as .

[0081] At the same time, the moment when the second intersection-over-union (IOU) of the basketball bounding box and the net bounding box reaches its peak can be recorded as . Then it gradually decreases and finally becomes 0 (record this moment as ), the change constraint conditions of the basketball bounding box boundary and the net bounding box are the same as the change conditions of the basketball bounding box boundary and the basket hoop bounding box.

[0082] Although the first IOU of the basketball bounding box and the basket hoop bounding box and the second IOU of the basketball bounding box and the net bounding box have the same changing trend, in a goal-scoring video, the second IOU of the net bounding box and the basketball bounding box lags behind the first IOU of the basket hoop bounding box and the basketball bounding box. In the goal-scoring video clip, the basketball first passes through the hoop and then the net.

[0083] Therefore, in a video frame, when the first intersection-over-union (IOU) of the basketball bounding box and the basket frame bounding box is greater than the first threshold, the second intersection-over-union (IOU) of the basketball bounding box and the net bounding box must be greater than 0, and the second intersection-over-union (IOU) of the basketball bounding box and the net bounding box must gradually increase and be greater than the second threshold, then it is considered that there may be a goal frame and a goal action in this sequence.

[0084] In the above step 5, the relative distance detection between the basketball and the basket is as follows: When judging that there may be a goal frame in the video sequence through the first intersection-over-union (IOU) of the basketball bounding box and the basket bounding box and the second intersection-over-union (IOU) of the basketball bounding box and the net bounding box (the reason why it cannot be completely determined is that misjudgment may occur in the scene where the perspectives overlap), the relative distance between the basketball and the basket is used for further judgment, such as Figure 5 As shown, Figure 5 Detailed steps of goal detection according to an embodiment of the present disclosure are shown in FIG.

[0085] The sizes of basketballs and baskets are known. For example, the standard men's game ball is a size 7 basketball with a diameter of 246 mm, the standard women's game ball is a size 6 basketball with a diameter of 226 mm, and the inner diameter of the international standard basket is 450 mm.

[0086] Step 5.1): First, determine whether the basketball bounding box, the basket frame bounding box and the net bounding box coincide with each other. If so, it means that there may be a goal frame. Go to step 5.2). Otherwise, if not, then end.

[0087] Step 5.2): Intercept the video clip where the first intersection of the basketball bounding box and the basket bounding box has an IOU greater than 0, that is, the video clip determined in step 4 to Video clips for a period of time.

[0088] In the embodiment, the relative distance between the basketball and the basket is used for judgment because the sizes of the basketball and the basket are known to be standard, but the size of the net has no fixed standard.

[0089] Step 5.3): Traverse frame by frame to Video clips for a period of time.

[0090] Step 5.4): Use the object detection model to detect the basketball bounding box and the basket hoop bounding box. Further, use the segmentation model to segment the pixel area of ​​the basketball from the ROI area of ​​the basketball bounding box and convert it into a basketball mask, and segment the pixel area of ​​the basket hoop from the ROI area of ​​the basket hoop bounding box and convert it into a basket hoop mask.

[0091] Step 5.5): Perform morphological processing on the basketball mask and the basket mask respectively. The mask needs to be expanded first to remove the scattered black spots in the mask, and then perform reverse corrosion processing to restore the size of the mask area to the size before expansion.

[0092] Step 5.6): Further, filter the basketball mask and the basket frame mask respectively. For example, median filtering, mean filtering, Gaussian filtering, etc. Through filtering, the burrs on the edge of the mask can be removed, which facilitates the extraction of the contour.

[0093] Step 5.7): Extract the contour information of the filtered basketball mask and the basket mask respectively. In actual implementation, the findContours function of opencv can be used to extract the contour information of the image and the hierarchical relationship between the contours.

[0094] Step 5.8): Calculate the diameter of the basketball. The basketball is a sphere, which is a circle in a two-dimensional image. Get the point set of the outermost contour of the basketball mask contour and calculate the minimum enclosing rectangle of the point set. In actual implementation, you can use the minAreaRect function of opencv to get the minimum enclosing rectangle of a set of points. The length or width of the minimum enclosing rectangle (or square) is the diameter of the basketball in the image.

[0095] Step 5.9): Calculate the diameter of the basket. The basket is a circle, which is an ellipse in a two-dimensional image. In the implementation, the inner diameter of the basket is used as the diameter of the basket. First, obtain the point set of the innermost contour of the basket mask contour, and calculate the minimum enclosing rectangle of the point set. In actual implementation, you can use OpenCV's minAreaRect to obtain the minimum enclosing rectangle of a set of points. The length of the minimum enclosing rectangle is the inner diameter of the basket in the image.

[0096] Step 5.10) Calculate the first distance between the basketball and the camera, and the second distance between the basket and the camera in the actual plane. Fig. 6A As shown, the specific situation is as follows Figure 6B As shown, Fig. 6A is a schematic diagram of a relative distance calculation principle according to an embodiment of the present disclosure, Figure 6BIt is a schematic diagram of the actual relative distance according to an embodiment of the present disclosure.

[0097] a. Principle Fig. 6A As shown, it is known that the length of a line segment in the pixel coordinate system is (unit is pixel), the length in the actual coordinate system is , the focal length of the device camera is .

[0098] Depend on It can be seen that: . (Use appropriate units of measurement, such as mm for calculations).

[0099] b. The first distance between the basketball and the camera in the actual plane and the second distance between the basket and the camera are calculated as follows: Figure 6B shown.

[0100] For example, the diameter of a basketball in a real plane is , the diameter of the basketball on the pixel plane is , the focal length of the device is f, the above three values ​​are all known, and the calculation formula of s in a can be used to obtain the first distance between the basketball and the camera in the actual plane. .

[0101] Similarly: For example, the inner diameter of the basket in the actual plane is , the diameter of the basketball on the pixel plane is , the focal length of the device is f, the above three values ​​are all known, and the calculation formula of s in a can be used to obtain the second distance between the basketball frame and the camera in the actual plane. .

[0102] Step 5.11): Calculate the first distance between the basketball and the camera in the actual plane And the second distance between the basket and the camera in the actual plane Afterwards, the relative distance between the basketball and the basket is calculated, which is the difference between the first distance and the second distance.

[0103] In the video sequence of basketball goal, the first intersection-over-union ratio (IOU) of the basketball bounding box and the basket bounding box is greater than 0, so it can be considered that the angle between the two straight lines starting from the camera and pointing to the center of the basketball and the center of the basket is very small. At this time, an approximate calculation can be performed to As the relative distance between the center of the basketball and the center of the basket.

[0104] Step 5.12): Determine whether the relative distance between the center of the basketball and the center of the basket is less than the distance threshold. If it is less than the threshold, it is considered that the invention content and the above arrive There are goal frames in the video of the time period. In actual implementation, The video frame at the moment is taken as the goal frame, and the loop ends at the moment when the first intersection-union ratio (IOU) between the basketball bounding box and the net bounding box is the largest.

[0105] On the contrary, if the relative distance between the center of the basketball and the center of the basket is not less than the distance threshold, continue to arrive The subsequent video frames in the time period video are judged.

[0106] It should be noted that when determining the distance threshold of the above relative distance, three critical states are discussed, such as FIG. 7A to FIG. 7C As shown: like Fig. 7A As shown, when the center of the basketball completely coincides with the center of the inner circle of the basket, the relative distance between the center of the basketball and the center of the inner circle of the basket is 0.

[0107] like Figure 7B As shown, when the outside of the basketball contacts the inner circle of the basket, the relative distance between the center of the basketball and the center of the inner circle of the basket is the inner radius of the basket minus the radius of the basketball d2-d1.

[0108] like Figure 7C As shown, when the outside of the basketball contacts the outer circle of the basket, the relative distance between the center of the basketball and the center of the inner circle of the basket is the inner radius of the basket plus the radius of the basketball d2+d1.

[0109] In actual implementation, considering the calibration error and calculation error, you can choose Figure 7C The critical scene is used as the distance threshold, that is, d2+d1. That is, when the above bounding box overlap detection change conditions are met, and arrive When the image frame in the video of the time period where the relative distance between the center of the basketball and the center of the inner circle of the basket is less than the distance threshold d2+d1 is found, The image frame at that moment is taken as the goal frame.

[0110] This embodiment proposes a method using the relative distance between the basketball and the basket to further judge whether there is a goal frame in the basketball video frame, thereby reducing the situation of misjudging the goal in the scene where the perspective overlaps. A method for calculating the diameter of the basketball in the pixel plane and the diameter inside the basket is also provided. By first using the detection model to detect the bounding box where the target is located, further segmenting the ROI sub-image, and then performing image segmentation on the sub-image, the inference speed of the image segmentation model is accelerated and the computing power is reduced.

[0111] This implementation can be realized by relying only on a monocular camera. Currently, most photographing devices are monocular. The method proposed in this implementation has good universality and can be implemented based on existing monocular hardware.

[0112] The present disclosure also provides an electronic device, comprising: at least one processor; and a memory connected to the at least one processor. The memory stores a computer program executable by the at least one processor, and the computer program is used to enable the electronic device to perform the method of the present disclosure when executed by the at least one processor.

[0113] The embodiments of the present disclosure also provide a non-transitory machine-readable medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute the method of the embodiments of the present disclosure.

[0114] The embodiments of the present disclosure also provide a computer program product, including a computer program, wherein when the computer program is executed by a processor of a computer, it is used to enable the computer to execute the method of the embodiments of the present disclosure.

[0115] refer to Figure 8 , a block diagram of an electronic device that can be used as a server or client of an embodiment of the present disclosure will now be described, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0116] like Figure 8 As shown, the electronic device includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 to a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0117] Multiple components in the electronic device are connected to the I / O interface 805, including: an input unit 806, an output unit 807, a storage unit 808, and a communication unit 809. The input unit 806 can be any type of device that can input information to the electronic device, and the input unit 806 can receive input digital or character information, and generate key signal input related to user settings and / or function control of the electronic device. The output unit 807 can be any type of device that can present information, and can include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 808 can include but is not limited to a disk, an optical disk. The communication unit 809 allows the electronic device to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and can include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0118] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a CPU, a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above. For example, in some embodiments, the method embodiments of the present disclosure may be implemented as a computer program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on an electronic device via a ROM 802 and / or a communication unit 809. In some embodiments, the computing unit 801 may be configured to perform the above method in any other appropriate manner (e.g., by means of firmware).

[0119] The computer programs for implementing the methods of the embodiments of the present disclosure may be written in any combination of one or more programming languages. These computer programs may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0120] In the context of the embodiments of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable signal medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, or infrared systems, devices, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0121] It should be noted that the term "including" and its variations used in the embodiments of the present disclosure are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". The modifications of "one" and "multiple" mentioned in the embodiments of the present disclosure are illustrative and not restrictive. Those skilled in the art should understand that unless the context clearly indicates otherwise, it should be understood as "one or more".

[0122] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0123] The various steps described in the method implementation methods provided in the embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method implementation methods may include additional steps and / or omit the steps shown. The scope of protection of the present disclosure is not limited in this respect.

[0124] The term "embodiment" in this specification refers to specific features, structures or characteristics described in conjunction with the embodiment that can be included in at least one embodiment of the present disclosure. The appearance of this phrase in various places in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. The various embodiments in this specification are described in a related manner, and the same and similar parts between the various embodiments refer to each other. In particular, for the device, equipment, and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts refer to the partial description of the method embodiment.

[0125] The above-described embodiments only express several implementation methods of the present disclosure, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present disclosure, and these all belong to the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure shall be subject to the attached claims.

Claims

1. A goal detection method, comprising: Acquire candidate image frames in the video captured by the camera, wherein the candidate image frames are image frames in the video that include a basketball boundary frame, a basket frame boundary frame, and a net boundary frame; Calculating a first intersection-and-union ratio of the basketball bounding box and the basket frame bounding box, and a second intersection-and-union ratio of the basketball bounding box and the net bounding box in the candidate image frame; When the first intersection-and-union ratio and the second intersection-and-union ratio satisfy a preset condition, determining a relative distance between the basketball and the basket in the candidate image frame, wherein the relative distance is a difference between a first distance from the basketball to the camera and a second distance from the basket to the camera in the candidate image frame; In the case that there is a target image frame whose relative distance is less than the corresponding distance threshold in the candidate image frames, it is determined that there is a goal.

2. The method according to claim 1, wherein: Determining whether the first intersection-and-union ratio and the second intersection-and-union ratio meet a preset condition includes: When the maximum value of the first intersection-and-union ratio reaches the corresponding first threshold, the second intersection-and-union ratio is greater than zero and increases from zero to the maximum value, and the maximum value of the second intersection-and-union ratio reaches the corresponding second threshold, it is determined that the preset condition is met.

3. The method according to claim 1, wherein: After determining that a goal has been scored, the method further comprises: An image frame at a moment when the second intersection-to-union ratio reaches a maximum value is acquired as a goal frame, wherein the goal frame is used to display a goal.

4. The method according to claim 1, wherein: Determining the relative distance between the basketball and the basket in the candidate image frame includes: Performing image segmentation on the candidate image frame to obtain a basketball pixel region and a basket frame pixel region; Determine a first diameter of the basketball in the candidate image frame based on the basketball pixel area, and determine a second diameter of the basket frame in the candidate image frame based on the basket frame pixel area; Obtaining the first distance based on the focal length information of the camera, the first diameter, and a real basketball diameter, and obtaining the second distance based on the focal length information, the second diameter, and a real basketball ring diameter; The difference is determined based on the first distance and the second distance.

5. The method according to claim 4, wherein: Performing image segmentation on the candidate image frame to obtain a basketball pixel region and a basket frame pixel region includes: Detecting a basketball bounding box and a basket bounding box in the candidate image frame by a detection model; Determine and segment the region of interest of the basketball bounding box and the region of interest of the basket hoop bounding box; Based on the regions of interest of the basketball bounding box and the basket frame bounding box, the basketball pixel region and the basket frame pixel region in the candidate image frame are detected and segmented by a segmentation model.

6. The method according to claim 4, wherein: Determining a first diameter of the basketball in the candidate image frame based on the basketball pixel area, and determining a second diameter of the basket frame in the candidate image frame based on the basket frame pixel area, comprising: Converting the basketball pixel region into a basketball mask, and converting the basket pixel region into a basket mask; Performing morphological processing on the basketball mask and the basket frame mask, and obtaining first contour information of the basketball and second contour information of the basket frame based on the processing results; Extract and generate a first circumscribed rectangle based on the outermost first contour of the basketball, and generate a second circumscribed rectangle based on the innermost second contour information of the basket frame; The first diameter of the basketball is determined according to the size of the first circumscribed rectangle, and the second diameter of the basket frame is determined according to the size of the second circumscribed rectangle.

7. The method according to claim 4, wherein: Obtaining the first distance based on the focal length information of the camera, the first diameter, and a real basketball diameter, and obtaining the second distance based on the focal length information, the second diameter, and a real basketball ring diameter includes: The first distance and the second distance are calculated based on the following formula: In the formula, is the first distance or the second distance, is the focal length of the camera, is the actual basketball diameter or the actual basket diameter, is the first diameter or the second diameter.

8. An electronic device comprising a memory, a processor and a computer program stored in the memory, wherein the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

9. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images