Basketball game out-of-bound detection method and system based on computer vision

Abstract

The invention relates to a basketball match out-of-bound detection method and system based on computer vision in the technical field of match monitoring. The method comprises the following steps: deploying a plurality of overlook camera cameras and a plurality of look-up camera cameras at a preset position of a basketball court according to a preset coverage rule; collecting identity information of each member of the teams of the two parties of the game before the game is started to obtain personnel identity information; a look-up camera is adopted to judge out-of-bound persons, and a look-down camera is combined with figure postures and basketball tracking to scientifically judge out-of-bound persons. According to the method, the out-of-bound situation can be captured in real time through the picture of the upward-looking camera, and meanwhile, the out-of-bound situation can be captured more sensitively due to the fact that the out-of-bound position is closer and the judgment precision is high; the machine position advantage of the overlook angle can bypass shielding of the bottom line, the out-of-bound condition in the range of the bottom line is directly detected, and meanwhile the out-of-bound condition is judged by directly analyzing the last ball touch record.

Description

Technical Field

[0001] The present application relates to the technical field of game monitoring, and in particular to a method and system for detecting out-of-bounds in a basketball game based on computer vision. Background Art

[0002] A basketball game is a game played based on the sport of basketball. There are many forms of basketball games, including the more common five-person basketball and the now popular street three-person basketball, which is a three-on-three game that emphasizes personal skills. In a basketball game, it is inevitable that the ball will be intercepted and dropped, causing the basketball to go out of bounds. The general judgment of whether the basketball is out of bounds is made by the referee.

[0003] The existing visual solution uses a bird's-eye view to acquire images, extracts boundaries based on the color difference in the picture, detects shoes on the boundaries, and determines what is out of bounds and what objects are out of bounds.

[0004] There are some problems with the existing technology: a simple top-down view is not enough to achieve accurate out-of-bounds detection. Due to the occlusion of people's body parts, there will be a lot of missed detections and false detections in the sideline detection (players occlude each other, and the shoes used for detection are blocked); at the same time, due to the occlusion of the basketball stand, it is difficult to identify the out-of-bounds in the basketball baseline area. In addition, the low clarity and perspective of the top-down view make it difficult to use the video as a necessary basis for out-of-bounds judgment. Summary of the invention

[0005] Based on this, it is necessary to provide a basketball game out-of-bounds detection method and system based on computer vision to address the above technical issues.

[0006] A method for detecting out-of-bounds in a basketball game based on computer vision, the method comprising: A plurality of overhead cameras and a plurality of upward cameras are deployed at preset positions of the basketball court according to preset coverage rules.

[0007] Before the game starts, the identity information of each member of both teams is collected to obtain the personnel identity information.

[0008] When the upward-looking camera recognizes that the pixel integrity of the edge area within its field of view is damaged, an out-of-bounds event warning is issued to the cloud server; the out-of-bounds event warning includes the corresponding time and the number of the current upward-looking camera and several frames before and after the out-of-bounds event warning is generated.

[0009] The cloud server analyzes the out-of-bounds event warning, retrieves video images from other overhead cameras at the same time, compares the positions of the two images, and uses posture recognition and target tracking algorithms to confirm the out-of-bounds member.

[0010] Re-identify the out-of-bounds member based on the personnel identity information to obtain the number and name of the out-of-bounds member.

[0011] The cloud server sends the out-of-bounds event information and the corresponding judgment basis to the penalty result presentation device; the out-of-bounds event information includes the out-of-bounds picture of the overhead camera located at the upward camera position in the corresponding area where the out-of-bounds event occurred as the basis, the out-of-bounds time, the out-of-bounds person and the team he belongs to.

[0012] A basketball game out-of-bounds detection system based on computer vision, the system comprises: a personnel information input device, a game out-of-bounds monitoring device, a cloud server and a penalty result presentation device.

[0013] The personnel information input device is used to input the identity information of each member of the two teams before the game to obtain the personnel identity information.

[0014] The monitoring equipment for out-of-bounds in the game includes several overhead cameras and several upward cameras; the several overhead cameras and several upward cameras are deployed at preset positions on the basketball court according to preset coverage rules; the upward camera is used to shoot the game screen from an upward angle and upload it to the cloud server, and is also used to calculate and detect the pixel integrity of the image, and when it is judged to be out of bounds, it will issue an out-of-bounds event warning to the cloud server; the overhead camera is used to shoot the game video screen and upload it to the cloud server; the out-of-bounds event warning includes the corresponding time and the number of the current upward camera and several frames before and after the out-of-bounds event warning is generated.

[0015] The cloud server is used to process the video images uploaded by the overhead camera and the upward camera, analyze the out-of-bounds event warning, retrieve the video images of other overhead cameras at the same time, compare the positions between the two images, and use posture recognition and target tracking algorithms to confirm the out-of-bounds members; re-identify the out-of-bounds members according to the personnel identity information to obtain the number and name of the out-of-bounds personnel.

[0016] The penalty result presentation device is used to present the complete result of the out-of-bounds event information for the referee to make a penalty decision; the out-of-bounds event information includes the out-of-bounds picture of the overhead camera located at the upward camera position in the corresponding area where the out-of-bounds event occurred, the out-of-bounds time, the out-of-bounds person and the team he belongs to.

[0017] The above-mentioned out-of-bounds detection method and system for basketball games based on computer vision, the method deploys several overhead cameras and several upward cameras at preset positions on the basketball court according to preset coverage rules; collects the identity information of each member of the two teams before the game starts to obtain the identity information of the personnel; uses the upward camera to judge out-of-bounds, and the overhead camera combines the posture of the person and the basketball tracking to scientifically judge the out-of-bounds personnel. This method can capture the out-of-bounds situation in real time through the picture of the upward camera, and at the same time, because it is closer to the out-of-bounds position, the judgment accuracy is high, and it can capture the occurrence of out-of-bounds more sensitively; the advantage of the camera position with an overhead angle can bypass the occlusion of the baseline and directly detect the out-of-bounds situation within the baseline range, and directly analyze the last touch record to judge the out-of-bounds situation. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 The present invention is a flowchart of a method for detecting out-of-bounds in a basketball game based on computer vision in one embodiment. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0020] In one embodiment, Figure 1 As shown, a method for detecting out-of-bounds in a basketball game based on computer vision is provided, and the method comprises the following steps: Step 100: deploy a plurality of overhead cameras and a plurality of upward cameras at preset positions of the basketball court according to preset coverage rules.

[0021] Specifically, this method does not require additional sideline brackets, and has a high degree of freedom in deployment angles, and has detection effects on playing fields of different specifications. In contrast, the traditional solution will reduce the accuracy of sideline recognition when facing playing fields of different colors, which directly leads to a decrease in the accuracy of out-of-bounds recognition.

[0022] Multiple overhead cameras and upward cameras are deployed at preset intervals on the sidelines of the basketball court; the deployment angles are determined according to the specific conditions of the basketball court.

[0023] The upward-looking camera determines whether the person is out of bounds, while the downward-looking camera combines the person's posture and basketball tracking to scientifically determine whether the person is out of bounds.

[0024] Step 102: Before the game starts, the identity information of each member of the two teams is collected to obtain the personnel identity information.

[0025] Specifically, personnel identity information is identifiable personnel features, including but not limited to personnel number, name, team name, full-length and half-length photos of players, etc. For example, before the start of a game, the number of the game team and each member of the team, the name of the member (optional), and the face and full-length photos of each player are entered using a personnel information entry device as personnel identity information. Personnel identity information is used for personnel identity information matching when out of bounds.

[0026] Step 104: When the upward-looking camera recognizes that the pixel integrity of the edge area within its field of view is damaged, an out-of-bounds event warning is issued to the cloud server; the out-of-bounds event warning includes the corresponding time and the number of the current upward-looking camera and several frames before and after the out-of-bounds event warning is generated.

[0027] Specifically, the images shot by the upward-looking camera can capture the out-of-bounds situation in real time, and because the distance to the out-of-bounds position is closer, the judgment accuracy is high.

[0028] The advantage of the overhead camera is that it can bypass the obstruction of the baseline and directly detect the out-of-bounds situation within the baseline range, while directly analyzing the last touch record to determine the out-of-bounds situation.

[0029] The close proximity of the camera to the sideline can more sensitively detect out-of-bounds situations. In contrast, the traditional solution uses the pixel difference of the image taken by the camera to determine the boundary line, which is subject to noise interference and is prone to misjudgment due to environmental disturbances.

[0030] Step 106: The cloud server analyzes the out-of-bounds event warning, retrieves video images from other overhead cameras at the same time, compares the positions of the two images, and uses posture recognition and target tracking algorithms to confirm the out-of-bounds member.

[0031] Specifically, when faced with a player's save behavior in an actual game, this method can distinguish whether the out-of-bounds object is above the sideline or has stepped on the sideline. However, the traditional solution, which only has a bird's-eye view, cannot distinguish this, which will directly lead to misjudgment. At the same time, based on posture recognition and target tracking technology, this method can more accurately capture the ball-touching records of upper limb movements, thereby providing a more accurate basis for out-of-bounds judgment; this method does not require additional input of specific parts (such as sneakers) to achieve out-of-bounds recognition.

[0032] Posture recognition can be implemented using algorithms such as YOLO-Pose and Mediapipe, and target tracking can be implemented using the YOLO algorithm. Based on posture recognition, the coordinates of the key points of the player's wrist are extracted, and based on target tracking, the real-time position coordinates of the basketball are extracted. The difference between the coordinates of the key points of each player's wrist in the picture and the coordinates of the basketball position is calculated and recorded. If the difference is lower than a certain value, it is recorded as a touch event, and the current timestamp and the identity information of the touch player are recorded. According to the most recent touch event when the out-of-bounds event occurs, the out-of-bounds member is determined.

[0033] Step 108: Re-identify the out-of-bounds member according to the personnel identity information to obtain the number and name of the out-of-bounds member.

[0034] Step 110: The cloud server sends the out-of-bounds event information and corresponding judgment basis to the penalty result presentation device; the out-of-bounds event information includes the out-of-bounds picture of the overhead camera located at the upward camera position in the corresponding area where the out-of-bounds event occurred, the out-of-bounds time, the out-of-bounds person and the team he belongs to.

[0035] In the above-mentioned out-of-bounds detection method for basketball games based on computer vision, the method deploys several overhead cameras and several upward cameras at preset positions on the basketball court according to preset coverage rules; collects the identity information of each member of the two teams before the game starts to obtain the identity information of the personnel; uses the upward camera to judge out-of-bounds, and the overhead camera combines the posture of the person and the basketball tracking to scientifically judge the out-of-bounds personnel. This method can capture the out-of-bounds situation in real time through the picture of the upward camera, and because it is closer to the out-of-bounds position, the judgment accuracy is high, and it can capture the occurrence of out-of-bounds more sensitively; the advantage of the camera position with an overhead angle can bypass the occlusion of the baseline and directly detect the out-of-bounds situation within the baseline range, and directly analyze the last touch record to judge the out-of-bounds situation.

[0036] In one embodiment, step 104 includes: the upward-looking camera captures the game picture from an upward angle close to the sideline of the court; the upward-looking camera transmits the picture in the current angle of view before the game to the cloud server, the cloud server identifies the sideline in the current image through segmentation technology, and returns the result of the sideline area to the upward-looking camera, sets the boundary area of ​​the game, and monitors the line integrity of the boundary area; convolution and noise reduction are performed on the current game picture to obtain two-dimensional pixel points; pixel integrity is calculated for a specific boundary area, and when the line pixel integrity is lower than a preset threshold, it is considered out of bounds; the upward-looking camera returns the current timestamp and several frames of video pictures attached to the timestamp to the cloud server, and issues an out-of-bounds event warning.

[0037] In one embodiment, the sideline monitoring process includes: before the start of the game, the upward-looking camera sends the image in the current field of view to the cloud server, and the cloud server uses an image segmentation method to segment the image corresponding to the current slowness and identify the sideline position; after identifying the sideline position, a semantic segmentation method is used to process potential occlusions in the slowness, perform noise reduction, and convert the discrete sideline area into a continuous area.

[0038] Specifically, before the game starts, the upward-looking camera sends the current field of view to the cloud server, which uses the image segmentation technology in the computer vision algorithm to segment the current sideline. The advantage of this technology is that it improves the robustness of sideline recognition and the accuracy of sideline recognition. Through semantic segmentation, this technology can intelligently identify debris in the picture and accurately mark the current sideline, while ignoring the occlusion in the picture. Facing the lighting problems of different courts, it can better demonstrate the robustness of sideline monitoring. The semantic segmentation technology uses deep learning technology based on the sideline recognition pictures of previous games, and its framework is specifically a CNN convolutional neural network. The input of semantic segmentation is the corresponding picture output, which is the position of the sideline in the picture recognized in the current picture. After identifying the sideline position, semantic segmentation technology is used to process potential occlusions in the picture, perform noise reduction, and convert discrete sideline areas into continuous areas.

[0039] In one embodiment, step 106 includes: after the cloud server receives the out-of-bounds event warning, query the images of the remaining overhead cameras according to the time report in the out-of-bounds event warning; use a personnel recognition method to identify the personnel according to the current image of the overhead camera and the image of the upward camera at the current time, and obtain the identity information of the personnel in the video image; extract features of several frames of the upward camera and the overhead camera before the out-of-bounds event warning occurs, and determine the personnel corresponding to each part in the two perspective images based on feature similarity; according to the identity information obtained after personnel identification, regard the personnel in the identification result as the out-of-bounds personnel, and return the out-of-bounds personnel and the out-of-bounds time point information, and the out-of-bounds video images of the upward camera and the downward camera to the out-of-bounds result viewing device; if the identity of the corresponding person cannot be identified in the current frame, then deduce forward according to the time point of the out-of-bounds event warning, and judge in sequence according to the several frames of the images before the out-of-bounds event warning is generated until it is sufficient to judge the identity information of the current out-of-bounds person.

[0040] In one of the embodiments, the time point of the out-of-bounds monitoring event is deduced forward, and the images of several frames before the time when the out-of-bounds event warning is generated are judged in sequence until the identity information of the current out-of-bounds person is sufficient to be determined, including: extracting the video image of the time when the out-of-bounds event warning occurs according to the overhead camera of the out-of-bounds event warning area, using human posture recognition technology to perform posture modeling on the person in the image, obtaining the corresponding human key point bitmap, and finding the key node coordinates of the corresponding wrist node position; using object tracking technology to identify the basketball and locate the basketball position coordinates according to the pre-trained object recognition model; replaying the video image for several frames in reverse, continuously extracting the position coordinates of the basketball in the image, the identity number of the person in the image, and the corresponding wrist node coordinates, and calculating the difference distance between the basketball position coordinates and the wrist node coordinates, taking the identity number of the person with the smallest difference from the basketball from large to small according to the timestamp, and considering the person corresponding to the number as the out-of-bounds person, and recording the timestamp and saving it as the basis for judging the out-of-bounds event; sending the basis for judging the out-of-bounds event, the out-of-bounds person, and the time point of the video image required for judging the identity information of the out-of-bounds person back to the penalty result presentation device.

[0041] Specifically, after the cloud server receives the out-of-bounds event warning, it will query the remaining overhead camera positions based on the time report in the out-of-bounds event warning content, and further compare the current picture with the current time picture of the upward camera position. The identity information of the person in the video screen will be obtained through personnel recognition technology. By extracting features from several frames of the upward camera position before the out-of-bounds monitoring event occurs, and performing personnel re-identification analysis with the viewpoint at the time of the out-of-bounds monitoring event, the corresponding person in each part of the two viewpoints is determined based on feature similarity. According to the identity information obtained after personnel identification, the person in this identification result is regarded as an out-of-bounds person, and this result, together with the out-of-bounds time point information, the out-of-bounds video screen (including the upward camera position and the downward camera position) is returned to the out-of-bounds result viewing device.

[0042] If the identity of the corresponding person cannot be identified in the current frame, the time point of the out-of-bounds monitoring event is deduced forward, and the images of several frames before the out-of-bounds event warning are judged in sequence until the current out-of-bounds person's identity information is sufficient. The previous video images are extracted from the overhead camera of the out-of-bounds event area, and the posture modeling of the characters in the image is performed using human posture recognition technology to obtain the corresponding human key point bitmap, and the key node coordinates of the corresponding wrist node position are found; at the same time, the basketball is identified and the position coordinates of the basketball are located based on the object tracking technology and the pre-trained object recognition model. Then, the video image is played back for several frames, and the position coordinates of the basketball in the image, the identity number of the person in the image, and the corresponding wrist node coordinates are continuously extracted, and the difference distance between the basketball position coordinates and the wrist node coordinates is calculated. After the calculation is completed, the identity number of the person with the smallest difference from the basketball is taken from large to small according to the timestamp. This person is regarded as an out-of-bounds person, and the timestamp is recorded and saved as the basis for judging the out-of-bounds event. After that, this result and the time point of the video image required to judge the identity information of the out-of-bounds person are sent back to the penalty result presentation device.

[0043] In one embodiment, step 108 includes: when the cloud server receives an out-of-bounds event warning, the ReID personnel re-identification technology is used to identify the current player according to the out-of-bounds member and the personnel identity information to obtain the number and name of the out-of-bounds person.

[0044] Specifically, the process of personnel identification includes: 1) Data entry stage: Before the game starts, players will enter the corresponding information and facial information in the form of pictures on this device. These pictures will then be uploaded to the cloud server for storage and used later to compare the identity of the out-of-bounds players.

[0045] 2) When performing identity recognition after out-of-bounds detection: When the cloud server performs out-of-bounds personnel recognition after receiving an out-of-bounds detection warning, personnel recognition technology is required to identify the identity of the members in the picture. This system uses ReID (Re-identification) personnel re-identification technology to identify the current player. This technology uses personnel recognition to cut out the image of the person in the picture from the current video picture, extract the image features, and match the image features with the features of the person entering the database for similarity, and determine the result with the highest matching degree as the current person's identity.

[0046] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0047] In one embodiment, a computer vision-based out-of-bounds detection system for basketball games is provided, which includes: a personnel information input device, a game out-of-bounds monitoring device, a cloud server, and a penalty result presentation device. The device can directly add more game event recognition functions to the existing framework.

[0048] The personnel information input device is used to input the identity information of each member of the two teams before the game to obtain the personnel identity information.

[0049] Specifically, the personnel information input device is used to input the team, the number of each member in the team, the name of each member (optional), and the face and full-body photos of each player before the game starts. It is used to match the personnel identity information when out of bounds.

[0050] The monitoring equipment for out-of-bounds in the game includes several overhead cameras and several upward cameras; the several overhead cameras and several upward cameras are deployed at preset positions on the basketball court according to preset coverage rules; the upward camera is used to shoot the game screen from an upward angle and upload it to the cloud server, and is also used to calculate and detect the pixel integrity of the image, and when it is judged to be out of bounds, it will issue an out-of-bounds event warning to the cloud server; the overhead camera is used to shoot the game video screen and upload it to the cloud server; the out-of-bounds event warning includes the corresponding time and the number of the current upward camera and several frames before and after the out-of-bounds event warning is generated.

[0051] Specifically, the upward-looking camera has the functions of calculating pixel integrity of the image, detecting and issuing out-of-bounds event warnings and judgments to the cloud server, and taking and uploading images, which can generally be achieved by a small development board. The downward-looking camera only has the functions of video recording and uploading. The above-mentioned multiple cameras will be installed at specific locations on the basketball court according to certain coverage rules.

[0052] The upward-looking camera is responsible for providing an upward-looking game screen close to the sideline. The upward-looking camera will first transmit the current view to the cloud server with the team before the game. The cloud server uses image segmentation technology to identify the sidelines in the current picture and returns the results of the sideline area to the upward-looking camera. The upward-looking camera uses this to set the boundary area of ​​the game and monitor the line integrity of this area. By performing simple convolution and noise reduction processing on the current game screen, two-dimensional pixel points can be obtained. By calculating the regional pixel integrity of a specific boundary area, it can be determined whether there is an out-of-bounds situation at this time. When the line pixel integrity is lower than a certain threshold, the trigger judgment is considered to be out-of-bounds. The upward-looking camera will return the current timestamp and the video screen of the last few frames to the server, and issue an out-of-bounds monitoring warning.

[0053] The cloud server is used to process the video images uploaded by the overhead camera and the upward camera, analyze the out-of-bounds event warning, retrieve the video images of other overhead cameras at the same time, compare the positions between the two images, and use posture recognition and target tracking algorithms to confirm the out-of-bounds members; re-identify the out-of-bounds members according to the personnel identity information to obtain the number and name of the out-of-bounds personnel.

[0054] Specifically, the cloud server is responsible for processing uploaded videos, monitoring out-of-bounds events, analyzing the personnel involved in out-of-bounds events, and sending out-of-bounds event results to the penalty result presentation device.

[0055] The penalty result presentation device is used to present the complete result of the out-of-bounds event information for the referee to make a penalty decision; the out-of-bounds event information includes the out-of-bounds picture of the overhead camera located at the upward camera position in the corresponding area where the out-of-bounds event occurred, the out-of-bounds time, the out-of-bounds person and the team he belongs to.

[0056] Specifically, the devices for presenting penalty results include but are not limited to dedicated display screens outside the competition venue, mobile phones, display screens facing the audience, etc.

[0057] The working process of the out-of-bounds detection system for basketball games based on computer vision is as follows: First, before the game starts, the players on both sides need to enter the personnel information, current clothing photos and face pictures in the equipment for personnel identity monitoring and identification of subsequent events. This part of information will be uploaded to the back-end server for storage. Next, the upward camera in the equipment for out-of-bounds monitoring is initialized. The upward camera will shoot the current scene and transmit the picture to the cloud server. The cloud server uses segmentation technology to identify the corresponding sideline and return the value of the sideline area in the picture to each upward camera. In this way, the sideline is monitored. Next, calibrate the equipment. Before the game starts, the upward camera will monitor whether there is any damage to the pixel integrity of the sideline area in the current field of view, and the downward camera will continuously record and transmit the video to the cloud server. When the upward camera recognizes that the pixel integrity of the sideline area in its field of view is damaged, it will issue an out-of-bounds warning, where the out-of-bounds warning contains the corresponding time, the number of the current upward camera, and several frames before and after the out-of-bounds warning is generated. The cloud server will analyze the out-of-bounds event at this time, retrieve the video images of other overhead cameras in the same time period, identify the people in the images, and use gesture recognition and target tracking technology to confirm the object that last touched the ball by comparing the positions of the two images to determine the members who are out of bounds. Re-identify the personnel based on the personnel data collected before the game to obtain the specific number and name of the out-of-bounds object. Subsequently, the cloud server sends the out-of-bounds event information to the penalty result presentation device so that the referee can make a timely judgment; among them, the out-of-bounds event information includes the out-of-bounds image of the overhead camera in the corresponding area where the out-of-bounds event occurred, the out-of-bounds time, the out-of-bounds person and team, and the basis for the judgment. The referee can view the result of the penalty and the basis of the corresponding image on the penalty result presentation device, and can play a whole video as needed.

[0058] In one of the embodiments, the cloud server is further used to query the images of other overhead cameras according to the time report in the out-of-bounds event warning after receiving the out-of-bounds event warning, and use a personnel recognition method to identify the personnel according to the current image of the overhead camera and the image of the upward camera at the current time to obtain the identity information of the personnel in the video image; by extracting features from several frames of the upward camera and the overhead camera before the time when the out-of-bounds event warning occurs, determine the personnel corresponding to each part in the two perspective images based on feature similarity; according to the identity information obtained after personnel identification, regard the personnel in the identification result as the out-of-bounds personnel, and return the out-of-bounds personnel and the out-of-bounds time point information, and the out-of-bounds video images of the upward camera and the downward camera to the out-of-bounds result viewing device; if the identity of the corresponding person cannot be identified in the current frame, then deduce forward according to the time point of the out-of-bounds event warning, and judge in sequence according to the several frames of the images before the time when the out-of-bounds event warning is generated until it is sufficient to judge the identity information of the current out-of-bounds person.

[0059] Specifically, the system uses visual algorithms to detect out-of-bounds in basketball games. The system monitors each sideline in real time during the game, and identifies the location, person, and time of the out-of-bounds event based on the algorithm as soon as it occurs, thereby detecting the out-of-bounds event.

[0060] In one of the embodiments, the cloud server is also used to extract the video picture at the time when the out-of-bounds event warning occurs based on the overhead camera of the out-of-bounds event warning area, use human posture recognition technology to perform posture modeling on the characters in the picture, obtain the corresponding human key point map, and find the key node coordinates of the corresponding wrist node position; use object tracking technology to identify the basketball and locate the basketball position coordinates based on the pre-trained object recognition model; play the video picture in reverse for several frames, continuously extract the position coordinates of the basketball in the picture and the identity number of the person in the picture and the corresponding wrist node coordinates, and calculate the difference distance between the basketball position coordinates and the wrist node coordinates, take the identity number of the person with the smallest difference from the basketball from large to small according to the timestamp, regard the person corresponding to the number as the out-of-bounds person, and record the timestamp and save it as the basis for judging the out-of-bounds event; send the basis for judging the out-of-bounds event, the out-of-bounds person, and the time point of the video picture required for judging the identity information of the out-of-bounds person back to the penalty result presentation device.

[0061] In one of the embodiments, the overhead camera is also used to shoot the game scene from an upward angle close to the sideline of the court; the scene in the current angle of view before the game is transmitted to the cloud server, and the cloud server receives the result of the sideline area obtained by identifying the sidelines in the current image using segmentation technology, and sets the result of the sideline area as the boundary area of ​​this game, and monitors the line integrity of the boundary area; convolution and noise reduction processing are performed on the current game scene to obtain two-dimensional pixel points; pixel integrity calculation is performed on a specific boundary area, and when the line pixel integrity is lower than a preset threshold, it is considered out of bounds; then the current timestamp and several frames of video pictures attached to the timestamp are returned to the cloud server, and an out-of-bounds event warning is issued.

[0062] For the specific definition of the out-of-bounds detection system for basketball games based on computer vision, please refer to the definition of the out-of-bounds detection method for basketball games based on computer vision above, which will not be repeated here. Each module in the above-mentioned out-of-bounds detection system for basketball games based on computer vision can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0063] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0064] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A basketball game out-of-bounds detection method based on computer vision, characterized in that: The method comprises: Deploy a plurality of overhead view cameras and a plurality of upward view cameras at preset positions of the basketball court according to preset coverage rules; Before the game starts, the identity information of each member of both teams is collected to obtain the personnel identity information; When the upward-looking camera recognizes that the pixel integrity of the edge area in its field of view is damaged, an out-of-bounds event warning is issued to the cloud server; the out-of-bounds event warning includes the corresponding time and the number of the current upward-looking camera and a number of frames before and after the out-of-bounds event warning is generated; The cloud server analyzes the out-of-bounds event warning, retrieves video images from other overhead cameras at the same time, compares the positions between the two images, and uses posture recognition and target tracking algorithms to confirm the out-of-bounds member; Re-identify the out-of-bounds member based on the personnel identity information to obtain the number and name of the out-of-bounds member; The cloud server sends the out-of-bounds event information and the corresponding judgment basis to the penalty result presentation device; the out-of-bounds event information includes the out-of-bounds picture of the overhead camera located at the upward camera position in the corresponding area where the out-of-bounds event occurred as the basis, the out-of-bounds time, the out-of-bounds person and the team he belongs to.

2. The method for detecting out-of-bounds in a basketball game based on computer vision according to claim 1, characterized in that: When the upward-looking camera detects that the pixel integrity of the edge area within its field of view is damaged, it issues an out-of-bounds event warning to the cloud server, including: The upward-looking camera shoots the game scene from an upward angle close to the sideline of the court; The upward-looking camera transmits the image in the current viewing angle before the game to the cloud server. The cloud server recognizes the sidelines in the current image through segmentation technology and returns the results of the sideline area to the upward-looking camera, sets the boundary area of ​​the game, and monitors the line integrity of the boundary area. Perform convolution and noise reduction on the current game screen to obtain two-dimensional pixel points; Calculate the pixel integrity of a specific boundary area. When the pixel integrity of a line is lower than a preset threshold, it is considered out of bounds. The upward-looking camera returns the current timestamp and several frames of video attached to the timestamp to the cloud server, and issues an out-of-bounds event warning.

3. The method for detecting out-of-bounds in a basketball game based on computer vision according to claim 1, characterized in that: The sideline monitoring process includes: Before the game starts, the upward-looking camera sends the current field of view to the cloud server, and the cloud server uses an image segmentation method to segment the current slowly corresponding image and identify the sideline position; After identifying the edge position, the semantic segmentation method is used to deal with potential occlusion in the slowness, and noise reduction is performed to convert the discrete edge area into a continuous area.

4. The method for detecting out-of-bounds in a basketball game based on computer vision according to claim 1, characterized in that: The cloud server analyzes the out-of-bounds event warning, retrieves the video images of other overhead cameras at the same time, compares the positions between the two images, and uses posture recognition and target tracking algorithms to confirm the out-of-bounds member, including: After the cloud server receives the out-of-bounds event warning, it queries the images of other overhead cameras according to the time report in the out-of-bounds event warning; According to the current picture of the overhead camera and the picture of the upward camera at the current time, a person identification method is used to identify the person, and the identity information of the person in the video picture is obtained; By extracting features from several frames of the upward-looking camera and the downward-looking camera before the out-of-bounds event warning occurs, the personnel corresponding to each part in the two perspective images are determined based on feature similarity; According to the identity information obtained after the person is identified, the person in the identification result is regarded as an out-of-bounds person, and the out-of-bounds person and the out-of-bounds time point information, and the out-of-bounds video images of the upward and downward camera positions are returned to the out-of-bounds result viewing device; If the identity of the corresponding person cannot be identified in the current frame, the time point of the out-of-bounds event warning is deduced forward, and several frames before the out-of-bounds event warning are judged in sequence until the identity information of the current out-of-bounds person is sufficient to be determined.

5. The method for detecting out-of-bounds in a basketball game based on computer vision according to claim 4, characterized in that: Then, the process is deduced forward from the time when the out-of-bounds monitoring event occurs, and the images of several frames before the out-of-bounds event warning is generated are judged in sequence until the identity information of the current out-of-bounds person is sufficiently determined, including: The video picture at the time when the out-of-bounds event warning occurs is extracted from the camera with a bird's-eye view of the area where the out-of-bounds event warning occurs, and the posture modeling of the person in the picture is performed using the human posture recognition technology to obtain the corresponding human key point bitmap, and find the key node coordinates of the corresponding wrist node position; Using object tracking technology, the basketball is identified and its coordinates are located based on the pre-trained object recognition model. Reverse the video for several frames, continuously extract the position coordinates of the basketball in the picture, the identity number of the person in the picture, and the corresponding wrist node coordinates, and calculate the difference between the basketball position coordinates and the wrist node coordinates. According to the timestamp, the identity number of the person with the smallest difference from the basketball is taken from large to small, and the person corresponding to the number is regarded as an out-of-bounds person, and the timestamp is recorded and saved as the basis for judging the out-of-bounds event; The basis for judging the out-of-bounds event, the out-of-bounds person, and the time point of the video image required for judging the identity information of the out-of-bounds person are sent back to the penalty result presentation device.

6. The method for detecting out-of-bounds in a basketball game based on computer vision according to claim 1, characterized in that: Re-identify the out-of-bounds member based on the personnel identity information and obtain the number and name of the out-of-bounds person, including: When the cloud server receives an out-of-bounds event warning, it uses the ReID personnel re-identification technology to identify the current player based on the out-of-bounds member and the personnel identity information, and obtains the number and name of the out-of-bounds person.

7. A basketball game out-of-bounds detection system based on computer vision, characterized in that: The system includes: a personnel information input device, a match out-of-bounds monitoring device, a cloud server, and a penalty result presentation device; The personnel information input device is used to input the identity information of each member of the two teams before the game to obtain the personnel identity information; The out-of-bounds monitoring equipment includes a plurality of overhead cameras and a plurality of upward cameras; the plurality of overhead cameras and the plurality of upward cameras are deployed at preset positions on the basketball court according to preset coverage rules; the upward camera is used to shoot the game picture from an upward angle and upload it to the cloud server, and is also used to calculate and detect the pixel integrity of the picture, and when it is judged to be out of bounds, it sends an out-of-bounds event warning to the cloud server; the overhead camera is used to shoot the game video picture and upload it to the cloud server; the out-of-bounds event warning includes the corresponding time and the number of the current upward camera and a plurality of frames before and after the out-of-bounds event warning is generated; The cloud server is used to process the video images uploaded by the overhead camera and the upward camera, analyze the out-of-bounds event warning, retrieve the video images of other overhead cameras at the same time, compare the positions between the two images, and use posture recognition and target tracking algorithms to confirm the out-of-bounds member; re-identify the out-of-bounds member according to the personnel identity information to obtain the number and name of the out-of-bounds person; The penalty result presentation device is used to present the complete result of the out-of-bounds event information for the referee to make a penalty decision; the out-of-bounds event information includes the out-of-bounds picture of the overhead camera located at the upward camera position in the corresponding area where the out-of-bounds event occurred, the out-of-bounds time, the out-of-bounds person and the team he belongs to.

8. The computer vision-based basketball game out-of-bounds detection system according to claim 7, characterized in that: The cloud server is also used to query the images of other overhead cameras according to the time report in the out-of-bounds event warning after receiving the out-of-bounds event warning, and use a personnel recognition method to identify personnel according to the current image of the overhead camera and the image of the upward camera at the current time to obtain the identity information of the personnel in the video image; by extracting features from several frames of images of the upward camera and the overhead camera before the time when the out-of-bounds event warning occurs, determine the personnel corresponding to each part in the two perspective images based on feature similarity; according to the identity information obtained after personnel identification, regard the personnel in the identification result as the out-of-bounds personnel, and return the out-of-bounds personnel and the out-of-bounds time point information, and the out-of-bounds video images of the upward camera and the downward camera to the out-of-bounds result viewing device; if the identity of the corresponding person cannot be identified in the current frame, then deduce forward according to the time point of the out-of-bounds event warning, and judge in sequence according to the several frames of images before the time when the out-of-bounds event warning is generated, until it is sufficient to judge the identity information of the current out-of-bounds person.

9. The computer vision-based basketball game out-of-bounds detection system according to claim 7, characterized in that: The cloud server is also used to extract the video picture of the time when the out-of-bounds event warning occurs according to the overhead camera of the out-of-bounds event warning occurrence area, use human posture recognition technology to perform posture modeling on the characters in the picture, obtain the corresponding human key point map, and find the key node coordinates of the corresponding wrist node position; use object tracking technology to identify the basketball and locate the basketball position coordinates according to the pre-trained object recognition model; play the video picture in reverse for several frames, continuously extract the position coordinates of the basketball in the picture and the identity number of the person in the picture and the corresponding wrist node coordinates, and calculate the difference distance between the basketball position coordinates and the wrist node coordinates, take the identity number of the person with the smallest difference from the basketball from large to small according to the timestamp, regard the person corresponding to the number as the out-of-bounds person, record the timestamp, and save it as the basis for judging the out-of-bounds event; send the basis for judging the out-of-bounds event, the out-of-bounds person, and the time point of the video picture required for judging the identity information of the out-of-bounds person back to the penalty result presentation device.

10. The computer vision-based basketball game out-of-bounds detection system according to claim 7, characterized in that: The overhead camera is also used to shoot the game scene from an upward angle close to the sideline of the court; transmit the picture in the current angle of view before the game to the cloud server, and receive the result of the sideline area obtained by the cloud server using segmentation technology to identify the sidelines in the current image, and set the result of the sideline area as the boundary area of ​​this game, and monitor the line integrity of the boundary area; perform convolution and noise reduction processing on the current game picture to obtain two-dimensional pixel points; calculate the pixel integrity of a specific boundary area, and when the line pixel integrity is lower than a preset threshold, it is considered out of bounds; then return the current timestamp and several frames of video pictures attached to the timestamp to the cloud server, and issue an out-of-bounds event warning.

Images