Basketball game out-of-bounds detection method and system based on computer vision

Abstract

The present application relates to a method and system for detecting out-of-bounds in a basketball game based on computer vision in the field of game monitoring technology. The method deploys a number of overhead cameras and a number of upward cameras at preset positions on a basketball court according to preset coverage rules; collects identity information of each member of both teams before the game starts to obtain personnel identity information; uses the upward camera to determine out-of-bounds, and uses the downward camera to scientifically determine the out-of-bounds personnel by combining the posture of the person and basketball tracking. This method can capture out-of-bounds situations in real time through the image of the upward camera. At the same time, due to the closer distance to the out-of-bounds position, the judgment accuracy is high, and the occurrence of out-of-bounds can be captured more sensitively; the advantage of the camera position with an overhead angle can bypass the occlusion of the baseline and directly detect out-of-bounds situations within the baseline range, while directly analyzing the last touch record to determine out-of-bounds situations.

Description

Technical Field

[0001] The present application relates to the field of game monitoring technology, and in particular to a method and system for detecting out-of-bounds in basketball games based on computer vision. Background Art

[0002] A basketball game is a game played based on the sport of basketball. There are various forms of basketball games, including the more common five-person basketball and the now popular street three-person basketball, which is a three-on-three game that emphasizes individual skills. In a basketball game, it is inevitable that the ball will be intercepted and dropped from the hands, causing the basketball to go out of bounds. Generally, the referee determines whether the basketball is out of bounds.

[0003] Existing visual solutions use a bird's-eye view to acquire images, extract boundaries based on color differences in the image, and detect shoes on the boundaries to determine out-of-bounds objects.

[0004] Existing technologies have several issues: A simple overhead view is insufficient for accurate out-of-bounds detection. Due to occlusion by human body parts, sideline detection often results in missed and false detections (players obstructing each other, and shoes used for detection being blocked). Furthermore, the basketball hoop obstructs the detection of out-of-bounds situations at the baseline. Furthermore, the low clarity and viewing angle of the overhead view make video difficult to use as the necessary basis for out-of-bounds judgments. Summary of the Invention

[0005] Based on this, it is necessary to provide a basketball game out-of-bounds detection method and system based on computer vision to address the above technical problems.

[0006] A method for detecting out-of-bounds in a basketball game based on computer vision, the method comprising:

[0007] Several overhead cameras and several upward cameras are deployed at preset positions on the basketball court according to preset coverage rules.

[0008] Before the game starts, the identity information of each member of the two teams is collected to obtain the personnel identity information.

[0009] When the upward-looking camera recognizes that the pixel integrity of the edge area within its field of view is damaged, it issues an out-of-bounds event warning to the cloud server; the out-of-bounds event warning includes the corresponding time and the number of the current upward-looking camera and several frames before and after the out-of-bounds event warning is generated.

[0010] The cloud server analyzes the out-of-bounds event warning, retrieves video images from other overhead cameras captured at the same time, compares the positions of the two images, and uses posture recognition and target tracking algorithms to confirm the out-of-bounds member.

[0011] Re-identify the out-of-bounds members based on their identity information and obtain their numbers and names.

[0012] The cloud server sends the out-of-bounds event information and the corresponding judgment basis to the penalty result presentation device; the out-of-bounds event information includes the out-of-bounds images of the overhead camera position and the upward camera position of the corresponding area where the out-of-bounds event occurred, the out-of-bounds time, the out-of-bounds person and the team he belongs to.

[0013] A computer vision-based out-of-bounds detection system for basketball games includes: a personnel information input device, a game out-of-bounds monitoring device, a cloud server, and a penalty result presentation device.

[0014] The personnel information input device is used to input the identity information of each member of the two teams before the game to obtain the personnel identity information.

[0015] The monitoring equipment for out-of-bounds in the game includes several overhead cameras and several upward cameras; the several overhead cameras and several upward cameras are deployed at preset positions on the basketball court according to preset coverage rules; the upward cameras are used to shoot the game pictures from an upward angle and upload them to the cloud server, and are also used to calculate and detect the pixel integrity of the pictures, and when it is judged to be out of bounds, an out-of-bounds event warning is issued to the cloud server; the overhead cameras are used to shoot the game video pictures and upload them to the cloud server; the out-of-bounds event warning includes the corresponding time and the number of the current upward camera and several frames before and after the out-of-bounds event warning is generated.

[0016] The cloud server is used to process the video images uploaded by the overhead camera and the upward camera, analyze the out-of-bounds event warning, retrieve the video images of other overhead cameras at the same time, compare the positions between the two images, and use posture recognition and target tracking algorithms to confirm the out-of-bounds members; re-identify the out-of-bounds members based on their identity information to obtain the number and name of the out-of-bounds person.

[0017] The penalty result presentation device is used to present the complete result of the out-of-bounds event information for the referee to make a penalty decision; the out-of-bounds event information includes the out-of-bounds images from the overhead and upward camera positions of the area corresponding to the out-of-bounds event, the out-of-bounds time, the out-of-bounds person and the team he belongs to.

[0018] The aforementioned computer vision-based out-of-bounds detection method and system for basketball games involves deploying several overhead cameras and several upward cameras at preset locations on the basketball court according to preset coverage rules. Before the game begins, the identity information of each player on both teams is collected to obtain personal identity information. The upward cameras are used to determine out-of-bounds situations, while the downward cameras combine human posture and basketball tracking to scientifically identify out-of-bounds individuals. This method, using the upward camera's imagery, can instantly capture out-of-bounds situations. Furthermore, due to its closer proximity to the out-of-bounds location, it offers high accuracy and can more sensitively detect out-of-bounds situations. The advantage of the upward camera's position allows it to bypass baseline obstructions and directly detect out-of-bounds situations within the baseline. Furthermore, out-of-bounds situations are determined by directly analyzing the last touch record. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 The figure is a flowchart of a method for detecting out-of-bounds in a basketball game based on computer vision in one embodiment. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0021] In one embodiment, Figure 1 As shown, a method for detecting out-of-bounds in a basketball game based on computer vision is provided, and the method includes the following steps:

[0022] Step 100: deploy a plurality of overhead cameras and a plurality of upward cameras at preset positions on the basketball court according to preset coverage rules.

[0023] Specifically, this method requires no additional sideline brackets, offers a wide range of deployment angles, and is effective for detecting surfaces of varying sizes. In contrast, traditional solutions experience reduced sideline recognition accuracy when used on surfaces of varying colors, leading to a decrease in out-of-bounds detection accuracy.

[0024] Multiple overhead cameras and upward cameras are deployed at preset intervals on the sidelines of the basketball court; the deployment angles are determined according to the specific conditions of the basketball court.

[0025] The upward-looking camera determines out-of-bounds, while the downward-looking camera combines the person's posture and basketball tracking to scientifically determine who is out of bounds.

[0026] Step 102: Before the game starts, collect the identity information of each member of the two teams to obtain the personnel identity information.

[0027] Specifically, personal identification information refers to identifiable personal features, including but not limited to player ID, name, team name, and full-length and half-length photos of players. For example, before a match, a player's ID information entry device is used to input the team and each player's ID number, name (optional), and face and full-length photos of each player as personal identification information. This personal identification information is used for personal identification matching when a player goes out of bounds.

[0028] Step 104: When the upward-looking camera recognizes that the pixel integrity of the edge area within its field of view is damaged, it issues an out-of-bounds event warning to the cloud server; the out-of-bounds event warning includes the corresponding time and the number of the current upward-looking camera and several frames before and after the out-of-bounds event warning is generated.

[0029] Specifically, the images shot by the upward-looking camera can instantly capture the out-of-bounds situation, and because the distance to the out-of-bounds position is closer, the judgment accuracy is high.

[0030] The advantage of the upward-looking camera is that it can bypass the obstruction of the baseline and directly detect out-of-bounds situations within the baseline range, while directly analyzing the last touch record to determine the out-of-bounds situation.

[0031] The upward-looking camera's close proximity to the sideline allows for more sensitive detection of out-of-bounds situations. In contrast, traditional approaches, which rely on pixel differences in images captured by the downward-looking camera, often use noise to determine boundary lines, making them susceptible to misjudgment due to environmental disturbances.

[0032] Step 106: The cloud server analyzes the out-of-bounds event warning, retrieves video images from other overhead cameras at the same time, compares the positions of the two images, and uses posture recognition and target tracking algorithms to confirm the out-of-bounds member.

[0033] Specifically, when faced with a player's actual game save attempt, this method can distinguish whether the out-of-bounds object is above the sideline or has already stepped on it. Traditional solutions, due to their limited overhead camera angle, cannot distinguish this, leading to misjudgments. Furthermore, based on posture recognition and target tracking technology, this method can more accurately capture upper limb contact records, providing a more precise basis for out-of-bounds judgments. This method does not require the additional recording of specific body parts (such as shoes) to detect out-of-bounds.

[0034] Pose recognition can be implemented using algorithms such as YOLO-Pose and Mediapipe, while target tracking can be achieved using the YOLO algorithm. Based on pose recognition, the coordinates of the player's wrist key points are extracted, and based on target tracking, the real-time position coordinates of the basketball are extracted. The difference between the coordinates of each player's wrist key point and the basketball's position in the image is calculated and recorded. If the difference is below a certain value, it is recorded as a touch event. The current timestamp and the identity of the player who touched the ball are recorded. The out-of-bounds player is determined based on the most recent touch event at the time of the out-of-bounds event.

[0035] Step 108: Re-identify the out-of-bounds member based on the personnel identity information to obtain the number and name of the out-of-bounds member.

[0036] Step 110: The cloud server sends the out-of-bounds event information and corresponding judgment basis to the penalty result presentation device; the out-of-bounds event information includes the out-of-bounds images of the overhead camera position and the upward camera position of the area corresponding to the out-of-bounds event, the out-of-bounds time, the out-of-bounds person and the team he belongs to.

[0037] The aforementioned computer vision-based out-of-bounds detection method for basketball games involves deploying several overhead cameras and several upward cameras at preset locations on the basketball court according to preset coverage rules. Before the game begins, the identity information of each player on both teams is collected to obtain personal identity information. The upward cameras are used to determine out-of-bounds situations, while the downward cameras combine human posture and basketball tracking to scientifically identify out-of-bounds individuals. This method, using the upward camera's imagery, can instantly capture out-of-bounds situations. Furthermore, due to its closer proximity to the out-of-bounds location, it offers high accuracy and can more sensitively detect out-of-bounds situations. The advantage of the upward camera's position allows it to bypass baseline obstructions and directly detect out-of-bounds situations within the baseline. Furthermore, the method directly analyzes the last touch record to determine out-of-bounds situations.

[0038] In one embodiment, step 104 includes: an upward-looking camera captures an upward-looking game image close to the sideline of the court; the upward-looking camera transmits the image within the current perspective before the game to a cloud server, the cloud server identifies the sidelines in the current image through segmentation technology, and returns the result of the sideline area to the upward-looking camera, sets the boundary area of ​​the current game, and monitors the line integrity of the boundary area; convolution and noise reduction are performed on the current game image to obtain two-dimensional pixel points; pixel integrity is calculated for a specific boundary area, and when the line pixel integrity is lower than a preset threshold, it is considered out of bounds; the upward-looking camera returns the current timestamp and several frames of video images near the timestamp to the cloud server, and issues an out-of-bounds event warning.

[0039] In one embodiment, the sideline monitoring process includes: before the start of the game, the upward-looking camera sends the current field of view to the cloud server, and the cloud server uses an image segmentation method to segment the image corresponding to the current image and identify the sideline position; after identifying the sideline position, a semantic segmentation method is used to process potential occlusions in the image, perform noise reduction, and convert discrete sideline areas into continuous areas.

[0040] Specifically, before the game begins, the overhead camera sends the current field of view to the cloud server, which then uses image segmentation techniques from computer vision algorithms to segment the current sidelines. This technology improves the robustness and accuracy of sideline recognition. Through semantic segmentation, this technology can intelligently identify objects within the image and accurately mark the current sideline, ignoring any obstructions. This allows for more robust sideline monitoring despite varying lighting conditions on various courts. Semantic segmentation utilizes deep learning techniques based on sideline recognition images from previous matches, specifically a convolutional neural network (CNN) framework. The input for semantic segmentation is the corresponding image, and the output is the identified sideline location within the current frame. After identifying the sideline location, semantic segmentation is used to address potential occlusions, perform noise reduction, and convert discrete sideline regions into continuous regions.

[0041] In one embodiment, step 106 includes: after the cloud server receives the out-of-bounds event warning, querying the images of the remaining overhead cameras according to the time report in the out-of-bounds event warning; using a personnel recognition method to identify the person according to the current image of the overhead camera and the image of the upward camera at the current time, and obtaining the identity information of the person in the video image; by extracting features from several frames of the upward camera and the downward camera before the out-of-bounds event warning occurs, determining the person corresponding to each part in the two perspective images based on feature similarity; according to the identity information obtained after personnel identification, the person in the identification result is regarded as the out-of-bounds person, and the out-of-bounds person and the out-of-bounds time point information, and the out-of-bounds video images of the upward camera and the downward camera are returned to the penalty result presentation device; if the identity of the corresponding person cannot be identified in the current frame, then deducing forward according to the time point of the out-of-bounds event warning, judging in sequence according to the several frames of images before the out-of-bounds event warning is generated until the identity information of the current out-of-bounds person is sufficient to be determined.

[0042] In one embodiment, the process is deduced forward based on the time point of the out-of-bounds monitoring event, and the image frames before the out-of-bounds event warning is generated are judged in sequence until the identity information of the current out-of-bounds person is sufficient to be determined, including: extracting the video image at the time of the out-of-bounds event warning based on the overhead camera of the out-of-bounds event warning area, using human posture recognition technology to perform posture modeling on the person in the image, obtaining a corresponding human key point bitmap, and finding the key node coordinates of the corresponding wrist node position; using object tracking technology to identify the basketball and locate the basketball position coordinates based on a pre-trained object recognition model; replaying the video image for several frames in reverse, continuously extracting the position coordinates of the basketball in the image, the identity number of the person in the image, and the corresponding wrist node coordinates, and calculating the difference between the basketball position coordinates and the wrist node coordinates, taking the identity number of the person with the smallest difference from the basketball according to the timestamp from large to small, deeming the person corresponding to the number as the out-of-bounds person, and recording the timestamp and saving it as the basis for judging the out-of-bounds event; and sending the basis for judging the out-of-bounds event, the out-of-bounds person, and the time points of the video image required for judging the identity information of the out-of-bounds person back to the penalty result presentation device.

[0043] Specifically, after the cloud server receives the out-of-bounds event warning, it will query the images of the remaining overhead camera positions based on the time report in the out-of-bounds event warning content, and further compare the current image with the image of the upward camera position at the current time. The identity information of the person in the video image will be obtained through personnel recognition technology. By extracting features from several frames of images before the out-of-bounds monitoring event occurs at the upward camera position, and performing personnel re-identification analysis with the image of the perspective at the time point of the out-of-bounds monitoring event, the corresponding person in each part of the two perspective images is determined based on feature similarity. Based on the identity information obtained after personnel identification, the person in this identification result is regarded as an out-of-bounds person, and this result, together with the out-of-bounds time point information, the out-of-bounds video image (including the upward camera position and the downward camera position) are returned to the penalty result presentation device.

[0044] If the person's identity cannot be identified in the current frame, the system then extrapolates forward from the time of the out-of-bounds event, using frames from several frames preceding the time the out-of-bounds event warning was generated, until sufficient information is available to determine the current out-of-bounds person's identity. Previous video footage is extracted from a camera overlooking the area where the out-of-bounds event occurred. Human gesture recognition technology is used to model the person's posture, generating a human key point map and finding the key node coordinates of the wrist node. Simultaneously, object tracking technology is used to identify the basketball and locate its coordinates using a pre-trained object recognition model. The video is then replayed several frames in reverse, continuously extracting the basketball's coordinates, the person's ID number, and the corresponding wrist node coordinates. The difference between the basketball's coordinates and the wrist node coordinates is calculated. After the calculation is complete, the ID number of the person with the smallest difference from the basketball is selected, ranked from largest to smallest timestamp. This person is considered out-of-bounds, and the timestamp is recorded and saved as the basis for determining the out-of-bounds event. This result, along with the time point of the video frame required to determine the out-of-bounds person's identity, is then transmitted back to the penalty result presentation device.

[0045] In one embodiment, step 108 includes: when the cloud server receives the out-of-bounds event warning, it uses the ReID personnel re-identification technology to identify the current player based on the out-of-bounds member and the personnel identity information to obtain the number and name of the out-of-bounds person.

[0046] Specifically, the process of personnel identification includes:

[0047] 1) Data entry stage: Before the game begins, players will enter their corresponding information and facial information on this device in the form of pictures. These pictures will then be uploaded to the cloud server for storage and will be used later to compare the identity of the out-of-bounds players.

[0048] 2) Identity verification after out-of-bounds detection: When the cloud server identifies the out-of-bounds player after receiving an out-of-bounds detection warning, person recognition technology is required to identify the player in the video. This system uses ReID (Re-identification) technology to identify the current player. This technology extracts the person's image from the current video frame, extracts image features, and then matches these features with the incoming person features in the database. The result with the highest match is determined as the current player's identity.

[0049] It should be understood that although Figure 1The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0050] In one embodiment, a computer vision-based out-of-bounds detection system for basketball games is provided. The system includes a player information input device, an out-of-bounds monitoring device, a cloud server, and a device for presenting penalty results. This device can be directly equipped with more game event recognition functions based on the existing framework.

[0051] The personnel information input device is used to input the identity information of each member of the two teams before the game to obtain the personnel identity information.

[0052] Specifically, the personnel information entry device is used to enter the team, each player's number, name (optional), and face and full-body photos before the start of the game. It is used to match the personnel identity information when the player goes out of bounds.

[0053] The monitoring equipment for out-of-bounds in the game includes several overhead cameras and several upward cameras; the several overhead cameras and several upward cameras are deployed at preset positions on the basketball court according to preset coverage rules; the upward cameras are used to shoot the game pictures from an upward angle and upload them to the cloud server, and are also used to calculate and detect the pixel integrity of the pictures, and when it is judged to be out of bounds, an out-of-bounds event warning is issued to the cloud server; the overhead cameras are used to shoot the game video pictures and upload them to the cloud server; the out-of-bounds event warning includes the corresponding time and the number of the current upward camera and several frames before and after the out-of-bounds event warning is generated.

[0054] Specifically, the upward-looking camera performs pixel integrity calculations for images, detects and issues out-of-bounds warnings and judgments to the cloud server, and captures and uploads images. This functionality is typically implemented using a small development board. The downward-looking camera only features video recording and uploading. These multiple cameras are installed at specific locations on the basketball court according to specific coverage rules.

[0055] The upward-facing camera is responsible for providing an upward-angle view of the game, close to the sidelines. Before the game, the upward-facing camera transmits its current field of view to a cloud server. The cloud server uses image segmentation technology to identify the sidelines within the image and returns the results of the sideline area to the upward-facing camera. The upward-facing camera then uses this to define the boundary area for the game and monitor line integrity within this area. Simple convolution and noise reduction processing are performed on the current game footage to generate two-dimensional pixel points. Pixel integrity calculations within specific boundary areas determine whether the player is out of bounds. When the pixel integrity falls below a certain threshold, the player is considered out of bounds. The upward-facing camera returns the current timestamp and the last few frames of video to the server, issuing an out-of-bounds warning.

[0056] The cloud server is used to process the video images uploaded by the overhead camera and the upward camera, analyze the out-of-bounds event warning, retrieve the video images of other overhead cameras at the same time, compare the positions between the two images, and use posture recognition and target tracking algorithms to confirm the out-of-bounds members; re-identify the out-of-bounds members based on their identity information to obtain the number and name of the out-of-bounds person.

[0057] Specifically, the cloud server is responsible for processing uploaded videos, monitoring out-of-bounds events, analyzing the people involved in out-of-bounds events, and sending the out-of-bounds event results to the penalty result presentation device.

[0058] The penalty result presentation device is used to present the complete result of the out-of-bounds event information for the referee to make a penalty decision; the out-of-bounds event information includes the out-of-bounds images from the overhead and upward camera positions of the area corresponding to the out-of-bounds event, the out-of-bounds time, the out-of-bounds person and the team he belongs to.

[0059] Specifically, the devices for presenting penalty results include but are not limited to dedicated display screens outside the competition venue, mobile phones, and display screens facing the audience.

[0060] The computer vision-based out-of-bounds detection system for basketball games works as follows: First, before the game begins, both players enter their personal information, including photos of their current attire and facial images, for use in subsequent identification and identification. This information is uploaded to a backend server for storage. Next, the upward-facing camera within the out-of-bounds detection device is initialized. It captures the current scene and transmits the footage to a cloud server. The cloud server uses segmentation technology to identify the corresponding sidelines and returns the values ​​of the sideline areas in the image to each upward-facing camera. This ensures sideline monitoring. Next, the device is calibrated. Before the game begins, the upward-facing camera monitors the pixel integrity of the sideline area within its field of view, while the downward-facing camera continuously records and transmits this video to the cloud server. If the upward-facing camera detects a violation of the pixel integrity of the sideline area within its field of view, it issues an out-of-bounds warning. The out-of-bounds warning includes the corresponding time, the current upward-facing camera number, and the number of frames before and after the out-of-bounds warning. The cloud server then analyzes the out-of-bounds event, accessing footage from other overhead cameras during the same time period to identify the individuals in the footage. By comparing the positions of the two footages, gesture recognition and target tracking technology are used to determine the last person to touch the ball and determine the player who was out of bounds. Re-identification is performed based on the individual data collected before the match, obtaining the specific number and name of the person who was out of bounds. The cloud server then transmits the out-of-bounds event information to the penalty result presentation device for the referee to make a timely decision. This out-of-bounds event information includes the overhead and overhead camera footage of the area where the out-of-bounds event occurred, the time of the out-of-bounds, the individual and their team, and the basis for the decision. The referee can view the penalty result and the corresponding footage on the penalty result presentation device and can play the entire video clip as needed.

[0061] In one embodiment, the cloud server is further used to, after receiving an out-of-bounds event warning, query the images of the remaining overhead cameras according to the time report in the out-of-bounds event warning, and use a personnel recognition method to identify the personnel based on the current image of the overhead camera and the image of the upward camera at the current time to obtain the identity information of the personnel in the video image; by extracting features from several frames of the upward camera and the downward camera before the out-of-bounds event warning occurs, determine the personnel corresponding to each part in the two perspective images based on feature similarity; based on the identity information obtained after personnel identification, regard the personnel in the identification result as the out-of-bounds personnel, and return the out-of-bounds personnel and the out-of-bounds time point information, and the out-of-bounds video images of the upward camera and the downward camera to the penalty result presentation device; if the identity of the corresponding personnel cannot be identified in the current frame, then deduce forward based on the time point of the out-of-bounds event warning, and judge in sequence based on the several frames of images before the out-of-bounds event warning is generated until it is sufficient to determine the identity information of the current out-of-bounds personnel.

[0062] Specifically, this system uses visual algorithms to detect out-of-bounds situations in basketball games. This system monitors every sideline in real time during a game and, immediately after an out-of-bounds situation occurs, uses algorithms to identify the location, person, and time of the out-of-bounds situation, thereby detecting out-of-bounds events.

[0063] In one embodiment, the cloud server is also used to extract the video footage of the time when the out-of-bounds event warning occurs based on the overhead camera of the out-of-bounds event warning area, use human posture recognition technology to model the posture of the characters in the picture, obtain the corresponding human key point bitmap, and find the key node coordinates of the corresponding wrist node position; use object tracking technology to identify the basketball and locate the basketball position coordinates based on the pre-trained object recognition model; play the video picture in reverse for several frames, continuously extract the position coordinates of the basketball in the picture and the identity number of the person in the picture and the corresponding wrist node coordinates, and calculate the difference distance between the basketball position coordinates and the wrist node coordinates, take the identity number of the person with the smallest difference from the basketball from large to small according to the timestamp, regard the person corresponding to the number as the out-of-bounds person, and record the timestamp and save it as the basis for judging the out-of-bounds event; send the basis for judging the out-of-bounds event, the out-of-bounds person, and the time point of the video picture required for judging the identity information of the out-of-bounds person back to the penalty result presentation device.

[0064] In one embodiment, the overhead camera is also used to shoot game images from an upward angle close to the sidelines of the court; the images in the current angle of view before the game are transmitted to the cloud server, and the cloud server receives the results of the sideline area obtained by using segmentation technology to identify the sidelines in the current image, and sets the results of the sideline area as the boundary area of ​​the current game, and monitors the line integrity of the boundary area; convolution and noise reduction processing are performed on the current game image to obtain two-dimensional pixel points; pixel integrity is calculated for a specific boundary area, and when the line pixel integrity is lower than a preset threshold, it is considered out of bounds; then the current timestamp and several frames of video images near the timestamp are returned to the cloud server, and an out-of-bounds event warning is issued.

[0065] The specific limitations of the computer vision-based basketball out-of-bounds detection system can be found in the limitations of the computer vision-based basketball out-of-bounds detection method described above and will not be further elaborated here. Each module in the aforementioned computer vision-based basketball out-of-bounds detection system can be implemented in whole or in part through software, hardware, or a combination thereof. Each of these modules can be embedded in or independent of a processor in a computer device in hardware form, or stored in a computer device memory in software form, so that the processor can call and execute the corresponding operations of each of these modules.

[0066] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0067] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A basketball game out-of-bounds detection method based on computer vision, characterized in that: The method comprises: Deploy several overhead cameras and several upward cameras at preset locations on the basketball court according to preset coverage rules; Before the game starts, the identity information of each member of both teams is collected to obtain the personnel identity information; When the upward-looking camera detects that the pixel integrity of the edge area within its field of view is violated, it issues an out-of-bounds event warning to the cloud server; the out-of-bounds event warning includes the corresponding time and the number of frames before and after the out-of-bounds event warning is generated; The cloud server analyzes the out-of-bounds event warning, retrieves video images from other overhead cameras during the same period, compares the positions of the two images, and uses gesture recognition and target tracking algorithms to identify the out-of-bounds player. The process of identifying the out-of-bounds player includes: extracting the coordinates of the key points of the player's wrist based on gesture recognition, extracting the real-time position coordinates of the basketball based on target tracking, calculating and recording the difference between the coordinates of the key points of each player's wrist and the position coordinates of the basketball in the image, and recording it. If the difference is lower than a certain value, it is recorded as a touch event, and the current timestamp and identity information of the player who touched the ball are recorded. The out-of-bounds player is determined based on the most recent touch event when the out-of-bounds event occurs. Re-identify the out-of-bounds member based on the personnel identity information and obtain the number and name of the out-of-bounds member; The cloud server sends the out-of-bounds event information and the corresponding judgment basis to the penalty result presentation device; the out-of-bounds event information includes the out-of-bounds images of the overhead camera position and the upward camera position of the corresponding area where the out-of-bounds event occurred, the out-of-bounds time, the out-of-bounds person and the team he belongs to.

2. The method for detecting out-of-bounds in a basketball game based on computer vision according to claim 1, wherein: When the upward-looking camera detects that the pixel integrity of the edge area within its field of view is violated, it issues an out-of-bounds event warning to the cloud server, including: The upward-looking camera shoots the game from an upward angle close to the sidelines of the court; The upward-looking camera transmits the image within its current viewing angle before the match to the cloud server. The cloud server uses segmentation technology to identify the sidelines in the current image and returns the sideline area results to the upward-looking camera. This determines the boundary area for the match and monitors the line integrity of the boundary area. Perform convolution and noise reduction on the current game image to obtain two-dimensional pixel points; Calculate pixel integrity for a specific boundary area. If the pixel integrity of a line is lower than a preset threshold, it is considered out of bounds. The upward-looking camera returns the current timestamp and several frames of video near the timestamp to the cloud server, and issues an out-of-bounds event warning.

3. The method for detecting out-of-bounds in a basketball game based on computer vision according to claim 1, wherein: The sideline monitoring process includes: Before the game starts, the upward-looking camera sends the current field of view to the cloud server, which uses image segmentation methods to segment the image corresponding to the current scene and identify the sideline position; After identifying the edge position, the semantic segmentation method is used to deal with potential occlusions in the picture, perform noise reduction, and convert discrete edge areas into continuous areas.

4. The method for detecting out-of-bounds in a basketball game based on computer vision according to claim 1, wherein: The cloud server analyzes the out-of-bounds event warning, retrieves video images from other overhead cameras during the same period, compares their positions, and uses gesture recognition and target tracking algorithms to identify the out-of-bounds member, including: After receiving the out-of-bounds event warning, the cloud server queries the images of other overhead cameras according to the time report in the out-of-bounds event warning; Based on the current image of the overhead camera and the image of the upward camera at the current time, a person identification method is used to identify people and obtain the identity information of the people in the video image; By extracting features from several frames of the upward-looking camera and the downward-looking camera before the out-of-bounds event warning occurs, the corresponding person in each part of the two perspective images is determined based on feature similarity; Based on the identity information obtained after personnel identification, the identified person is deemed to be out of bounds, and the out-of-bounds person and the time of the out-of-bounds information, as well as the out-of-bounds video images from the upward and downward camera positions, are returned to the penalty result presentation device; If the identity of the corresponding person cannot be identified in the current frame, the time point of the out-of-bounds event warning is deduced forward, and several frames before the out-of-bounds event warning are judged in sequence until the identity information of the current out-of-bounds person is sufficient to be determined.

5. The method for detecting out-of-bounds in a basketball game based on computer vision according to claim 4, wherein: The process is then deduced forward based on the time when the out-of-bounds monitoring event occurred, and judged in sequence based on the images of several frames before the out-of-bounds event warning was generated, until sufficient information on the identity of the person who has currently gone out of bounds is determined, including: The video footage of the out-of-bounds warning occurs is extracted from the camera with an overhead view of the area where the out-of-bounds warning occurs. The human posture recognition technology is used to perform posture modeling on the person in the video, obtain a bitmap of the corresponding human key points, and find the key node coordinates of the corresponding wrist node position. Using object tracking technology, the pre-trained object recognition model is used to identify the basketball and locate its coordinates. Rewind the video for several frames, continuously extract the position coordinates of the basketball in the picture, the identity number of the person in the picture, and the corresponding wrist node coordinates, and calculate the difference between the basketball position coordinates and the wrist node coordinates. According to the timestamps from large to small, the identity number of the person with the smallest difference from the basketball is taken, and the person corresponding to the number is considered to be out of bounds. The timestamp is recorded and saved as the basis for judging the out-of-bounds event; The basis for judging the out-of-bounds event, the out-of-bounds person, and the time point of the video image required for judging the identity information of the out-of-bounds person are sent back to the penalty result presentation device.

6. The method for detecting out-of-bounds in a basketball game based on computer vision according to claim 1, wherein: Re-identify the out-of-bounds member based on their identity information and obtain their number and name, including: When the cloud server receives an out-of-bounds event warning, it uses ReID personnel re-identification technology to identify the current player based on the out-of-bounds member and the personnel identity information, and obtains the number and name of the out-of-bounds person.

7. A basketball game out-of-bounds detection system based on computer vision, characterized in that: The system includes: a personnel information input device, a game out-of-bounds monitoring device, a cloud server, and a penalty result presentation device; The personnel information input device is used to input the identity information of each member of the two teams before the game to obtain the personnel identity information; The out-of-bounds monitoring equipment includes a plurality of overhead cameras and a plurality of upward cameras; the plurality of overhead cameras and the plurality of upward cameras are deployed at preset positions on the basketball court according to preset coverage rules; the upward cameras are used to capture game images from an upward angle and upload them to a cloud server, and are also used to calculate and detect pixel integrity of the images, and issue an out-of-bounds event warning to the cloud server when it is determined to be out of bounds; the overhead cameras are used to capture game video images and upload them to the cloud server; the out-of-bounds event warning includes the corresponding time and the number of the current upward camera, as well as a plurality of frames before and after the time when the out-of-bounds event warning is generated; The cloud server is used to process the video images uploaded by the overhead camera and the upward camera, analyze the out-of-bounds event warning, and confirm the out-of-bounds member by using posture recognition and target tracking algorithms by retrieving the video images of other overhead cameras at the same time and comparing the positions between the two images; re-identify the out-of-bounds member based on the person's identity information to obtain the number and name of the out-of-bounds member; the process of confirming the out-of-bounds member includes: extracting the coordinates of the key points of the player's wrist based on posture recognition, extracting the real-time position coordinates of the basketball based on target tracking, calculating and recording the difference between the coordinates of the key points of the wrist of each player and the position coordinates of the basketball in the image; if the difference is lower than a certain value, it is recorded as a touch event, recording the current timestamp and the identity information of the touch player, and determining the out-of-bounds member based on the most recent touch event when the out-of-bounds event occurs; The penalty result presentation device is used to present the complete result of the out-of-bounds event information for the referee to make a penalty decision; the out-of-bounds event information includes the out-of-bounds images from the overhead and upward camera positions of the area corresponding to the out-of-bounds event, the out-of-bounds time, the out-of-bounds person and the team he belongs to.

8. The computer vision-based basketball out-of-bounds detection system according to claim 7, characterized in that: The cloud server is also used to query the images of other overhead cameras according to the time report in the out-of-bounds event warning after receiving the out-of-bounds event warning, and use a personnel recognition method to identify the personnel according to the current image of the overhead camera and the image of the upward camera at the current time to obtain the identity information of the personnel in the video image; by extracting features from several frames of images of the upward camera and the downward camera before the out-of-bounds event warning occurs, determine the personnel corresponding to each part in the two perspective images based on feature similarity; according to the identity information obtained after personnel identification, regard the personnel in the identification result as the out-of-bounds personnel, and return the out-of-bounds personnel and the out-of-bounds time point information, and the out-of-bounds video images of the upward camera and the downward camera to the penalty result presentation device; if the identity of the corresponding personnel cannot be identified in the current frame, then deduce forward according to the time point of the out-of-bounds event warning, and judge in sequence according to the several frames of images before the out-of-bounds event warning is generated until it is sufficient to determine the identity information of the current out-of-bounds personnel.

9. The computer vision-based basketball out-of-bounds detection system according to claim 7, characterized in that: The cloud server is also used to extract the video picture of the time when the out-of-bounds event warning occurs based on the overhead camera of the out-of-bounds event warning area, use human posture recognition technology to model the posture of the characters in the picture, obtain the corresponding human key point map, and find the key node coordinates of the corresponding wrist node position; use object tracking technology to identify the basketball and locate the basketball position coordinates based on the pre-trained object recognition model; play the video picture in reverse for several frames, continuously extract the position coordinates of the basketball in the picture and the identity number of the person in the picture and the corresponding wrist node coordinates, and calculate the difference distance between the basketball position coordinates and the wrist node coordinates, take the identity number of the person with the smallest difference from the basketball from large to small according to the timestamp, regard the person corresponding to the number as the out-of-bounds person, record the timestamp, and save it as the basis for judging the out-of-bounds event; send the basis for judging the out-of-bounds event, the out-of-bounds person, and the time point of the video picture required for judging the identity information of the out-of-bounds person back to the penalty result presentation device.

10. The computer vision-based basketball game out-of-bounds detection system according to claim 7, characterized in that: The overhead camera is also used to shoot the game scene from an upward angle close to the sidelines of the court; the picture in the current perspective before the game is transmitted to the cloud server, and the cloud server receives the result of the sideline area obtained by using segmentation technology to identify the sidelines in the current image, and sets the result of the sideline area as the boundary area of ​​this game, and monitors the line integrity of the boundary area; convolution and noise reduction processing are performed on the current game picture to obtain two-dimensional pixel points; pixel integrity is calculated for a specific boundary area, and when the line pixel integrity is lower than a preset threshold, it is considered out of bounds; then the current timestamp and several frames of video pictures near the timestamp are returned to the cloud server, and an out-of-bounds event warning is issued.

Images