A method and video shooting device for automatically generating player point map in basketball games

Abstract

The invention relates to the technical field of sports event video recording, and discloses a method and a video shooting device for automatically generating a player point map for a basketball game. The method calculates a homography matrix between a current picture and a basketball court plane map through a calibrated parameter matrix of a wide-angle camera and corner information of a binary segmentation map of a basketball court. The image is subjected to a homography transformation or a perspective transformation through the homography matrix to map the positions of the players onto the basketball court plane map. The generated point map is integrated into an original video image and outputted, thereby realizing displaying the positions of the players while shooting and recording a video image of a basketball game. The method can automatically generate a point map of the plane positions of the players of both teams on the basketball court in real time, which is convenient for tactical analysis and planning during or after the game, and greatly improves the efficiency and intelligence of point map drawing.

Description

Technical Field

[0001] The present invention relates to the technical field of sports event video recording, and in particular to a method for automatically generating a player point map for a basketball game and a video shooting device. Background Art

[0002] At present, the planar position diagrams (point maps) of athletes in existing basketball games are mostly drawn manually. After the game, professionals watch the game video and draw the positions of each athlete in the game video accordingly into the point map. Manually drawing the point map of athletes is time-consuming, labor-intensive and inefficient. Although some software can make the work of drawing trajectories easier, these software need to be used after the game and cannot generate point maps while the game video is being shot and recorded. Summary of the invention

[0003] The main purpose of the present invention is to propose a method and a video shooting device for automatically generating a player position map in a basketball game, aiming to generate a planar position diagram (position map) of the players while shooting and recording the basketball game video, which is used for tactical analysis and planning during or after the game.

[0004] To achieve the above object, the method for automatically generating a player position map for a basketball game proposed by the present invention comprises the following steps:

[0005] S1: selecting a position of a wide-angle camera, and using the wide-angle camera to capture a full-court picture of a basketball game;

[0006] S2: Calibration of the wide-angle camera and distortion correction of the image. The wide-angle camera image is distorted by modifying the parameter matrix.

[0007] S3: Basketball court boundary detection and image segmentation: Through the encoding-decoding model, the basketball court boundary detection and image binary segmentation are completed;

[0008] S4: Athlete detection and tracking, which generates personnel tracking information through multi-target detection and tracking models, and uses filters to combine basketball court boundary detection and image binary segmentation information to identify the athletes on the court and track them in real time. The two teams of athletes and referees are distinguished through color filters;

[0009] S5: Athlete position mapping: the homography matrix between the current image and the basketball court plane is calculated through the parameter matrix of the calibrated wide-angle camera and the corner information of the binary segmentation map of the basketball court. The image is subjected to homography transformation or perspective transformation through the homography matrix to complete the mapping of the athlete position to the basketball court plane.

[0010] S6: Automatically generate a video image with a bitmap, merge the bitmap generated in step S5 into the original video image and output it, and display the positions of each athlete in the video image.

[0011] Furthermore, the specific method of performing distortion correction on the wide-angle camera image by changing the parameter matrix in step S2 is as follows:

[0012] Step S2.1: The wide-angle camera takes 10 to 20 calibration images of the calibration plate printed with black and white chessboard at different positions, angles and postures;

[0013] Step S2.2: extracting vertex position information of each corner on each calibration image;

[0014] Step S2.3: Calculate the translation vector and rotation vector through optimization iteration;

[0015] Step S2.4: Calculate the parameter matrix KK of the wide-angle camera;

[0016] Step S2.5: Use the obtained parameter matrix KK to perform geometric transformation on the image, thereby completing the distortion correction of the wide-angle camera image.

[0017] Furthermore, in step S4, the working method of the filter is specifically as follows:

[0018] Step S4.1: Calculate the coordinate position of the bottom midpoint of the box based on the person detection and the box finally provided;

[0019] Step S4.2: Based on the binary segmentation information, determine whether the midpoint position of the box in the previous step is within the white area;

[0020] Step S4.3: If the midpoint is within the white area, keep the box; otherwise, discard the box.

[0021] Furthermore, in step S5, the method for solving and transforming the homography matrix is ​​as follows:

[0022] Step S5.1: According to the corner point information of the binary segmentation image of the court in the video, the coordinates x(i) and y(i) of the four corner points are obtained, where i=1-4;

[0023] Step S5.2: The height-to-width ratio of the basketball court is a fixed value. The coordinates of the four corner points, x^(i) and y^(i), are set according to the height-to-width ratio, where i= i=1 to 4, and the numerical values ​​are in the same order of magnitude as the numbers in the first step;

[0024] Step S5.3: The two sets of coordinates satisfy the following relationship:

[0025]

[0026] Substituting the coordinates of each point into the expansion, we get:

[0027] ;

[0028] Step S5.4: Set h 33 Set it to 1 and solve it by least squares estimation to get the homography matrix H;

[0029] Step S5.5: Use the H matrix to calculate the video coordinates x of each player 2 and 2 The information is transformed to generate the point coordinates x 1 and 1 , the conversion formula is as follows:

[0030] .

[0031] Furthermore, the present invention also proposes a video shooting device for automatically generating a player position map of a basketball game, comprising a camera body, a support rod and a tripod, wherein the upper end of the support rod is detachably fixedly connected to the lower end wall of the camera body, the lower end of the support rod is connected to the upper end of the tripod so as to slide up and down and rotate left and right, the front end of the camera body is provided with a wide-angle camera and a microphone, the upper end of the camera body is provided with a viewfinder eyepiece, and the camera body is provided with a main control chip and a CMOS image sensor and an image signal processing unit electrically connected to the main control chip respectively.

[0032] Furthermore, a threaded hole is provided at the upper end of the support rod, and a screw rod is protruding from the lower end wall of the camera body, and the screw rod is detachably screwed into the threaded hole.

[0033] Furthermore, it also includes a fastening screw. The upper end of the tripod is provided with a mounting sleeve. The lower end of the support rod is embedded in the mounting sleeve so as to slide up and down and rotate left and right. The fastening screws are respectively screwed on the outer peripheral wall of the upper end of the mounting sleeve, and the front end of the fastening screw can be respectively abutted against the outer peripheral wall of the support rod.

[0034] Furthermore, a handle is provided at the upper end of the camera body.

[0035] Furthermore, the resolution of the wide-angle camera is 2K or 4K.

[0036] Furthermore, the main control chip adopts NXP's ASC8850 microprocessor chip, and the CMOS image sensor adopts SONY's ICX445CMOS image sensor.

[0037] The technical solution of the present invention has the following beneficial effects: the technical solution of the present invention calculates the homography matrix of the current picture and the basketball court plan through the calibrated parameter matrix of the wide-angle camera and the corner information of the binary segmentation map of the basketball court, performs homography transformation or perspective transformation on the image through the homography matrix, completes the mapping of the player's position to the basketball court plan, and integrates the generated point map into the original video image for output, thereby realizing the display of each player's position while shooting and recording the video image of the basketball game, and can automatically generate a point map of the plane position of the players of both teams on the basketball court in real time, which is convenient for tactical analysis and planning during or after the game, and greatly improves the efficiency and intelligence of point map drawing. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0039] Figure 1 This is a diagram of the overall process steps of a method for automatically generating a player position map for a basketball game proposed by the present invention;

[0040] Figure 2 A flow chart of the steps of basketball court boundary detection and image binary segmentation in a method for automatically generating a player point map for a basketball game proposed by the present invention;

[0041] Figure 3 This is a schematic diagram of the encoder and decoder architecture of a method for automatically generating a player point map for a basketball game proposed by the present invention;

[0042] Figure 4 A schematic diagram of the process of detecting and tracking players in a method for automatically generating a player point map in a basketball game proposed by the present invention;

[0043] Figure 5 This is an overall structural diagram of a video shooting device for automatically generating a player position map for a basketball game proposed by the present invention;

[0044] Figure 6 This is an overall structural diagram from another perspective of a video shooting device for automatically generating a player position map for a basketball game proposed by the present invention.

[0045] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0048] In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the fact that ordinary technicians in the field can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0049] The invention provides a method for automatically generating a player point map in a basketball game and a video shooting device.

[0050] like Figures 1 to 4 As shown, in one embodiment of the present invention, the method for automatically generating a player point map of a basketball game includes the following steps: S1: selecting the position of a wide-angle camera, and shooting the full-court picture of the basketball game through the wide-angle camera; S2: calibration of the wide-angle camera and distortion correction of the picture, and performing distortion correction on the wide-angle camera image through a modified parameter matrix; S3: basketball court boundary detection and image segmentation, and completing basketball court boundary detection and image binary segmentation through a coding-decoding model. The process steps are shown in the figure. Figure 2 As shown, the architecture of Encoder and Decoder is as follows Figure 3 As shown; S4: Athlete detection and athlete tracking, through the multi-target detection and tracking model to generate personnel tracking information, and through the filter, combined with the basketball court boundary detection and image binary segmentation information, determine the athletes on the court and track them in real time, the two teams of athletes and referees of the game are distinguished through color filters, the process steps are as follows Figure 4As shown; S5: athlete position mapping, through the calibrated wide-angle camera parameter matrix and the corner information of the binary segmentation map of the basketball court, the homography matrix of the current picture and the basketball court plan is calculated, and the image is subjected to homography transformation or perspective transformation through the homography matrix to complete the mapping of the athlete position to the basketball court plan; S6: automatically generate a video image with a point map, merge the point map generated in step S5 into the original video image for output, and display the position of each athlete in the video image. The point map position of each athlete is determined by the lowest point of the midline of the border tracked by the on-site athlete in step S4.

[0051] Optionally, the specific method of performing distortion correction on the wide-angle camera image by changing the parameter matrix in step S2 is as follows:

[0052] Step S2.1: The wide-angle camera takes 10 to 20 calibration images of the calibration plate printed with black and white chessboard at different positions, angles and postures;

[0053] Step S2.2: extract the vertex position information of each corner on each calibration image, and the corner detector adopts Harris or Shi-Tomasi;

[0054] Step S2.3: Calculate the translation vector and rotation vector through optimization iteration;

[0055] Step S2.4: Calculate the parameter matrix KK of the wide-angle camera;

[0056] Step S2.5: Use the obtained parameter matrix KK to perform geometric transformation on the image, thereby completing the distortion correction of the wide-angle camera image.

[0057] Among them, the pixel position x of the calibrated image p and p , and the pixel position x of the image before calibration d (1) and x d The relationship between (2) is:

[0058]

[0059] KK is the camera parameter matrix. By using this matrix and the above formula, we can make corresponding position transformations for each point on the distorted image, thereby correcting the image. The description of the parameters in the KK camera parameter matrix is ​​as follows:

[0060]

[0061] in,

[0062] fc (1) and fc (2) The focal length of the camera in pixels in each direction;

[0063] cc (1) and cc (2) is the coordinates of the camera principal point in pixels;

[0064] alpha_c is the skewness coefficient;

[0065] kc is the distortion parameter of the image, including radial distortion and tangential distortion;

[0066] The above parameters are all obtained by the steps of camera parameter calculation.

[0067] Optionally, in step S4, the working method of the filter is specifically as follows:

[0068] Step S4.1: Calculate the coordinate position of the bottom midpoint of the box based on the person detection and the box finally provided;

[0069] Step S4.2: Based on the binary segmentation information, determine whether the midpoint position of the box in the previous step is within the white area;

[0070] Step S4.3: If the midpoint is within the white area, keep the box; otherwise, discard the box.

[0071] Optionally, in step S5, the method for solving and transforming the homography matrix is ​​specifically as follows:

[0072] Step S5.1: According to the corner point information of the binary segmentation image of the court in the video, the coordinates x(i) and y(i) of the four corner points are obtained, where i=1-4;

[0073] Step S5.2: The height-to-width ratio of the basketball court is a fixed value. The coordinates of the four corner points, x^(i) and y^(i), are set according to the height-to-width ratio, where i= i=1 to 4, and the numerical values ​​are in the same order of magnitude as the numbers in the first step;

[0074] Step S5.3: The two sets of coordinates satisfy the following relationship:

[0075]

[0076] Substituting the coordinates of each point into the expansion, we get:

[0077] ;

[0078] Step S5.4: Set h 33 Set it to 1 and solve it by least squares estimation to get the homography matrix H;

[0079] Step S5.5: Use the H matrix to calculate the video coordinates x of each player 2 and 2The information is transformed to generate the point coordinates x 1 and 1 , the conversion formula is as follows:

[0080] .

[0081] Furthermore, if Figure 5 and Figure 6 As shown, the present invention also proposes a video shooting device for automatically generating a player position map of a basketball game, comprising a camera body 101, a support rod 102 and a tripod 103, wherein the upper end of the support rod 102 is detachably fixedly connected to the lower end wall of the camera body 101, the lower end of the support rod 102 is connected to the upper end of the tripod 103 so as to slide up and down and rotate left and right, a wide-angle camera 1011 and a microphone (not shown) are provided at the front end of the camera body 101, a viewfinder eyepiece 1012 is provided at the upper end of the camera body 101, a main control chip (not shown) and a CMOS image sensor (not shown) and an image signal processing unit (not shown) electrically connected to the main control chip are provided in the camera body 101.

[0082] Optionally, a threaded hole is provided at the upper end of the support rod 102, and a screw rod is protruding from the lower end wall of the camera body 101, and the screw rod is detachably screwed into the threaded hole.

[0083] Optionally, it also includes a fastening screw (not shown), and the upper end of the tripod 103 is provided with a mounting sleeve 1031, and the lower end of the support rod 102 is embedded in the mounting sleeve 1031 so as to slide up and down and rotate left and right, and the fastening screws are respectively screwed on the outer peripheral wall of the upper end of the mounting sleeve 1031, and the front end of the fastening screw can be respectively abutted against the outer peripheral wall of the support rod 102.

[0084] Optionally, a handle 1013 is provided at the upper end of the camera body 101 .

[0085] Optionally, the resolution of the wide-angle camera 1011 is 2K or 4K.

[0086] Optionally, the main control chip adopts NXP's ASC8850 microprocessor chip, and the CMOS image sensor adopts SONY's ICX445CMOS image sensor.

[0087] Specifically, the present invention calculates the homography matrix of the current picture and the basketball court plan through the parameter matrix of the calibrated wide-angle camera and the corner information of the binary segmentation map of the basketball court, and performs homography transformation or perspective transformation on the image through the homography matrix to complete the mapping of the player's position to the basketball court plan, and integrates the generated point map into the original video image for output, thereby realizing the display of each player's position while shooting and recording the video image of the basketball game, and can automatically generate a point map of the plane position of the players of both teams on the basketball court in real time, which is convenient for tactical analysis and planning during or after the game, and greatly improves the efficiency and intelligence of point map drawing.

[0088] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the inventive concept of the present invention, or directly / indirectly applied in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. A method for automatically generating a player point map for a basketball game, characterized in that: The steps include: S1: selecting a position of a wide-angle camera, and using the wide-angle camera to capture a full-court picture of a basketball game; S2: Calibration of the wide-angle camera and distortion correction of the image. The wide-angle camera image is distorted by modifying the parameter matrix. S3: Basketball court boundary detection and image segmentation: Through the encoding-decoding model, the basketball court boundary detection and image binary segmentation are completed; S4: Athlete detection and tracking, which generates personnel tracking information through multi-target detection and tracking models, and uses filters to combine basketball court boundary detection and image binary segmentation information to identify the athletes on the court and track them in real time. The two teams of athletes and referees are distinguished through color filters; S5: Athlete position mapping: the homography matrix between the current image and the basketball court plane is calculated through the parameter matrix of the calibrated wide-angle camera and the corner information of the binary segmentation map of the basketball court. The image is subjected to homography transformation or perspective transformation through the homography matrix to complete the mapping of the athlete position to the basketball court plane. S6: automatically generate a video image with a point map, merge the point map generated in step S5 into the original video image and output it, and display the positions of each athlete in the video image; The specific method of performing distortion correction on the wide-angle camera image by changing the parameter matrix in step S2 is as follows: Step S2.1: The wide-angle camera takes 10 to 20 calibration images of the calibration plate printed with black and white chessboard at different positions, angles and postures; Step S2.2: extracting vertex position information of each corner on each calibration image; Step S2.3: Calculate the translation vector and rotation vector through optimization iteration; Step S2.4: Calculate the parameter matrix KK of the wide-angle camera; Step S2.5: Use the obtained parameter matrix KK to perform geometric transformation on the image, thereby completing the distortion correction of the wide-angle camera image; In step S4, the working method of the filter is as follows: Step S4.1: Calculate the coordinate position of the bottom midpoint of the box based on the person detection and the box finally provided; Step S4.2: Based on the binary segmentation information, determine whether the midpoint position of the box in the previous step is within the white area; Step S4.3: If the midpoint is within the white area, keep the box; otherwise, discard the box; In step S5, the solution and transformation method of the homography matrix is ​​as follows: Step S5.1: According to the corner point information of the binary segmentation image of the court in the video, the coordinates x(i) and y(i) of the four corner points are obtained, where i=1-4; Step S5.2: The height-to-width ratio of the basketball court is a fixed value. The coordinates of the four corner points, x^(i) and y^(i), are set according to the height-to-width ratio, where i= i=1 to 4, and the numerical values ​​are in the same order of magnitude as the numbers in the first step; Step S5.3: The two sets of coordinates satisfy the following relationship: Substituting the coordinates of each point into the expansion, we get: ； Step S5.4: h 33 Set it to 1 and solve it by least squares estimation to get the homography matrix H; Step S5.5: Through the H matrix, transform the video coordinates x2 and y2 of each player to generate point coordinates x1 and y1. The conversion formula is as follows: 。 2. A video shooting device for automatically generating a player position map in a basketball game, used to implement the method of claim 1, characterized in that: The invention comprises a camera body, a support rod and a tripod, wherein the upper end of the support rod is detachably fixedly connected to the lower end wall of the camera body, the lower end of the support rod is connected to the upper end of the tripod so as to slide up and down and rotate left and right, a wide-angle camera and a microphone are arranged at the front end of the camera body, a viewfinder eyepiece is arranged at the upper end of the camera body, a main control chip, a CMOS image sensor and an image signal processing unit which are respectively electrically connected to the main control chip are arranged in the camera body.

3. The video shooting device for automatically generating a player position map for a basketball game according to claim 2, characterized in that: The upper end of the support rod is provided with a threaded hole, and the lower end wall of the camera body is protrudingly provided with a screw rod, and the screw rod is detachably screwed into the threaded hole.

4. The video shooting device for automatically generating a player position map for a basketball game according to claim 2, characterized in that: It also includes a fastening screw. The upper end of the tripod is provided with a mounting sleeve. The lower end of the support rod is embedded in the mounting sleeve so as to slide up and down and rotate left and right. The fastening screws are respectively screwed on the outer peripheral wall of the upper end of the mounting sleeve, and the front end of the fastening screw can be respectively abutted against the outer peripheral wall of the support rod.

5. The video shooting device for automatically generating a player position map for a basketball game according to claim 2, characterized in that: A handle is arranged on the upper end of the camera body.

6. The video shooting device for automatically generating a player position map for a basketball game according to claim 2, characterized in that: The resolution of the wide-angle camera is 2K or 4K.

7. The video shooting device for automatically generating a player position map for a basketball game according to claim 2, characterized in that: The main control chip adopts NXP's ASC8850 microprocessor chip, and the CMOS image sensor adopts SONY's ICX445CMOS image sensor.

Images