Method for measuring and calculating standing positions of athletes in court based on virtual identification points

Abstract

The invention relates to the technical field of television relaying, in particular to a court athlete standing position measuring and calculating method based on virtual identification points, which comprises the following steps of: separating from a current image frame to obtain court side lines, respectively extending towards two ends of each court side line and intersecting with other court side lines to construct virtual identification points, the total number of the virtual identification points and the identification points reaches four, then a homography matrix corresponding to the current image frame is constructed according to the physical position and the pixel position, and the actual position of the athlete is measured according to the homography matrix. In order to solve the problem that the homography matrix in the prior art cannot meet the multi-angle and multi-focal-segment calculation requirements in the rebroadcasting process, the additional virtual identification points are introduced, so that the number of the identification points which can be used for constructing the homography matrix in the current image frame is increased, and the calculation efficiency of the homography matrix is improved. The problem that the homography matrix cannot be calculated in real time due to the fact that the number of the identification points is too small is solved, and the position of an athlete can be directly calculated based on the rebroadcast image in the live rebroadcast process.

Description

Technical Field

[0001] The present invention relates to the technical field of television broadcasting, and in particular to a method for measuring the positions of players on a court based on virtual marking points. Background Art

[0002] On live sports television, we often hear announcers passionately describe the action on the field: "Player No. 10 fired a powerful shot from 25 meters away, and the ball went into the net," "Player No. 20 sprinted 60 meters from the backfield to the front," and "These two players were just 30 meters apart, and now they're so close." These commentaries are filled with data. Traditionally, hosts often rely on their experience to make these calculations, which can lead to biased data.

[0003] To address this issue, in the prior art, a homography matrix is ​​usually constructed based on the correspondence between the camera viewing angle and the physical position, thereby realizing the measurement of the athlete's position based on image data.

[0004] For example, Chinese patent CN202110917836.8 discloses a method and video shooting device for automatically generating a player point map in a basketball game. The method calculates the homography matrix of the current picture and the plane map of the basketball court through the calibrated parameter matrix of the wide-angle camera and the corner information of the binary segmentation map of the basketball court. The image is subjected to homography transformation or perspective transformation through the homography matrix to complete the mapping of the player position to the plane map of the basketball court. The generated point map is integrated into the original video image and output, thereby realizing the display of each player position while shooting and recording the video image of the basketball game. It can automatically generate a point map of the plane position of the players of both teams on the basketball court in real time, which is convenient for tactical analysis and planning during or after the game, greatly improving the efficiency and intelligence of point map drawing.

[0005] For another example, Chinese patent CN202210209801.3 discloses a method for acquiring motion information, a calibration method and a device, which do not require remote athletes to wear wearable sensors, thereby improving the convenience of acquiring motion information. By deploying multiple acquisition devices around the sports venue, athletes are tracked based on the homography matrix between pre-calibrated cameras. Convenience can be provided without the need for athletes to wear any equipment. This application uses mobile calibration objects for calibration, which gets rid of the influence of the inherent visual features / calibration points of the venue, and is applicable to a wider range of scenarios, and is also applicable to scenes with larger venues. The problem of large-field calibration can also be solved without using a camera with a large field of view, which is beneficial to improving the accuracy of target detection and tracking in large scenes.

[0006] However, during the actual implementation process, the inventors found that due to the limitations of the actual broadcast perspective, the homography matrix constructed for a single perspective cannot solve the orientation of objects in the changing perspective during the live broadcast; at the same time, since the homography matrix needs to set multiple reference objects during the calculation process, during the live broadcast process, it is often impossible to ensure that there must be reference objects in the broadcast perspective that can be used for real-time solution, which leads to certain limitations in the application of the above-mentioned solution in the live broadcast perspective. Summary of the Invention

[0007] In view of the above problems existing in the prior art, a method for measuring the positions of athletes on a court based on virtual marking points is provided.

[0008] The specific technical solutions are as follows:

[0009] A method for calculating the position of players on a court based on virtual markers, comprising:

[0010] Step S1: Acquire a current image frame, identify the identification points in the current image frame, and turn to step S2 when the number of the identification points is less than four;

[0011] Step S2: Separating court sidelines from the current image frame, extending each court sideline to both ends and intersecting with other court sidelines to construct virtual marker points, so that the total number of virtual marker points and marker points reaches four;

[0012] Step S3: respectively determining the physical positions of the virtual marking point and the marking point in the court and the pixel positions in the current image frame, and then constructing a homography matrix corresponding to the current image frame according to the physical positions and the pixel positions;

[0013] Step S4: Obtain the pixel position of the player from the current image frame, and calculate the actual position of the player according to the homography matrix.

[0014] On the other hand, the step S1 includes:

[0015] Step S11: acquiring the current image frame, and performing binarization processing on the current image frame to obtain a binarized image corresponding to the court sideline;

[0016] Step S12: performing a directed traversal on the edge lines in the binary image to determine an inflection point on the edge lines as the identification point;

[0017] Step S13: Count the identification points and go to step S2 when the number of the identification points is less than four.

[0018] On the other hand, in step S1, when the number of the identification points is greater than or equal to four, the process goes to step A2:

[0019] Step A2: Select four marking points from the marking points as target marking points, then construct the homography matrix according to the physical positions of the target marking points in the court and the pixel positions in the current image frame, and then turn to step S4.

[0020] On the other hand, the step S12 includes:

[0021] Step S121: Separating the edge region from the binary image;

[0022] Step S122: selecting an endpoint from the edge area as a traversal starting point, and constructing a measuring circle based on the traversal starting point;

[0023] Step S123: expanding the measurement circle so that the measurement circle covers the horizontal width of the traversal starting point;

[0024] Step S124: Move the measuring circle to traverse the edge area, and determine whether the inflection point exists according to the circumscribed pixels of the measuring circle during the traversal process, and then turn to step S13 after the traversal is completed.

[0025] On the other hand, the step S2 includes:

[0026] Step S21: obtaining straight edge points corresponding to the court sideline from the edge of the image of the binary image;

[0027] Step S22: sequentially obtaining multiple groups of adjacent points with the same color according to a preset step size for each straight line edge point;

[0028] Step S23: assembling the court sideline corresponding to the straight edge point according to the adjacent points, and extending the sideline toward both ends along the straight line to form a plurality of intersection points;

[0029] Step S24: removing the marking point from the intersection point to obtain the virtual marking point;

[0030] Step S25: Select the virtual identification points so that the total number of virtual identification points and identification points reaches four.

[0031] On the other hand, the step S25 includes:

[0032] Step S251: selecting a line with less than two marking points from the court sideline as an auxiliary point selection line;

[0033] Step S252: selecting two auxiliary marking points from the auxiliary point selection line;

[0034] Step S253: Calculate the virtual identification point based on the auxiliary identification point.

[0035] On the other hand, in step S253, the method for calculating the virtual identification point includes:

[0036]

[0037] In the formula, (x v ,y v ) is the virtual identification point, (x1, y1), (x2, y3) are the auxiliary identification points on the first auxiliary point selection line, (x3, y3), (x4, y4) are the auxiliary identification points on the second auxiliary point selection line, k1, k2, b1, b2 are intermediate variables.

[0038] On the other hand, after executing step S4, the method further includes:

[0039] Step S5: Calculating the player's movement speed according to the player's position in two adjacent frames.

[0040] A memory includes computer instructions. When a computer device runs the computer instructions, the above-mentioned method for calculating the position of players on the court is executed.

[0041] The above technical solution has the following advantages or beneficial effects:

[0042] To address the problem that image mapping solutions in existing technologies cannot effectively cope with the multi-angle and multi-focal length calculation requirements during broadcasting, this solution introduces additional virtual landmarks on the basis of the original court sidelines, thereby increasing the number of landmarks that can be used to construct the homography matrix in the current image frame. This avoids the problem of too few landmarks to calculate the homography matrix in real time, and facilitates the direct calculation of athlete positions based on the broadcast image during live broadcasting. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The embodiments of the present invention will be described more fully with reference to the accompanying drawings, which are provided for illustration and description only and are not intended to limit the scope of the present invention.

[0044] Figure 1 This is a schematic diagram of the marking points on the football field;

[0045] Figure 2 A schematic diagram of a method in an embodiment of the present invention;

[0046] Figure 3 Schematic diagram of virtual identification points in an embodiment of the present invention;

[0047] Figure 4This is a schematic diagram of step A2 in an embodiment of the present invention;

[0048] Figure 5 This is a schematic diagram of step S1 in an embodiment of the present invention;

[0049] Figure 6 This is a schematic diagram of step S12 in an embodiment of the present invention;

[0050] Figure 7 This is a schematic diagram of step S2 in an embodiment of the present invention;

[0051] Figure 8 This is a schematic diagram of step S25 in an embodiment of the present invention;

[0052] Figure 9 Schematic diagram of step S5 in an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0054] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0055] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0056] Based on the background technology, it can be seen that for a camera with a fixed viewing angle, it is easy to set up targets or calibration reference objects in the venue in advance and solve the homography matrix on the camera's viewing angle. However, when the broadcast viewing angle changes, the homography matrix constructed for a single viewing angle cannot solve the orientation of objects in the changing viewing angle during the live broadcast. At the same time, since the homography matrix requires setting up multiple reference objects during the calculation process, during the live broadcast process, it is often impossible to ensure that there will be reference objects in the broadcast viewing angle that can be used for real-time solution.

[0057] Let’s take the most widely used football field as an example.

[0058] The shape and size of standard football fields around the world are usually fixed. Due to international standards, the length, position and relative position of each line on the field are fixed. Figure 1As shown. Assuming that the upper left corner is the coordinate of the dot (that is, the point marked "1" in the figure), the horizontal line (that is, the upper edge of the field) is the x-axis, and the vertical line (that is, the bottom line of the left half of the field) is the y-axis, a coordinate system can be established. Since the length of a standard football field is 105 meters and the width is 68 meters, it is not difficult to find the coordinates of the four points marked 8, 20, 32, and 39 in the figure as (0, 68), (52.5, 34), (105, 0), and (105, 68). The units of these coordinates are all meters. Similarly, for any point on the field, as long as the distance from the point to the x-axis and y-axis is known, the coordinates of the person on the field relative to the upper left dot (0, 0) are also known. If there are two such points on the field, the straight-line distance between the two points can also be conveniently calculated using the "distance formula between two points".

[0059] Distance formula: len = sqrt((x1-x2)^2+(y1-y2)^2) (sqrt represents the square root)

[0060] However, in reality, this is not feasible, both practically and theoretically. At this location, the resulting image from this perspective is not very visually appealing, and even if the image is directly above the center of the court (i.e., point "20"), the distance from the baselines on both sides will still result in the data being accurate directly below and moving toward the baselines, with the measurement becoming increasingly distorted the closer you get to the baselines.

[0061] In actual filming, the camera (or camera, camera) will be located at a high place outside the field, and its position is usually fixed. When taking a photo of the field at this position, there is a one-to-one correspondence between the photo and the actual field. This is mathematically called "homography mapping". In terms of data, it is converted through the "homography matrix". The method is: Figure 1 In this way, 39 points (called "marking points") are marked on a standard football field. The coordinates of each point can also be obtained from the length of each line of the standard football field. Assuming that a camera can capture the entire field, the positions of the upper left, lower left, upper right, and lower right corners of the entire football field on the screen can be obtained from the image captured by the camera (in pixels). Then, these four positions are compared with Figure 1 In the example, the four coordinates generated by the markers 1, 8, 32, and 29 are passed to the getPerspectiveTransform function of OpenCV in order to obtain a "homography matrix". After that, any coordinate on the camera screen is multiplied by the "homography matrix" to convert it into Figure 1 The point corresponding to the position in the given coordinates. As mentioned earlier, this point is exactly where the person is on the football field. The coordinate units before and after the homography can be different. For example, all screen coordinates are in pixels, while all actual field coordinates are in meters.

[0062] In actual use, this method does not conform to the viewing habits of the public. The images that the audience likes to see are those in which the shooting angle (i.e. where to shoot) and the focus (i.e. distance) are constantly changing according to the needs of the game scene. This means that in most images, the audience cannot see all four corners of the court at the same time. People are more likely to see a certain part of the court. In order to solve this problem, all the "points" on the court are specially marked, such as Figure 1 As shown, a total of 39 identification points were selected. Strictly speaking, only 11, 20, and 29 are "isolated points", and the rest are all "intersection points" generated by the correlation of two lines. First, the coordinates of these 39 points in the above coordinate system on the standard football field are calculated through geometric relationships. Then, in the live video, the AI ​​algorithm is used to find any 4 "screen points" among these 39 points that are not on the same straight line. The coordinates of these 4 points (in pixels) are used as parameters, together with the coordinates of the points at the same position of these 4 points on the actual field (in meters) as parameters (the order of the two groups of 4 points must also correspond one to one), and passed to OpenCV, and the "homography matrix" can also be obtained. With the help of this matrix, after giving the position of a player on the screen (generally the coordinates of the middle of the player's two feet), the coordinate position of the player in the corresponding coordinate system of the standard football field can be calculated.

[0063] However, in practice, this method also has certain shortcomings. Figure 1 As shown, most of the points are concentrated near the goals on both sides. In the midfield, although there are as many as seven points (numbered 17 to 23), these points are either difficult for AI to identify (such as numbers 17, 20, and 23) or are all on the same straight line (such as numbers 18, 19, 21, and 22). Furthermore, this area is far from the penalty area, making it difficult to learn from each other, resulting in a large area between the left and right penalty areas. Even near the penalty area on either side, it is not foolproof. For example, the current image only has three points, or although there are four points, one "point" is artificially blocked by an athlete. Therefore, although the AI ​​algorithm recognizes the person's position, it cannot accurately identify the location of the point. The above algorithms all fail due to the inability to find "four points that are not on the same straight line."

[0064] To solve the above problems, this solution provides a method for measuring the position of athletes on the court based on virtual marking points. Figure 2 、 Figure 3 Shown, including:

[0065] Step S1: Acquire the current image frame, identify the identification points in the current image frame, and turn to step S2 when the number of identification points is less than four;

[0066] Step S2: Separate the court sidelines from the current image frame, extend each court sideline to both ends and intersect with other court sidelines to construct virtual marker points, so that the total number of virtual marker points and marker points reaches four;

[0067] Step S3: determining the physical positions of the virtual marker point and the marker point in the court and the pixel positions in the current image frame respectively, and then constructing a homography matrix corresponding to the current image frame according to the physical positions and the pixel positions;

[0068] Step S4: Obtain the pixel position of the player from the current image frame, and calculate the actual position of the player according to the homography matrix.

[0069] Specifically, to address the problem that image mapping solutions in the existing technology cannot effectively meet the calculation requirements of multiple angles and multiple focal lengths during the broadcast process, this solution introduces additional virtual landmarks on the basis of the original court sidelines, thereby increasing the number of landmarks that can be used to construct the homography matrix in the current image frame, avoiding the problem of too few landmarks to calculate the homography matrix in real time, and facilitating the direct calculation of athlete positions based on the broadcast image during live broadcasts.

[0070] During implementation, the above-mentioned method for calculating the position of players on the field is mainly configured as a software embodiment in a computer device, such as a director's workstation of a sports event, to process the real-time captured broadcast images and calculate the position of specific players.

[0071] The broadcast images are collected by multiple channel camera equipment set up in the stadium and transmitted to the director's desk via cables or other equivalent equipment. The director's desk monitors the images in real time and forwards them to the recording equipment for recording and storage. It also selects the images from specific camera positions and forwards them to the broadcast equipment for broadcast.

[0072] Based on the above-mentioned device, the above-mentioned method can be implemented by configuring a specific computer program in the computer equipment on the director's station side.

[0073] Specifically, when the director's station receives the broadcast footage, it is composed of multiple consecutive image frames. For each camera position, as the game progresses, including the movement of the ball and the running of the players, the shooting angle of the camera will rotate, and the focal length of the lens will be switched as needed, for example, from a larger, wide-angle focal length that reflects the entire scene to a telephoto focal length that focuses on a specific player. As the shooting parameters switch, the image information contained in the current single frame, especially the information of the court sidelines, will also change accordingly, making some sidelines and their intersections impossible to capture directly in the image.

[0074] For each current image frame, it is considered as an isolated image during calculation, that is, assuming that the viewing angle and focal length have changed, the real-time homography matrix needs to be recalculated.

[0075] When a current image frame is acquired, a computer program is first used to identify the marking points in the current image frame. The marking points refer to the inflection points that can be directly observed on the court. Figure 1 Taking the football field shown in as an example, such identification points are inflection points and other obvious feature points that can be directly observed based on the intersection lines, such as the two end points of the penalty area, the intersection of the penalty area and the bottom edge, the two end points of the goal line, the two side vertices of the center circle, etc.

[0076] Then, the number of identification points is counted. When the number of identification points is less than four, it is considered that the current number of identification points is insufficient to construct a real-time homography matrix, and the process goes to step S2 to construct virtual identification points.

[0077] Virtual markers are points that cannot be directly observed on the actual court but can be determined based on the geometric relationship of the court's sidelines.

[0078] by Figure 3 Taking the virtual marking points constructed based on the football field as an example, if the penalty area line is extended to both ends, it is easy to find two pairs of virtual marking points 41 and 51, 42 and 52 on the side lines of the field, and there will also be new intersection points 45 and 48 at the center line.

[0079] Based on this method, the goal line is extended, and finally Figure 1 Based on the 39 directly observable endpoints of the football field shown, 14 virtual identification points were expanded, and finally 53 observation points that can be used to construct the homography matrix were obtained, which greatly improved the success rate of selecting 4 points in a single current image frame to construct the homography matrix.

[0080] It should be noted that for ease of description, only a football field is used as an example in this application. However, in other sports, such as tennis and other stadium sports that also have sidelines, the above method can also be used to add new virtual identification points and construct a new homography matrix in real time to measure the athlete's position.

[0081] The aforementioned markers and virtual markers must be pre-calibrated in real life to construct a physical coordinate system corresponding to the course. The physical locations of each marker and virtual marker are then recorded. Virtual markers can be easily extended using tools to make them appear on the actual course. The tools can be removed after measurement.

[0082] To simplify the calculation, for a standard stadium, the physical position of each actual identification point in the physical coordinate system can be directly determined according to the relevant standards of the stadium, and the corresponding physical position of the virtual identification point in the physical coordinate system can be calculated through geometric operations.

[0083] By extending each court sideline to both ends and intersecting it with other court sidelines to construct virtual identification points, it is easy to determine the virtual identification points that have been identified and the identification points that have been identified before in the current image frame.

[0084] For the identified virtual landmarks and landmarks, it is easy to extract the corresponding pixel positions in the current image frame. Subsequently, since the physical positions corresponding to each point are already determined, a homography matrix can be constructed based on the pixel positions and physical positions.

[0085] Based on the constructed homography matrix, any pixel in the current image frame can be mapped to a physical location in the physical coordinate system. This, combined with pre-trained AI models such as pose estimation models, makes it easy to annotate the athlete's position in the image. Based on the athlete's footfall, the homography matrix is ​​used to map it to the physical coordinate system, thereby reading the athlete's actual position.

[0086] To make some additional explanations on the above algorithm, in the calculation of "4", any point on the image is not limited to the football field in the image. If the AI ​​algorithm recognizes that someone is outside the field (spectators, referees, coaches, or an athlete serving outside the field), it does not affect the correctness of the algorithm. It's just that the position of the person found is obviously somewhere outside the standard football field. In addition, there are many points that can participate in this algorithm. For example, there is a quarter-circle arc with a radius of 1 meter on the inside of each of the four corner kicks, and the intersection with the edge can also participate in the calculation. The line connecting the midfield serving point and the penalty kick point can also assist in finding the midpoint coordinates of the "midfield semicircle square arc" and the "large penalty area top arc". It's just that these points are more difficult for AI to identify, so in Figure 3 It is not listed in detail and is not used in the algorithm, but it does not contradict the algorithm itself. In addition, the "homography matrix" in the algorithm requires two sets of data, a total of 8 data, when it is generated. The first set of data represents the site identification in the current image. In addition to the coordinates, the value also needs to give the coordinates corresponding to the Figure 1The second set of data represents the coordinates of the four numbers in the first set, labeled 1-39, corresponding to the labels 1-39 in the figure, on an actual standard football field. The first set of data changes with each frame, while the second set of data does not change after a one-time calculation. Therefore, the "homography matrix" is always changing, and the "homography matrix" corresponding to each frame must be calculated. If the camera is fixed and stationary as described above, then the two sets of eight data remain unchanged. Therefore, only one "homography matrix" is used for the entire process, which can be used as a special case of the present invention.

[0087] In one embodiment, Figure 4 As shown, in step S1, when the number of identification points is greater than or equal to four, go to step A2:

[0088] Step A2: Select four marker points from the marker points as target marker points, then construct a homography matrix based on the physical positions of the target marker points in the court and the pixel positions in the current image frame, and then go to step S4.

[0089] Specifically, in order to realize the measurement of the athlete's position based on the real-time broadcast screen, in this embodiment, image processing is first performed based on the current image frame, and the identification points visible in the current image frame are marked according to a pre-constructed algorithm, and then statistics are performed. When it is determined that the number of identification points is greater than or equal to four, four identification points are selected from the identification points as target identification points. Then, a homography matrix is ​​constructed according to the physical position of the target identification point in the court and the pixel position in the current image frame, and the athlete's position is calculated based on the homography matrix.

[0090] In one embodiment, Figure 5 As shown, step S1 includes:

[0091] Step S11: obtaining a current image frame, and performing a binarization process on the current image frame to obtain a binary image corresponding to the sideline of the court;

[0092] Step S12: performing a directed traversal on the edge lines in the binary image to determine the inflection points on the edge lines as identification points;

[0093] Step S13: Count the identification points and go to step S2 when the number of identification points is less than four.

[0094] Specifically, in order to achieve effective extraction of the marking points on the sidelines of the court, the above-mentioned processing method is constructed in this embodiment.

[0095] First, for the current image frame, since its background part is usually the court ground photographed at a certain angle, the court sidelines are usually fixed in white or other colors with high contrast relative to the ground. Therefore, before starting processing, the color of the court sidelines can be calibrated and the corresponding binarization threshold can be constructed. In actual processing, processing is performed based on the binarization threshold, only the part corresponding to the sidelines is retained and irrelevant background of other colors is removed to obtain a binary image corresponding to the court sidelines.

[0096] The binary image should only contain a number of court sidelines, which should intersect with each other. A directed traversal of the sidelines in the binary image is performed using a correlation algorithm to construct a set of vectors corresponding to the pixels. The inflection points on the sidelines are then identified based on the changes in the vector directions.

[0097] Finally, the identification points are counted, and when the number of identification points is less than four, the process goes to step S2; if the number is greater than or equal to four, a sufficient number of identification points are selected according to other processes to construct the homography matrix.

[0098] In one embodiment, Figure 6 As shown, step S12 includes:

[0099] Step S121: Separate the edge region from the binary image;

[0100] Step S122: selecting an endpoint from the edge area as a traversal starting point, and constructing a measuring circle based on the traversal starting point;

[0101] Step S123: expanding the measurement circle so that the measurement circle covers the horizontal width of the traversal starting point;

[0102] Step S124: Move the measuring circle to traverse the edge area, and determine whether there is an inflection point based on the circumscribed pixels of the measuring circle during the traversal process, and then go to step S13 after the traversal is completed.

[0103] Specifically, in order to achieve a better traversal effect on the binary image, in this embodiment, firstly, based on the construction of the binary image, further detection is performed on it to remove isolated noise parts, thereby separating the edge area.

[0104] Then, an endpoint is selected in the edge area as a traversal starting point. The endpoint is usually caused by frame cropping during the imaging process, so the endpoint appears at the edge of the image.

[0105] Then, a measuring circle is constructed based on the traversal starting point. The starting center of the measuring circle is the midpoint of multiple pixel lengths corresponding to the endpoint part, and the diameter is the length of the pixels adjacent to the midpoint. The diameter is scaled as the traversal process proceeds.

[0106] The measuring circle is then moved to traverse the border area. During the traversal, the presence of an inflection point is determined based on the circumscribed pixels of the measuring circle. This process typically involves moving the center of the measuring circle in the direction of the directed traversal and adjusting the diameter of the measuring circle until it coincides with the edge pixels of the border area. At this point, the pixel position of the circumscribed pixels is further determined. If they are not in the forward or backward direction, an inflection point is considered to have occurred. This point is marked as a marker, and a new measuring circle is generated and moved in that direction until the traversal process is complete.

[0107] In one embodiment, Figure 7 As shown, step S2 includes:

[0108] Step S21: obtaining straight edge points corresponding to the court sideline from the edge of the image of the binary image;

[0109] Step S22: sequentially obtaining multiple groups of adjacent points with the same color according to a preset step size for each straight line edge point;

[0110] Step S23: Assembling court sidelines corresponding to the edge points of the straight line according to the adjacent points, and extending the sidelines toward both ends along the straight line to form a plurality of intersection points;

[0111] Step S24: removing the marker points from the intersection points to obtain virtual marker points;

[0112] Step S25: Select virtual identification points so that the total number of virtual identification points and identification points reaches four.

[0113] Specifically, to achieve a better virtual marker point selection process, this embodiment first obtains linear edge points corresponding to the court sideline from the edge of the image based on the binary image. These linear edge points are a series of points along the sideline at the edge of the image. Subsequently, these points are moved along the sideline area toward the center by a preset step size, such as 3 pixels, to obtain the next series of adjacent points. This step is repeated to obtain multiple groups of adjacent points with the same color.

[0114] By fitting the adjacent points, we can construct the court sideline corresponding to the straight edge point. The sideline has a certain width in the image, and the width will change slightly with the perspective relationship.

[0115] The extracted court sidelines are further fitted to extend along the straight line to both ends to form multiple intersection points and record them. Then, the marking points are removed from the intersection points to obtain virtual marking points.

[0116] In addition, other AI algorithms can be used to identify the vertices of circular edges such as the vertices of the penalty area and the end points on both sides of the kick-off area.

[0117] In one embodiment, Figure 8 As shown, step S25 includes:

[0118] Step S251: selecting a line with less than two marking points from the court sideline as an auxiliary point selection line;

[0119] Step S252: Select two auxiliary marking points on the auxiliary point selection line;

[0120] Step S253: Calculate the virtual marking point based on the auxiliary marking point.

[0121] Specifically, in order to achieve a better calculation process of the coordinates of the virtual marking points in the image, in this embodiment, first, a line with less than two marking points is selected from the sidelines of the court as an auxiliary point selection line, and the number of auxiliary point selection lines should be two.

[0122] In addition, the auxiliary point selection line should also be checked to remove the penalty area arc and kick-off circle, which are two line segments that cannot be used to fit the straight line. This process can be based on the previous identification point recognition process to determine whether the arc vertex appears on the edge line.

[0123] For each auxiliary point selection line, two auxiliary marking points are selected respectively, and then the auxiliary marking points are calculated according to the straight line formula to fit the straight line corresponding to the pair of auxiliary marking points.

[0124] Subsequently, the intersection of the two fitted straight lines is calculated to obtain a virtual identification point.

[0125] In one embodiment, in step S253, the method for calculating the virtual identification point includes:

[0126]

[0127] In the formula, (x v ,y v ) is a virtual identification point, (x1, y1), (x2, y3) are auxiliary identification points on the first auxiliary point selection line, (x3, y3), (x4, y4) are auxiliary identification points on the second auxiliary point selection line, k1, k2, b1, b2 are intermediate variables.

[0128] Specifically, in order to achieve a better calculation of the position of the virtual identification point in the current image frame, in this embodiment, first, for the fitted court sideline, only one identification point or a court sideline with no identification point is selected, and for the selected first auxiliary point selection line and second auxiliary point selection line, two non-overlapping points are randomly selected as auxiliary identification points.

[0129] Then, the two straight lines are fitted according to the above formula, and finally the intersection of the straight lines is found to obtain the virtual identification point.

[0130] In one embodiment, Figure 9 As shown, after executing step S4, the method further includes:

[0131] Step S5: Calculate the player's movement speed according to the player's position in two adjacent frames.

[0132] Specifically, based on the homography matrix, the coordinates of any point in the current frame image can be quickly converted to the position coordinates of the point on the actual standard football field. If you want to measure the distance, you only need to indicate two coordinate points on the image, and then perform two "homography conversions" on these two points to find the distance between the two points. If you need to know the speed of a person's movement, you only need to convert the position of the same person between different frames, find the position of the person in the two frames, and then find the distance. Then, with the help of the fixed time interval between any two adjacent frames of the video screen, you can calculate the time interval between the two frames (the frame number of the next frame - the frame number of the previous frame) / the time interval between two adjacent frames. For general videos, the time interval between two adjacent frames is 0.04 seconds, so the formula is:

[0133] speed1 = (next frame number - previous frame number) / 0.04 (unit: meters per second) speed2 = speed1 / 3.6 (unit: kilometers per hour)

[0134] speed3=100 / speed1(times per 100 meters)

[0135] A memory includes computer instructions. When a computer device runs the computer instructions, the above-mentioned method for calculating the position of players on the court is executed.

[0136] Those skilled in the art will appreciate that various aspects of the present invention, or possible implementations of various aspects, may be embodied as systems, methods, or computer program products. Thus, various aspects of the present invention, or possible implementations of various aspects, may take the form of complete hardware embodiments, complete software embodiments (including firmware, resident software, etc.), or embodiments combining software and hardware aspects, all collectively referred to herein as "circuits," "modules," or "systems." Furthermore, various aspects of the present invention, or possible implementations of various aspects, may take the form of computer program products, which refer to computer instructions stored in a memory.

[0137] The memory may be a computer-readable signal medium or a computer-readable storage medium. Computer-readable storage media include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or apparatuses, or any suitable combination thereof, such as random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, and portable read-only memory (CD-ROM).

[0138] The processor in the computer reads the computer instructions stored in the memory, so that the processor can perform the functional actions specified in each step or the combination of steps in the flowchart; and generate a device that implements the functional actions specified in each block or the combination of blocks in the block diagram.

[0139] It should be understood that the processor in the computer can be understood as one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components implemented to execute the aforementioned computer instructions.

[0140] Computer instructions can be executed entirely on the user's local computer, partially on the user's local computer, as a separate software package, partially on the user's local computer and partially on a remote computer, or entirely on a remote computer or server. It should also be noted that in certain alternative embodiments, the functions noted in each step of the flow chart or each block in the block diagram may not occur in the order noted in the figure. For example, depending on the functions involved, two steps or two blocks shown in succession may actually be executed approximately simultaneously, or the blocks may sometimes be executed in reverse order.

[0141] In practice, the various components of a computer system are coupled together via a bus system. It is understood that the bus system is used to enable communication between these components. In addition to the data bus, the bus system also includes a power bus, a control bus, and a status signal bus.

[0142] Although the present application includes many specific implementation details, these should not be interpreted as limiting the scope of any disclosure or the scope of protection claimed, but are mainly used to describe the features of the specific disclosed embodiments. Certain features described in multiple embodiments of the present application can also be implemented in combination in a single embodiment. On the other hand, the various features described in a single embodiment can also be implemented separately in multiple embodiments or implemented in any suitable sub-combination. In addition, although features can work in some combinations as described and even initially claim protection, one or more features from the claimed combination can be removed from the combination in some cases, and the claimed combination can point to a variation of a sub-combination or a sub-combination.

[0143] Similarly, although configurations are depicted in a particular order in the accompanying drawings, this should not be understood as requiring that these configurations be performed in the particular order shown or sequentially, or that all illustrated configurations be performed to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product, or packaged into multiple software products.

[0144] The above are only preferred embodiments of the present invention and do not limit the implementation mode and protection scope of the present invention. For those skilled in the art, it should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the description and illustrations of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for calculating the position of players on a court based on virtual markers, characterized in that: include: Step S1: Acquire a current image frame, identify the identification points in the current image frame, and turn to step S2 when the number of the identification points is less than four; Step S2: Separating court sidelines from the current image frame, extending each court sideline to both ends and intersecting with other court sidelines to construct virtual marker points, so that the total number of virtual marker points and marker points reaches four; Step S3: respectively determining the physical positions of the virtual marking point and the marking point in the court and the pixel positions in the current image frame, and then constructing a homography matrix corresponding to the current image frame according to the physical positions and the pixel positions; Step S4: Obtain the pixel position of the player from the current image frame, and calculate the actual position of the player according to the homography matrix.

2. The method for calculating the position of players on the court according to claim 1, wherein: The step S1 comprises: Step S11: acquiring the current image frame, and performing binarization processing on the current image frame to obtain a binarized image corresponding to the court sideline; Step S12: performing a directed traversal on the edge lines in the binary image to determine an inflection point on the edge lines as the identification point; Step S13: Count the identification points and go to step S2 when the number of the identification points is less than four.

3. The method for calculating the position of players on the court according to claim 1, wherein: In step S1, when the number of identification points is greater than or equal to four, proceed to step A2: Step A2: Select four marking points from the marking points as target marking points, then construct the homography matrix according to the physical positions of the target marking points in the court and the pixel positions in the current image frame, and then turn to step S4.

4. The method for calculating the position of players on the court according to claim 2, wherein: The step S12 includes: Step S121: Separating the edge region from the binary image; Step S122: selecting an endpoint from the edge area as a traversal starting point, and constructing a measuring circle based on the traversal starting point; Step S123: expanding the measurement circle so that the measurement circle covers the horizontal width of the traversal starting point; Step S124: Move the measuring circle to traverse the edge area, and determine whether the inflection point exists according to the circumscribed pixels of the measuring circle during the traversal process, and then turn to step S13 after the traversal is completed.

5. The method for calculating the position of players on the court according to claim 2, wherein: The step S2 comprises: Step S21: obtaining straight edge points corresponding to the court sideline from the edge of the image of the binary image; Step S22: sequentially obtaining multiple groups of adjacent points with the same color according to a preset step size for each straight line edge point; Step S23: assembling the court sideline corresponding to the straight edge point according to the adjacent points, and extending the sideline toward both ends along the straight line to form a plurality of intersection points; Step S24: removing the marking point from the intersection point to obtain the virtual marking point; Step S25: Select the virtual identification points so that the total number of virtual identification points and identification points reaches four.

6. The method for calculating the position of players on the court according to claim 5, characterized in that: The step S25 includes: Step S251: selecting a line with less than two marking points from the court sideline as an auxiliary point selection line; Step S252: selecting two auxiliary marking points from the auxiliary point selection line; Step S253: Calculate the virtual identification point based on the auxiliary identification point.

7. The method for calculating the position of players on the court according to claim 6, characterized in that: In step S253, the method for calculating the virtual identification point includes: In the formula, (x v ,y v ) is the virtual identification point, (x1, y1), (x2, y3) are the auxiliary identification points on the first auxiliary point selection line, (x3, y3), (x4, y4) are the auxiliary identification points on the second auxiliary point selection line, k1, k2, b1, b2 are intermediate variables.

8. The method for calculating the position of players on the court according to claim 1, wherein: After executing step S4, the method further includes: Step S5: Calculating the player's movement speed according to the player's position in two adjacent frames.

9. A memory comprising computer instructions, characterized in that When the computer device runs the computer instructions, the method for calculating the position of players on the court as described in any one of claims 1 to 8 is executed.

Images