Method and system for generating match condition top view based on multi-camera information fusion

Abstract

The invention discloses a method for generating a match condition top view based on multi-camera information fusion, which comprises the following steps of: deploying cameras which can cover the whole stadium and form an overlapped region at four corners of the stadium, separating and extracting straight lines and intersection points in an image by utilizing an HSV color space, and calculating and constructing a rectangular feature point set of the stadium; based on geometric features and a coordinate transformation matrix, mapping a local view angle into a unified overlook coordinate system, screening out an optimally matched rectangular coordinate through a rectangular matching score, and calculating the coordinate difference of the athletes in the overlapping region transformed into the overlook from different view angles; weight distribution is carried out on the rectangular matching error and the coordinate error of the athlete through a CRITIC weight method, the objective function is solved, and an optimal dimension transformation matrix is determined. According to the invention, real-time top view generation of the sports event can be realized.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and image processing, and specifically relates to a method and system for generating a bird's-eye view of a game situation based on multi-camera information fusion. Background Art

[0002] When analyzing sports events tactically, it is usually more intuitive to look down at the game. However, most current cameras are set up around the venue, making it difficult to shoot from a bird's-eye view. Therefore, image processing technology is needed to convert images from other perspectives into a bird's-eye view.

[0003] At present, in the field of computer vision and intelligent perception, the detection and recognition of scene targets and the generation of bird's-eye views are mainly achieved by using the target detection algorithm under monocular vision with BEV projection transformation technology, or by using the perspective transformation method based on OpenCV. The technologies applied include multi-object recognition and positioning target detection based on deep learning, feature extraction and classification methods based on traditional machine learning, target detection based on instance segmentation, and multi-scale target detection for objects of different sizes. Based on the above technologies, it is currently possible to identify multiple objects in the video and give the category identification and screen position data of each object. For example, a method for real-time generation of a bird's-eye view of a sports event with a dynamic perspective is disclosed in the publication number CN118154728A. This method uses a single camera to perform real-time image capture and coordinate initialization, image preprocessing and key point extraction, initial bird's-eye view generation and region division, rectangle screening and error optimization, rematching recognition and real-time bird's-eye view generation.

[0004] However, due to factors such as camera viewing angle, shooting distance and clarity, it is difficult to conduct in-depth data analysis based solely on the position data of a single-frame image from a single camera. When conducting a time series analysis of an event, a large amount of manpower is often required to manually acquire data. Taking sports events as an example, it is difficult to fully grasp the overall situation of the stadium by processing the single-frame image recognition of a single camera with a fixed position. For example, a real-time generation method for a bird's-eye view of a sports event with a dynamic perspective is disclosed in publication number CN118154728A. It only uses one camera, has a limited viewing angle, and lacks accuracy when covering the entire standard stadium. It is also limited by optical distortion and resolution, and it is difficult to meet the requirements of real-time tactical analysis for target positioning accuracy and panoramic situation awareness. Especially under complex working conditions such as high-speed moving target tracking and occlusion scene analysis, the data completeness and reliability of the single-perspective system face severe challenges. Summary of the invention

[0005] The present invention aims to solve one of the technical problems existing in the related art at least to a certain extent.

[0006] One object of the present invention is to provide a method for generating a bird's-eye view of a game situation based on multi-camera information fusion. The method realizes global target positioning in a complete stadium by fusing local information of multiple cameras, converts image dimensions frame by frame, and thus generates a complete game situation in real time, so as to assist coaches and spectators to better grasp the status of sports events.

[0007] Another object of the present invention is to provide a system for generating a bird's-eye view of the game situation based on multi-camera information fusion for implementing the above method.

[0008] In order to achieve the above-mentioned purpose, the present invention provides, on one hand, a method for generating a bird's-eye view of a game situation based on multi-camera information fusion, comprising: S100, placing cameras at the four corners of the stadium, recording the stadium in real time, and capturing the current frame image in the video; using one of the corners of the stadium as the coordinate origin, establishing a plane rectangular coordinate system, and recording the relative positions of the four cameras and the height and width of the stadium; S200, perform HSV color space separation on the current frame image of each camera, set a value range to identify all straight lines and intersections on the image, randomly select a corner point on the image as a reference point, calculate and determine the intersection closest to the reference point, and then find other intersections related to the straight line where the intersection is located by analyzing the straight line, and gradually determine the coordinates of the four vertices of the rectangle through these intersections and store them in the point set in order. ;exist Always create a top view legend for the entire scene, and store all intersection points in the top view into a point set after transforming them through the transformation matrix In the example, the site is divided into Rectangles, record the coordinates of all rectangles in order from left to right and from small to large, and store them as a rectangular point set based on the top view ; S300, according to the four vertices of the rectangle on a certain frame image obtained in step S200, find and screen the corresponding matrix coordinate points in the top view, restore all the coordinates in the frame image to the top view by the coordinate restoration method, and then match the target rectangle and the selected rectangle through the rectangle matching and error analysis algorithm to accurately screen the rectangle; S400, introduce the athlete coordinate transformation error calculation method based on multi-view camera fusion, obtain the athlete coordinates in the overlapping area between cameras, and use the matrix transformation to the top view, calculate the Euclidean distance of the same athlete in the top view, and obtain the athlete coordinate transformation error; S500, superimpose the errors of the four-dimensional transformation matrices obtained in the non-overlapping areas of the four cameras to form an error point set ; Construct the error point set based on the coordinate transformation error of the athletes in the overlapping area of ​​the four cameras ; Set coefficients for the two error point sets respectively, take the weighted sum minimum as the objective function, and obtain the optimal dimension transformation matrix; S600, after the current sequence ends, switch to the next sequence, and perform dimension transformation on the entire motion field according to the optimal dimension transformation matrix obtained in step 500, and save all points of the required field information after the transformation in the point set In the step S100, the top view is generated by the information stored in the point set, and the other point sets are cleared, the frame image of the next time sequence in the video is intercepted, and steps S100 to S500 are repeated to continuously obtain the top view. A further preferred technical solution of the present invention is that in step S100, the lower left corner of the stadium is used as the coordinate origin to establish Plane rectangular coordinate system, recording the relative positions of the four cameras , , , The height of the stadium is and width is .

[0009] Preferably, in step S200, the current frame image of each camera is separated in HSV color space, a value range is set to identify all straight lines and intersections on the image, a corner point on the image is randomly selected as a reference point, and the intersection closest to the reference point is calculated and determined, and then the straight line where the intersection point is located is analyzed to find other intersection points related to it. Through these intersection points, the coordinates of the four vertices of the rectangle are gradually determined and stored in the point set in order. ; The specific method is: S210, convert the current frame image from the RGB three-color channel to the HSV color space, take the lower left corner of the current frame image as the origin, and establish Plane rectangular coordinate system, set the appropriate value range, and identify all straight line equations , calculate the intersection points of all straight lines and record them; S220, randomly select one of the four vertices of the current frame image as the reference point, and assume that the length of the image is , the height is , will be with the point The distance is the minimum The intersection of ,point It is produced by the intersection of two straight lines. Point coordinates, get the The analytical expression of the two straight lines of the point is recorded as and ; S230, With The intersection point is obtained by combining the equations of the identified lines outside , , … , similarly, With The identified straight lines outside are combined to obtain the intersection point , , … ;calculate and and Distance of points and , respectively select and The corresponding points are denoted as and , the other straight line where the two points lie is and ; S240, straight line With straight line The intersection point is obtained by combining the analytical expressions of , and then the four points obtained As vertex coordinates are stored in order in the point set middle; S250, repeat steps S210 to S240, traverse according to the difference in the distance between the two points, find the point set of the rectangle on the original image, until no new distance difference appears, then all the rectangular point sets on the original image are found, and stored in the point set in sequence As a preferred embodiment, in step S200, Always create a top view legend for the entire scene, and store all intersection points in the top view into a point set after transforming them through the transformation matrix In the example, the site is divided into Rectangles, record the coordinates of all rectangles in order from left to right and from small to large, and store them as a rectangular point set based on the top view ; The specific method is: S260, when the timing is When the image of the entire playing field or the dimensions of the field is obtained, a top-view coordinate system with the vertices of the playing field as the coordinate origin is created. , get the coordinates of all intersection points on the site , according to the coordinates of the four vertices of the site, the coordinates on the matching coordinate system Calculate a three-row and three-column transformation matrix, and multiply all points on the field by this transformation matrix to get the new intersection coordinates , so as to obtain the top view F of the site and record the time series Afterwards, directly obtain the top view F of the initial state; S270, obtained in step S260 By taking any two points whose horizontal and vertical coordinates are different, we can determine the four points of a rectangle and store them in an array in clockwise order. In the example, all the points are connected to generate n horizontal lines and m vertical lines, which divide the entire top view site into several small rectangles and large rectangles composed of small rectangles. Rectangle , record the coordinates of all rectangles in order from left to right and from small to large, using eight lines The obtained rectangular point set is stored in an array of columns. .

[0010] Preferably, in step S300, according to the four vertices of the rectangle on a frame image obtained in step S200, the corresponding matrix coordinate points in the top view are found and screened, all the coordinates in the frame image are restored to the top view by a coordinate restoration method, and then the target rectangle and the selected rectangle are matched by a rectangle matching and error analysis algorithm to accurately screen the rectangle, including: S310, based on the four vertices of the rectangle on a certain frame image obtained in step S200, a rectangle matching scoring algorithm model is introduced to preliminarily identify and select suitable rectangles; S320, adopting a loop method, that is, selecting the point set of the coordinates obtained when the target rectangle is selected Perform shift processing, and then repeat step S310 to obtain all optimal solutions; S330: After obtaining the most suitable rectangle pair, a dimensional transformation matrix is ​​calculated.

[0011] Preferably, in step S310, based on the four vertices of the rectangle on a certain frame image obtained in step S200, a rectangle matching scoring algorithm model is introduced to preliminarily identify and select suitable rectangles; the specific method is: S311, based on the point set obtained in step S200 and Get a new top view F1, first compare the boundary lengths of the top view F and F1. If a boundary length of F1 is greater than the corresponding boundary length in F, it means that the currently selected rectangle point is deviated or inappropriate. Then compare the rectangles in the figure. If the length of a side of the rectangle in F1 is obviously greater than the length of the corresponding side of the rectangle in F, it can be determined that the current target rectangle is not compatible with the selected rectangle in size and shape. Remove this part of the rectangle and store the remaining rectangle coordinate points in a new point set. middle; S312, based on point set , Calculate the length Length, width Width and the width of each rectangle under two viewing angles. , and obtain the centroid coordinates of each set of rectangles at the same time, and define the aspect ratio as , the angle difference is , the centroid distance is , calculate the Hausdorff distance of the point set of the rectangular coordinates under two perspectives, expressed as: According to the top view creation method in step S200, the top view F2 restored under the camera's viewing angle is obtained, thereby obtaining two straight lines representing the viewing angle in the top view F2, and the two straight lines are extended in the opposite direction and intersected to obtain the coordinates of an imaginary camera. , by comparing with the camera coordinates in step S100 Compare and get a Euclidean distance difference ; Then normalize the data and calculate the normalized similarity score :

[0012] in =0.12, =0.15, =0.13, =0.10, =0.5, traverse and All rectangular coordinate groups in , calculate the similarity scores of all matching combinations , stored in point sets , In the example, compare the scores, get the matching rectangles, and update them in a one-to-one order. and .

[0013] S320: It should be noted that when comparing the coordinates of the selected target rectangle with the selected rectangle, it cannot be guaranteed that the order of the coordinate points in their corresponding point sets is consistent. Therefore, a loop method is used, that is, the point set of the coordinates obtained when selecting the target rectangle is Perform the shift processing, and then repeat step S310. Since a rectangle can select four coordinate points, it only needs to be cycled four times to obtain all the optimal solutions; S330, after obtaining the most suitable rectangle pair, compare the point set and The coordinate points in can be calculated to obtain a dimensional transformation matrix.

[0014] S400, transforming the athlete captured by the camera to the stadium under the bird's-eye view, calculating the coordinate difference of the athlete after the transformation at different perspectives as the error basis of the dimensional transformation matrix, the specific method is: S410, selecting two viewing angles of camera A and camera B, using YOLO to identify all athletes in the viewing angle of the camera and taking the coordinates (X, Y) of their center points as the positions of the athletes; S420, calculating the coordinates of the athletes in the overlapping area after dimension transformation at each viewing angle, and storing them in a point set. The specific implementation steps are as follows: S421, according to the dimension change matrix calculated in step S300, the position of the athlete under each camera's view is dimensionally transformed to obtain the coordinates (X', Y') after the dimension change, and the coordinates of all athletes photographed by the i-th camera after the dimension change are stored in the point set Si; S422, calculating the coordinate range of the overlapping area between cameras, selecting the coordinates that meet the range in the point set and storing them in a new point set S'i; S430, take point set S'1 and point set S'2, calculate the Euclidean distance between all point pairs in the two perspectives as the matching cost, construct a cost matrix and apply the Hungarian algorithm to match the point pairs; the specific implementation steps are: S431, define the detection point set of camera A as , the detection point set of camera B is , calculate the Euclidean distance between all point pairs in the two perspectives as the matching cost, and construct the cost matrix as , whose elements are S432, solving the row index and column index set , so that , get the ordered matching index list .

[0015] S440, for each matching row and column index (i, j), the point corresponding to camera A , and the point of camera B , reorder the point set according to the matching results to ensure that the coordinates of the corresponding positions belong to the same player, and for each matching point pair ( , ) to obtain the coordinate error ; Step 450: Repeat the process from step S410 to step S440 for viewing angles B and C, viewing angles C and D, and viewing angles D and A, and superimpose the error values ​​of the overlapping areas to obtain a final error value. .

[0016] S500, introducing a method for determining the optimal dimensional transformation matrix based on the minimum total error: in a sports event venue, the shooting ranges of the cameras at the four corners of the venue overlap with each other, forming a globally covered monitoring area; there are four dimensional transformation matrices obtained from the non-overlapping areas of the four cameras, and the error superposition of the rectangles of the four dimensional transformation matrices is defined to form an error point set. and the set of coordinate transformation errors of the players in the overlapping area of ​​each camera , respectively, the two errors are matched with appropriate coefficients, and the weighted sum is minimized as the objective function to obtain the optimal dimension transformation matrix. The result of the top-view perspective conversion after the optimal dimension transformation matrix is ​​determined in step S500 is shown in the figure. The specific method of step S500 is: S510, obtain a four-dimensional transformation matrix from a rectangle obtained by preliminary screening of a single camera, and superimpose the errors of the four-dimensional transformation matrix rectangles to form an error point set , , Indicates that from The error point set of the dimensional transformation matrix of the four cameras; the error value of the coordinate transformation of the athlete in the common area captured by the four cameras is recorded as ; After unifying the scale and standardizing the two error sets and , through the CRITIC weight method and Assign appropriate weights to each and , by weighted summing and solving the total minimum error to determine the optimal dimension transformation matrix; the specific method is as follows: S511, due to the different sources and scales of the two error sets, record , , let the target length be ,in , They are and The number of original elements of the error point set and error and Adjust to the same length , used for subsequent alignment of quantiles, using linear interpolation to extend the length of the shorter set, assuming that the shorter set is extended get ; Min-Max normalization method is used to and Perform standardization and calculate:

[0017]

[0018] get and ; S512, error point set and error and Assign appropriate weights to each of them and apply the CRITIC weight method to normalize the and Defined as two indicators, construct the matrix:

[0019] Then calculate the standard deviation, The standard deviation of is:

[0020] The standard deviation of is:

[0021] Compute the Pearson correlation coefficient between two columns:

[0022] According to the obtained Pearson correlation coefficient, and The amount of information, The amount of information is: , The amount of information is: ; The weight is obtained from this , ; S520, calculate the total minimum error and construct the objective function, which is expressed as:

[0023] Satisfy the constraints:

[0024] Then, step S511 to step S512 are repeated to traverse all rectangles of the dimensional transformation matrix and the coordinates of the athletes after the dimensional transformation. The dimensional transformation matrix with the smallest total error is the optimal dimensional transformation matrix.

[0025] Another aspect of the present invention provides a system for realizing the above-mentioned method of generating a bird's-eye view of a game situation based on multi-camera information fusion, comprising a module for obtaining information of a current video frame, a module for rectangular matching and dimensional transformation, a module for calculating coordinate transformation errors, a module for weighted evaluation of heterogeneous errors, and a module for restoring a bird's-eye view by fusing local information, wherein: The module for obtaining the information of the current frame of the video is used to perform HSV color space separation processing on the image of the current frame in the video, and obtain all the information points on the current scene captured by a certain camera and store them in a point concentration; The rectangle matching and dimension transformation module is used to extend and intersect the straight lines of the top view, record the rectangle coordinates, obtain the matching rectangle pair based on similarity matching, use the rectangle coordinates of the original view and the top view of the same camera, obtain the error of these rectangle coordinates and generate the dimension transformation matrix; A coordinate transformation error calculation module is used to transform the coordinates of the athletes in the overlapping area to a bird's-eye view, calculate the Euclidean distance of the same athlete in the bird's-eye view, and obtain the coordinate transformation error of the athlete; The heterogeneous error weighted assessment module is used to assign corresponding weight coefficients to the error types from different sources according to their respective importance and impact range, and to evaluate the overall error situation through error weighting and comprehensive assessment; The module for restoring the bird's-eye view by fusing local information is used to fuse the local information of the cameras at the four corners of the venue in the current frame according to the overlapping areas captured by the cameras, and obtain the global target of the stadium, thereby transforming the dimension of the full-field information of the stadium in the current frame to obtain the corresponding full-field bird's-eye view.

[0026] Another aspect of the present invention provides a non-transitory computer-readable storage medium having computer instructions stored thereon, which enable a computer to execute the above-mentioned method of generating a bird's-eye view of a match situation based on multi-camera information fusion.

[0027] Yet another aspect of the present invention provides an electronic device, comprising: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus, and the processor calls the logic instructions in the memory to execute the above-mentioned method of generating a bird's-eye view of the game situation based on multi-camera information fusion.

[0028] On the other hand, the present invention provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer executes the above-mentioned method for generating a bird's-eye view of the game situation based on multi-camera information fusion.

[0029] Beneficial effect: The method of the present invention is based on the fusion of multi-camera information to generate a bird's-eye view of the game situation. By fusing each frame of images taken by multiple cameras and then changing the dimension, a bird's-eye view of the entire field information is generated in real time. Ultimately, real-time bird's-eye view generation of sports events can be achieved, helping users to better grasp the on-field information and gain in-depth insight into the dynamic evolution of the field situation.

[0030] The present invention is based on a method for generating a bird's-eye view of the game situation based on multi-camera information fusion. Relying on the original video scene features obtained through comprehensive shooting, the present invention uses multivariate computational analysis methods to carry out various analysis tasks, effectively eliminating the influence of external interference factors. By converting the original video covering the entire stadium into a bird's-eye view image, the system overcomes the difficulties in observing the field situation caused by occlusion, poor viewing angles, incomplete images, and other problems, and provides users with more convenient and comprehensive support in the process of real-time viewing and field situation analysis of sports events.

[0031] The present invention transforms dimensions by identifying and screening rectangles, fuses field information from different perspectives according to overlapping parts, and constructs a bird's-eye view of the competition venue, so as to have a more reasonable and accurate grasp of the field conditions of sports events in real time. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 The overall flow chart of the method for generating a bird's-eye view of the game situation based on multi-camera information fusion. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments, and they should not be understood as limitations on the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. In the description of the present invention, it should be understood that the terms used are only for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0034] Combine the following Figure 1 The present invention describes a method and system for generating a bird's-eye view of a game situation based on multi-camera information fusion.

[0035] Embodiment 1: This embodiment provides a method for generating a bird's-eye view of a game situation based on multi-camera information fusion, such as Figure 1 As shown, the following steps are included: S100, a camera is placed at each of the four corners of the sports field. The cameras at the four corners of the field simultaneously shoot and record the sports field in real time, extract the current frame image in the video, and record the current time sequence as , and then establish the coordinate origin with the lower left corner of the stadium Plane rectangular coordinate system, simultaneously recording the relative positions of the four cameras , the height and width of the court are and .

[0036] S200, perform HSV color separation on the current frame image of each camera, set the value range of the required information points or lines, extract the required information on the current frame image and record it. Take any corner point on the image as the reference point, and obtain the four points that form the rectangle by continuously determining the straight line and calculating the closest point. according to Put the points into the order ;According to the above operation, rectangles at different distances are obtained respectively, and the four points are stored in the point set in the form of a two-dimensional array In the timing Create a top view of the entire field. According to all the boundary lines obtained on the field, extend the boundary lines to divide the field into N rectangles, record the coordinates of all rectangles in order, and store the point set of the rectangles. ; After the four cameras implement step S200 respectively, the corresponding rectangular point set will be obtained and : S210, convert the current frame image from the RGB three-color channel to the HSV color space, take the lower left corner of the current frame image as the origin, and establish Plane rectangular coordinate system, set the appropriate value range, and identify all straight line equations , calculate the intersection points of all straight lines and record them; S220, randomly select one of the four vertices of the current frame image as the reference point, and assume that the length of the image is , the height is , will be with the point The distance is the minimum The intersection of ,point It is produced by the intersection of two straight lines. Point coordinates, get the The analytical expression of the two straight lines of the point is recorded as and ; S230, With The intersection point is obtained by combining the equations of the identified lines outside , , … , similarly, With The identified straight lines outside are combined to obtain the intersection point , , … ;calculate and and Distance of points and , respectively select and The corresponding points are denoted by and , the other straight line where the two points lie is and ; S240, straight line With straight line The intersection point is obtained by combining the analytical expressions of , and then the four points obtained As vertex coordinates are stored in order in the point set middle; S250, repeat steps S210 to S240, traverse according to the difference in the distance between the two points, find the point set of the rectangle on the original image, until no new distance difference appears, then all the rectangular point sets on the original image are found, and stored in the point set in sequence Different rows of S260, when the timing is When the image of the entire playing field or the dimensions of the field is obtained, a top-view coordinate system with the vertices of the playing field as the coordinate origin is created. , get the coordinates of all intersection points on the site , according to the coordinates of the four vertices of the site, the coordinates on the matching coordinate system Calculate a three-row and three-column transformation matrix, and multiply all points on the field by this transformation matrix to get the new intersection coordinates , so as to obtain the top view F of the site and record the time series Afterwards, directly obtain the top view F of the initial state; S270, obtained in step S260 By taking any two points whose horizontal and vertical coordinates are different, we can determine the four points of a rectangle and store them in an array in clockwise order. In the example, all the points are connected to generate n horizontal lines and m vertical lines, which divide the entire top view site into several small rectangles and large rectangles composed of small rectangles. Rectangle , record the coordinates of all rectangles in order from left to right and from small to large, using eight lines The obtained rectangular point set is stored in an array of columns. .

[0037] S300, a rectangle screening and scoring algorithm is proposed: according to the four vertices of the rectangle on a certain frame image obtained in step S200, the corresponding matrix coordinate points in the top view are found and screened, so as to restore all the coordinates in the frame image to the top view by the method of restoring the coordinates, and then the target rectangle and the selected rectangle are matched by the rectangle matching and error analysis algorithm, so as to further accurately screen the rectangle. The specific method is: S310, based on the four vertices of the rectangle on a certain frame image obtained in step S200, introducing a rectangle matching scoring algorithm model to preliminarily identify and select suitable rectangles; including: S311, based on the point set obtained in step S200 and Get a new top view F1, first compare the boundary lengths of the top view F and F1. If a boundary length of F1 is greater than the corresponding boundary length in F, it means that the currently selected rectangle point is deviated or inappropriate. Then compare the rectangles in the figure. If the length of a side of the rectangle in F1 is obviously greater than the length of the corresponding side of the rectangle in F, it can be determined that the current target rectangle is not compatible with the selected rectangle in size and shape. Remove this part of the rectangle and store the remaining rectangle coordinate points in a new point set. middle; S312, based on point set , Calculate the length Length, width Width and the width of each rectangle under two viewing angles. , and obtain the centroid coordinates of each set of rectangles at the same time, and define the aspect ratio as , the angle difference is , the center of gravity distance is , calculate the Hausdorff distance of the point set of the rectangular coordinates under two perspectives, expressed as:

[0038] According to the top view creation method in step S200, the top view F2 restored under the camera's viewing angle is obtained, thereby obtaining two straight lines representing the viewing angle in the top view F2, and the two straight lines are extended in the opposite direction and intersected to obtain the coordinates of an imaginary camera. , by comparing with the camera coordinates in step S100 Compare and get a Euclidean distance difference ; Then normalize the data and calculate the normalized similarity score :

[0039] in =0.12, =0.15, =0.13, =0.10, =0.5, traverse and All rectangular coordinate groups in , calculate the similarity scores of all matching combinations , stored in point sets , In the example, compare the scores, get the matching rectangles, and update them in a one-to-one order. and .

[0040] S320: It should be noted that when comparing the coordinates of the selected target rectangle with the selected rectangle, it cannot be guaranteed that the order of the coordinate points in their corresponding point sets is consistent. Therefore, a circular method is used, that is, the point set of the coordinates obtained when selecting the target rectangle is Perform shift processing, and then repeat step S310. Since a rectangle can select four coordinate points, it only needs to be cycled four times to obtain all the optimal solutions. S330, after obtaining the most suitable rectangle pair, by comparing the point set and The coordinate points in can be calculated to obtain a dimensional transformation matrix.

[0041] S400, transforming the athlete captured by the camera to the stadium under the bird's-eye view, calculating the coordinate difference of the athlete after the transformation at different perspectives as the error basis of the dimensional transformation matrix, the specific method is: S410, selecting two viewing angles of camera A and camera B, using YOLO to identify all athletes in the viewing angle of the camera and taking the coordinates (X, Y) of their center points as the positions of the athletes; S420, calculating the coordinates of the athletes in the overlapping area after dimension transformation at each viewing angle, and storing them in a point set. The specific implementation steps are as follows: S421, according to the dimension change matrix calculated in step S300, the position of the athlete under each camera's view is dimensionally transformed to obtain the coordinates (X', Y') after the dimension change, and the coordinates of all athletes photographed by the i-th camera after the dimension change are stored in the point set Si; S422, calculating the coordinate range of the overlapping area between cameras, selecting the coordinates that meet the range in the point set and storing them in a new point set S'i; S430, take point set S'1 and point set S'2, calculate the Euclidean distance between all point pairs in the two perspectives as the matching cost, construct a cost matrix and apply the Hungarian algorithm to match the point pairs; the specific implementation steps are: S431, define the detection point set of camera A as , the detection point set of camera B is , calculate the Euclidean distance between all point pairs in the two perspectives as the matching cost, and construct the cost matrix as , whose elements are ; S432. Solve the row index and column index set , so that , get the ordered matching index list .

[0042] S440, for each matching row and column index (i, j), the point corresponding to camera A , and the point of camera B , reorder the point set according to the matching results to ensure that the coordinates of the corresponding positions belong to the same player, and for each matching point pair ( , ) to obtain the coordinate error ; Step 450: Repeat the process from step S410 to step S440 for viewing angles B and C, viewing angles C and D, and viewing angles D and A, and superimpose the error values ​​of the overlapping areas to obtain a final error value. .

[0043] S500, introducing a method for determining the optimal dimensional transformation matrix based on the minimum total error: in a sports event venue, the shooting ranges of the cameras at the four corners of the venue overlap with each other, forming a globally covered monitoring area; there are four dimensional transformation matrices obtained from the non-overlapping areas of the four cameras, and the error superposition of the rectangles of the four dimensional transformation matrices is defined to form an error point set. and the set of coordinate transformation errors of the players in the overlapping area of ​​each camera , respectively, the two errors are matched with appropriate coefficients, and the weighted sum is minimized as the objective function to obtain the optimal dimensional transformation matrix. The specific method of step S500 is: S510, obtain a four-dimensional transformation matrix from a rectangle obtained by preliminary screening of a single camera, and superimpose the errors of the four-dimensional transformation matrix rectangles to form an error point set , , Indicates that from The error point set of the dimensional transformation matrix of the four cameras; the error value of the coordinate transformation of the athlete in the common area captured by the four cameras is recorded as ; After unifying the scale and standardizing the two error sets and , through the CRITIC weight method and Assign appropriate weights to each and , by weighted summing and solving the total minimum error to determine the optimal dimension transformation matrix; the specific method is as follows: S511. Since the sources and scales of the two error sets are different, , , let the target length be ,in , They are and The number of original elements of the error point set and error and Adjust to the same length , used for subsequent alignment of quantiles, using linear interpolation to extend the length of the shorter set, assuming that the shorter set is extended get ; Min-Max normalization method is used to and Perform standardization and calculate:

[0044]

[0045] get and ; S512, error point set and error and Assign appropriate weights to each of them and apply the CRITIC weight method to normalize the and Defined as two indicators, construct the matrix:

[0046] Then calculate the standard deviation, The standard deviation of is:

[0047] The standard deviation of is:

[0048] Compute the Pearson correlation coefficient between two columns:

[0049] According to the obtained Pearson correlation coefficient, and The amount of information, The amount of information is: , The amount of information is: ; The weight is obtained from this , ; S520, calculate the total minimum error and construct the objective function, which is expressed as:

[0050] Satisfy the constraints:

[0051] Then, step S511 to step S512 are repeated to traverse all rectangles of the dimensional transformation matrix and the coordinates of the athletes after the dimensional transformation. The dimensional transformation matrix with the smallest total error is the optimal dimensional transformation matrix.

[0052] S600: After the current sequence ends, switch to the next sequence, perform dimension transformation on the entire sports field according to the optimal dimension transformation matrix obtained in step 500, and save all points of the required field information after transformation in the point set In the process, a bird's-eye view is generated by the information stored in the point set, and other point sets are cleared, and the frame image of the next time sequence in the video is captured, and steps S100 to S500 are repeated to continuously obtain a bird's-eye view.

[0053] The present invention will fuse each frame of images taken by multiple cameras and perform dimensional changes to generate a bird's-eye view of the entire field in real time. Ultimately, it can achieve the generation of real-time bird's-eye views of sports events, help users better grasp the on-field information, and help users gain in-depth insights into the dynamic evolution of the field situation.

[0054] The present invention designs a method for generating a global bird's-eye view of a sports event based on multi-camera information fusion. The bird's-eye view is generated in real time by converting the dimensions of the same frame of video captured by multiple cameras. The method is suitable for video broadcasting and field analysis scenarios of sports events. It relies on the original video scene features obtained through comprehensive shooting, and uses multivariate computing and analysis methods to carry out various analysis tasks, effectively eliminating the influence of external interference factors. By converting the original video covering the entire stadium into a bird's-eye view image, the system overcomes the difficulties in field observation caused by occlusion, poor viewing angle, incomplete images, etc., and provides users with more convenient and comprehensive support in the process of real-time viewing and field analysis of sports events.

[0055] The present invention transforms dimensions by identifying and screening rectangles, fuses field information from different perspectives according to overlapping parts, and constructs a bird's-eye view of the competition venue, so as to have a more reasonable and accurate grasp of the field conditions of sports events in real time.

[0056] Embodiment 2: This embodiment provides a system for generating a bird's-eye view of a match situation based on multi-camera information fusion for implementing the method of Embodiment 1. The system includes a module for obtaining current frame information of a video, a rectangular matching and dimensional transformation module, a coordinate transformation error calculation module, a heterogeneous error weighted evaluation module, and a module for fusing local information to restore a bird's-eye view.

[0057] Among them: the module for obtaining the information of the current frame of the video is used to perform HSV color space separation processing on the image of the current frame in the video, and obtain all the information points on the current scene captured by a certain camera and store them in a point concentration; The rectangle matching and dimension transformation module is used to extend and intersect the straight lines of the top view, record the rectangle coordinates, obtain the matching rectangle pair based on similarity matching, use the rectangle coordinates of the original view and the top view of the same camera, obtain the error of these rectangle coordinates and generate the dimension transformation matrix; A coordinate transformation error calculation module is used to transform the coordinates of the athletes in the overlapping area to a bird's-eye view, calculate the Euclidean distance of the same athlete in the bird's-eye view, and obtain the coordinate transformation error of the athlete; The heterogeneous error weighted assessment module is used to assign corresponding weight coefficients to the error types from different sources according to their respective importance and impact range, and to evaluate the overall error situation through error weighting and comprehensive assessment; The module for restoring the bird's-eye view by fusing local information is used to fuse the local information of the cameras at the four corners of the venue in the current frame according to the overlapping areas captured by the cameras, and obtain the global target of the stadium, thereby transforming the dimension of the full-field information of the stadium in the current frame to obtain the corresponding full-field bird's-eye view.

[0058] Embodiment 3: This embodiment provides a non-transitory computer-readable storage medium on which computer instructions are stored. The computer instructions enable a computer to execute a method for generating a bird's-eye view of a game situation based on multi-camera information fusion. The method includes the following steps: S100, placing cameras at the four corners of the stadium, recording the stadium in real time, and capturing the current frame image in the video; using one of the corners of the stadium as the coordinate origin, establishing a plane rectangular coordinate system, and recording the relative positions of the four cameras and the height and width of the stadium; S200, perform HSV color space separation on the current frame image of each camera, set a value range to identify all straight lines and intersections on the image, randomly select a corner point on the image as a reference point, calculate and determine the intersection closest to the reference point, and then find other intersections related to the straight line where the intersection is located by analyzing the straight line, and gradually determine the coordinates of the four vertices of the rectangle through these intersections and store them in the point set in order. ; exist Always create a top view legend for the entire scene, and store all intersection points in the top view into a point set after transforming them through the transformation matrix In the example, the site is divided into Rectangles, record the coordinates of all rectangles in order from left to right and from small to large, and store them as a rectangular point set based on the top view ; S300, according to the four vertices of the rectangle on a certain frame image obtained in step S200, find and filter the corresponding matrix coordinate points in the top view, restore all the coordinates in the frame image to the top view by a coordinate restoration method, and then match the target rectangle with the selected rectangle by a rectangle matching and error analysis algorithm to accurately filter the rectangle; S400, introducing a method for calculating the coordinate transformation error of an athlete based on multi-view camera fusion, obtaining the coordinates of the athlete in the overlapping area between cameras, and using a matrix to transform to a bird's-eye view, calculating the Euclidean distance of the same athlete in the bird's-eye view, and obtaining the coordinate transformation error of the athlete; S500, superimposing the errors of the four-dimensional transformation matrices obtained in the non-overlapping areas of the four cameras to form an error point set ; Construct the error point set based on the coordinate transformation error of the athletes in the overlapping area of ​​the four cameras ; Set coefficients for the two error point sets respectively, take the weighted sum minimum as the objective function, and obtain the optimal dimensional transformation matrix; S600: After the current sequence ends, switch to the next sequence, perform dimension transformation on the entire sports field according to the optimal dimension transformation matrix obtained in step 500, and save all points of the required field information after transformation in the point set In the process, a bird's-eye view is generated by the information stored in the point set, and other point sets are cleared, and the frame image of the next time sequence in the video is captured, and steps S100 to S500 are repeated to continuously obtain a bird's-eye view.

[0059] Embodiment 4: This embodiment provides an electronic device, which may include: a processor, a communications interface, a memory, and a communication bus, wherein the processor, the communications interface, and the memory communicate with each other via the communication bus. The processor may call the logic instructions in the memory to execute a method for generating a bird's-eye view of a game situation based on multi-camera information fusion, the method comprising the following steps: S100, placing cameras at the four corners of the stadium, recording the stadium in real time, and capturing the current frame image in the video; using one of the corners of the stadium as the coordinate origin, establishing a plane rectangular coordinate system, and recording the relative positions of the four cameras and the height and width of the stadium; S200, perform HSV color space separation on the current frame image of each camera, set a value range to identify all straight lines and intersections on the image, randomly select a corner point on the image as a reference point, calculate and determine the intersection closest to the reference point, and then find other intersections related to the straight line where the intersection is located by analyzing the straight line, and gradually determine the coordinates of the four vertices of the rectangle through these intersections and store them in the point set in order. ; exist Always create a top view legend for the entire scene, and store all intersection points in the top view into a point set after transforming them through the transformation matrix In the example, the site is divided into Rectangles, record the coordinates of all rectangles in order from left to right and from small to large, and store them as a rectangular point set based on the top view ; S300, according to the four vertices of the rectangle on a certain frame image obtained in step S200, find and filter the corresponding matrix coordinate points in the top view, restore all the coordinates in the frame image to the top view by a coordinate restoration method, and then match the target rectangle with the selected rectangle by a rectangle matching and error analysis algorithm to accurately filter the rectangle; S400, introducing a method for calculating the coordinate transformation error of an athlete based on multi-view camera fusion, obtaining the coordinates of the athlete in the overlapping area between cameras, and using a matrix to transform to a bird's-eye view, calculating the Euclidean distance of the same athlete in the bird's-eye view, and obtaining the coordinate transformation error of the athlete; S500, superimposing the errors of the four-dimensional transformation matrices obtained in the non-overlapping areas of the four cameras to form an error point set ; Construct the error point set based on the coordinate transformation error of the athletes in the overlapping area of ​​the four cameras ; Set coefficients for the two error point sets respectively, take the weighted sum minimum as the objective function, and obtain the optimal dimensional transformation matrix; S600: After the current sequence ends, switch to the next sequence, perform dimension transformation on the entire sports field according to the optimal dimension transformation matrix obtained in step 500, and save all points of the required field information after transformation in the point set In the process, a bird's-eye view is generated by the information stored in the point set, and other point sets are cleared, and the frame image of the next time sequence in the video is captured, and steps S100 to S500 are repeated to continuously obtain a bird's-eye view.

[0060] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

[0061] Embodiment 5: This embodiment provides a computer program product, which includes a computer program. The computer program can be stored in a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute a method for generating a bird's-eye view of a game situation based on multi-camera information fusion. The method includes the following steps: S100, placing cameras at the four corners of the stadium, recording the stadium in real time, and capturing the current frame image in the video; using one of the corners of the stadium as the coordinate origin, establishing a plane rectangular coordinate system, and recording the relative positions of the four cameras and the height and width of the stadium; S200, perform HSV color space separation on the current frame image of each camera, set a value range to identify all straight lines and intersections on the image, randomly select a corner point on the image as a reference point, calculate and determine the intersection closest to the reference point, and then find other intersections related to the straight line where the intersection is located by analyzing the straight line, and gradually determine the coordinates of the four vertices of the rectangle through these intersections and store them in the point set in order. ; exist Always create a top view legend for the entire scene, and store all intersection points in the top view into a point set after transforming them through the transformation matrix In the example, the site is divided into Rectangles, record the coordinates of all rectangles in order from left to right and from small to large, and store them as a rectangular point set based on the top view ; S300, according to the four vertices of the rectangle on a certain frame image obtained in step S200, find and filter the corresponding matrix coordinate points in the top view, restore all the coordinates in the frame image to the top view by a coordinate restoration method, and then match the target rectangle with the selected rectangle by a rectangle matching and error analysis algorithm to accurately filter the rectangle; S400, introducing a method for calculating the coordinate transformation error of an athlete based on multi-view camera fusion, obtaining the coordinates of the athlete in the overlapping area between cameras, and using a matrix to transform to a bird's-eye view, calculating the Euclidean distance of the same athlete in the bird's-eye view, and obtaining the coordinate transformation error of the athlete; S500, superimposing the errors of the four-dimensional transformation matrices obtained in the non-overlapping areas of the four cameras to form an error point set ; Construct the error point set based on the coordinate transformation error of the athletes in the overlapping area of ​​the four cameras ; Set coefficients for the two error point sets respectively, take the weighted sum minimum as the objective function, and obtain the optimal dimensional transformation matrix; S600: After the current sequence ends, switch to the next sequence, perform dimension transformation on the entire sports field according to the optimal dimension transformation matrix obtained in step 500, and save all points of the required field information after transformation in the point set In the process, a bird's-eye view is generated by the information stored in the point set, and other point sets are cleared, and the frame image of the next time sequence in the video is captured, and steps S100 to S500 are repeated to continuously obtain a bird's-eye view.

[0062] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0063] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0064] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating a bird's-eye view of a game situation based on multi-camera information fusion, characterized in that: include: S100, placing cameras at the four corners of the stadium, simultaneously shooting and recording the stadium in real time, and capturing the current frame image in the video; Using one of the corners of the stadium as the origin, a rectangular coordinate system is established to record the relative positions of the four cameras and the height and width of the stadium; S200, perform HSV color space separation on the current frame image of each camera, set a value range to identify all straight lines and intersections on the image, randomly select a corner point on the image as a reference point, calculate and determine the intersection closest to the reference point, and then find other intersections related to the straight line where the intersection is located by analyzing the straight line, and gradually determine the coordinates of the four vertices of the rectangle through these intersections and store them in the point set in order. ; exist Always create a top view legend for the entire scene, and store all intersection points in the top view into a point set after transforming them through the transformation matrix In the example, the site is divided into Rectangles, record the coordinates of all rectangles in order from left to right and from small to large, and store them as a rectangular point set based on the top view ; S300, according to the four vertices of the rectangle on a certain frame image obtained in step S200, find and filter the corresponding matrix coordinate points in the top view, restore all the coordinates in the frame image to the top view by a coordinate restoration method, and then match the target rectangle with the selected rectangle by a rectangle matching and error analysis algorithm to accurately filter the rectangle; S400, introducing a method for calculating the coordinate transformation error of an athlete based on multi-view camera fusion, obtaining the coordinates of the athlete in the overlapping area between cameras, and using a matrix to transform to a bird's-eye view, calculating the Euclidean distance of the same athlete in the bird's-eye view, and obtaining the coordinate transformation error of the athlete; S500, superimposing the errors of the four-dimensional transformation matrices obtained in the non-overlapping areas of the four cameras to form an error point set ; Construct the error point set based on the athlete coordinate transformation errors in the overlapping area of ​​the four cameras ; Set coefficients for the two error point sets respectively, take the weighted sum minimum as the objective function, and obtain the optimal dimensional transformation matrix; S600: After the current sequence ends, switch to the next sequence, perform dimension transformation on the entire sports field according to the optimal dimension transformation matrix obtained in step 500, and save all points of the required field information after transformation in the point set In the process, a bird's-eye view is generated by the information stored in the point set, and other point sets are cleared, and the frame image of the next time sequence in the video is captured, and steps S100 to S500 are repeated to continuously obtain a bird's-eye view.

2. The method for generating a bird's-eye view of a game situation based on multi-camera information fusion according to claim 1, characterized in that: In step S100, the lower left corner of the stadium is used as the coordinate origin to establish Plane rectangular coordinate system, recording the relative positions of the four cameras , , , The height of the stadium is and width is .

3. The method for generating a bird's-eye view of a game situation based on multi-camera information fusion according to claim 2, characterized in that: In step S200, the current frame image of each camera is separated in HSV color space, and a value range is set to identify all straight lines and intersections on the image. A corner point on the image is randomly selected as a reference point, and the intersection closest to the reference point is calculated and determined. Then, by analyzing the straight line where the intersection point is located, other intersections related to it are found. Through these intersections, the coordinates of the four vertices of the rectangle are gradually determined and stored in the point set in order. ; The specific method is: S210, convert the current frame image from the RGB three-color channel to the HSV color space, take the lower left corner of the current frame image as the origin, and establish Plane rectangular coordinate system, set the appropriate value range, and identify all straight line equations , calculate the intersection points of all straight lines and record them; S220, randomly select one of the four vertices of the current frame image as the reference point, and assume that the length of the image is , the height is , will be with the point The distance is the minimum The intersection of ,point It is produced by the intersection of two straight lines. Point coordinates, get the The analytical expression of the two straight lines of the point is recorded as and ; S230, With The intersection point is obtained by combining the equations of the identified lines outside , , … , similarly, With The identified straight lines outside are combined to obtain the intersection point , , … ;calculate and and Distance of points and , respectively select and The corresponding points are denoted as and , the other straight line where the two points lie is and ; S240, straight line With straight line The intersection point is obtained by combining the analytical expressions of , and then the four points obtained As vertex coordinates are stored in order in the point set middle; S250, repeat steps S210 to S240, traverse according to the difference in the distance between the two points, find the point set of the rectangle on the original image, until no new distance difference appears, then all the rectangular point sets on the original image are found, and stored in the point set in sequence different rows.

4. The method for generating a bird's-eye view of a game situation based on multi-camera information fusion according to claim 3, characterized in that: In step S200, Always create a top view legend for the entire scene, and store all intersection points in the top view into a point set after transforming them through the transformation matrix In the example, the site is divided into Rectangles, record the coordinates of all rectangles in order from left to right and from small to large, and store them as a rectangular point set based on the top view ; The specific method is: S260, when the timing is When the image of the entire playing field or the dimensions of the field is obtained, a top-view coordinate system with the vertices of the playing field as the coordinate origin is created. , get the coordinates of all intersection points on the site , according to the coordinates of the four vertices of the site, the coordinates on the matching coordinate system Calculate a three-row and three-column transformation matrix, and multiply all points on the field by this transformation matrix to get the new intersection coordinates , so as to obtain the top view F of the site and record the time series Afterwards, directly obtain the top view F of the initial state; S270, obtained in step S260 By taking any two points whose horizontal and vertical coordinates are different, we can determine the four points of a rectangle and store them in an array in clockwise order. In the example, all the points are connected to generate n horizontal lines and m vertical lines, which divide the entire top view site into several small rectangles and large rectangles composed of small rectangles. Rectangle , record the coordinates of all rectangles in order from left to right and from small to large, using eight lines The obtained rectangular point set is stored in an array of columns. .

5. The method for generating a bird's-eye view of a game situation based on multi-camera information fusion according to claim 4, characterized in that: In step S300, according to the four vertices of the rectangle on a frame image obtained in step S200, the corresponding matrix coordinate points in the top view are found and screened, all the coordinates in the frame image are restored to the top view by the coordinate restoration method, and then the target rectangle and the selected rectangle are matched by the rectangle matching and error analysis algorithm to accurately screen the rectangle, including: S310, based on the four vertices of the rectangle on a certain frame image obtained in step S200, a rectangle matching scoring algorithm model is introduced to preliminarily identify and select a suitable rectangle; S320, adopting a loop method, that is, selecting the point set of the coordinates obtained when the target rectangle is selected Perform shift processing, and then repeat step S310 to obtain all optimal solutions; S330: After obtaining the most suitable rectangle pair, a dimensional transformation matrix is ​​calculated.

6. The method for generating a bird's-eye view of a game situation based on multi-camera information fusion according to claim 5, characterized in that: In step S310, based on the four vertices of the rectangle on a certain frame image obtained in step S200, a rectangle matching scoring algorithm model is introduced to preliminarily identify and select suitable rectangles; the specific method is: S311, based on the point set obtained in step S200 and Get a new top view F1, first compare the boundary lengths of the top view F and F1. If a boundary length of F1 is greater than the corresponding boundary length in F, it means that the currently selected rectangle point is deviated or inappropriate. Then compare the rectangles in the figure. If the length of a side of the rectangle in F1 is obviously greater than the length of the corresponding side of the rectangle in F, it can be determined that the current target rectangle is not compatible with the selected rectangle in size and shape. Remove this part of the rectangle and store the remaining rectangle coordinate points in a new point set. Medium; S312, based on point set , Calculate the length Length, width Width and the width of each rectangle under two viewing angles. , and obtain the centroid coordinates of each set of rectangles at the same time, and define the aspect ratio as , the angle difference is , the center of gravity distance is , calculate the Hausdorff distance of the point set of the rectangular coordinates under two perspectives, expressed as: According to the top view creation method in step S200, the top view F2 restored under the camera's viewing angle is obtained, thereby obtaining two straight lines representing the viewing angle in the top view F2, and the two straight lines are extended in the opposite direction and intersected to obtain the coordinates of an imaginary camera. , by comparing with the camera coordinates in step S100 Compare and get a Euclidean distance difference ; Then normalize the data and calculate the normalized similarity score : in =0.12, =0.15, =0.13, =0.10, =0.5, traverse and All rectangular coordinate groups in , calculate the similarity scores of all matching combinations , stored in point sets , In the example, compare the scores, get the matching rectangles, and update them in a one-to-one order. and .

7. The method for generating a bird's-eye view of a game situation based on multi-camera information fusion according to claim 5, characterized in that: In step S400, the coordinates of the athletes in the overlapping area between the cameras are obtained, and the coordinates are transformed to the top view using a matrix, and the Euclidean distance of the same athlete in the top view is calculated to obtain the coordinate transformation error of the athlete; the specific method is: S410, selecting two viewing angles of camera A and camera B, using YOLO to identify all athletes in the viewing angle of the camera and taking the coordinates (X, Y) of their center points as the positions of the athletes; S420, calculating the coordinates of the athletes in the overlapping area after dimension transformation at each viewing angle, and storing them in a point set. The specific implementation steps are as follows: S421, according to the dimension change matrix calculated in step S300, the position of the athlete under each camera's view is dimensionally transformed to obtain the coordinates (X', Y') after the dimension change, and the coordinates of all athletes photographed by the i-th camera after the dimension change are stored in the point set Si; S422, calculating the coordinate range of the overlapping area between cameras, selecting the coordinates that meet the range in the point set and storing them in a new point set S'i; S430, taking point set S'1 and point set S'2, calculating the Euclidean distance between all point pairs in the two perspectives as the matching cost, constructing a cost matrix, and applying the Hungarian algorithm to match the point pairs; S440, for each matching row and column index (i, j), the point corresponding to camera A , and the point of camera B , reorder the point set according to the matching results to ensure that the coordinates of the corresponding positions belong to the same player, and for each matching point pair ( , ) to obtain the coordinate error ; Step 450: Repeat the process from step S410 to step S440 for viewing angles B and C, viewing angles C and D, and viewing angles D and A, and superimpose the error values ​​of the overlapping areas to obtain a final error value. .

8. The method for generating a bird's-eye view of a game situation based on multi-camera information fusion according to claim 7, characterized in that: The specific implementation steps of step S430 are: S431, define the detection point set of camera A as , the detection point set of camera B is , calculate the Euclidean distance between all point pairs in the two perspectives as the matching cost, and construct the cost matrix as , whose elements are ; S432. Solve the row index and column index set , so that , get the ordered matching index list .

9. The method for generating a bird's-eye view of a game situation based on multi-camera information fusion according to claim 7, characterized in that: The specific method of step S500 is: S510, obtain a four-dimensional transformation matrix from a rectangle obtained by preliminary screening of a single camera, and superimpose the errors of the four-dimensional transformation matrix rectangles to form an error point set , , Indicates that from The error point set of the dimensional transformation matrix of the four cameras; the error value of the coordinate transformation of the athlete in the common area captured by the four cameras is recorded as ; After unifying the scale and standardizing the two error sets and , through the CRITIC weight method and Assign appropriate weights to each and , by weighted summing and solving the total minimum error to determine the optimal dimension transformation matrix; the specific method is as follows: S511. Since the sources and scales of the two error sets are different, , , let the target length be ,in , They are and The number of original elements of the error point set and error and Adjust to the same length , used for subsequent alignment of quantiles, using linear interpolation to extend the length of the shorter set, assuming that the shorter set is extended get ; Min-Max normalization method is used to and Perform standardization and calculate: get and ; S512, error point set and error and Assign appropriate weights to each of them and apply the CRITIC weight method to normalize the and Defined as two indicators, construct the matrix: Then calculate the standard deviation, The standard deviation of is: The standard deviation of is: Compute the Pearson correlation coefficient between two columns: According to the obtained Pearson correlation coefficient, and The amount of information, The amount of information is: , The amount of information is: ; The weight is obtained from this , ; S520, calculate the total minimum error and construct the objective function, which is expressed as: Satisfy the constraints: Then, step S511 to step S512 are repeated to traverse all rectangles of the dimensional transformation matrix and the coordinates of the athletes after the dimensional transformation. The dimensional transformation matrix with the smallest total error is the optimal dimensional transformation matrix.

10. A system for generating a bird's-eye view of a game situation based on multi-camera information fusion for implementing any one of the methods of claims 1 to 9, characterized in that: It includes a module for obtaining the information of the current video frame, a module for rectangle matching and dimension transformation, a module for calculating the coordinate transformation error, a module for weighted evaluation of heterogeneous errors, and a module for restoring the bird's-eye view by fusing local information, wherein: the module for obtaining the information of the current video frame is used to perform HSV color space separation processing on the image of the current frame in the video, and obtain all the information points on the current scene captured by a certain camera and store them in a point concentration; The rectangle matching and dimension transformation module is used to extend and intersect the straight lines of the top view, record the rectangle coordinates, obtain the matching rectangle pair based on similarity matching, use the rectangle coordinates of the original view and the top view of the same camera, obtain the error of these rectangle coordinates and generate the dimension transformation matrix; A coordinate transformation error calculation module is used to transform the coordinates of the athletes in the overlapping area to a bird's-eye view, calculate the Euclidean distance of the same athlete in the bird's-eye view, and obtain the coordinate transformation error of the athlete; The heterogeneous error weighted assessment module is used to assign corresponding weight coefficients to the error types from different sources according to their respective importance and impact range, and to evaluate the overall error situation through error weighting and comprehensive assessment; The module for restoring the bird's-eye view by fusing local information is used to fuse the local information of the cameras at the four corners of the venue in the current frame according to the overlapping areas captured by the cameras, and obtain the global target of the stadium, thereby transforming the dimension of the full-field information of the stadium in the current frame to obtain the corresponding full-field bird's-eye view.

Images