Camera point complementing method for global tracking

Abstract

The invention discloses a camera point supplementing method for global tracking. The method comprises the following steps: constructing a three-dimensional model of a region, and carrying out projection to generate a grid coordinate system of a two-dimensional base map; solving a homography matrix of the existing camera according to a mapping relation between the acquired data of the existing camera and the three-dimensional model; calculating an accurate sensing area of the existing camera, and combining the grid areas with the continuous relation to form a closed sensing area; based on the obtained closed sensing area, carrying out reverse deduction to obtain a plurality of blind areas to be compensated which need to be monitored; and calculating the average sensing range of the existing cameras, calculating the minimum number of camera compensation points in each to-be-compensated blind area, and performing deployment and control. The method has the beneficial effects that the relation between the visual angle of the camera and the spatial range can be effectively recovered in the absence of detailed external parameters, so that the blind area range is judged; therefore, judgment and complementation of visual dead angles can be completed without replacing hardware of the original monitoring system.

Description

Technical Field

[0001] The present application relates to the field of intelligent transportation technology, and in particular to a camera point filling method for global tracking. Background Art

[0002] In semi-enclosed scenarios, such as highway service areas, these areas serve as critical nodes in the transportation system, fulfilling multiple functions, including temporary parking, energy supply, and emergency management. To ensure the daily management and safety of service areas, numerous surveillance cameras are typically deployed during the design and construction phases. However, these cameras often focus on regional monitoring and fail to systematically address the need for "global tracking" of vehicles.

[0003] As the demand for vehicle dynamic management, path reconstruction, and full-time and full-domain perception continues to increase, relying solely on existing cameras can no longer meet the requirements for spatiotemporal continuity and spatial accuracy in real-world scenarios. Specifically, the existing camera deployment has the following problems: Discontinuous spatial coverage: Due to the lack of initial planning for full-coverage monitoring in some service areas, there are blind spots in monitoring and discontinuities in the vehicle entry and exit trajectories.

[0004] Inaccurate frame mapping: Although some areas are visually covered by the camera, factors such as imaging distortion, angle deviation, or excessive height make it difficult to accurately map the image content to real-world coordinates, affecting 3D reconstruction and multi-camera tracking.

[0005] Weakened tracking capabilities: In the absence of camera internal and external parameters, it is difficult to accurately match image frames with physical space, which directly restricts the ability to continuously track targets across viewpoints and devices. Summary of the Invention

[0006] One of the objectives of the present application is to provide a camera point filling method for global tracking that can solve at least one of the defects in the above-mentioned background technology.

[0007] To achieve at least one of the above objectives, the present application adopts a technical solution: a camera point filling method for global tracking, comprising the following steps: S100: constructing a three-dimensional model of the region and projecting it to generate a two-dimensional base map, dividing the two-dimensional base map into grids and marking them to obtain a grid coordinate system that identifies each grid area; S200: solving a homography matrix for estimating extrinsic parameters of existing cameras in the region based on a mapping relationship between collected data of existing cameras in the region and the three-dimensional model; S300: calculating an accurate perception area based on an existing camera in a grid coordinate system according to the obtained homography matrix, and combining grid areas that have a continuous relationship in the accurate perception area to form a closed perception area; S400: inferring the inaccurately covered area in the grid coordinate system based on the obtained closed sensing area; classifying the inaccurately covered area to obtain a plurality of blind areas to be monitored; S500: Calculate the average perception range of existing cameras, calculate the minimum number of camera points required for blind spot filling based on the horizontal and vertical spans of each blind spot filling area, and perform control.

[0008] Preferably, step S200 includes the following process: selecting a static image with an empty frame in each existing camera captured data; selecting image points of several representative feature points in the static image, and determining plane points corresponding to each representative feature point in the two-dimensional base map; when the coordinates of the image points and the coordinates of the plane points are known, calculating the matrix with the minimum sum of square errors between the image points and the plane points by the least squares method as the homography matrix H, the specific expression is as follows; ; in, represents the homogeneous coordinates of the image points, represents the homogeneous coordinates of a plane point, Represents the third component of the matrix multiplication result, and n represents the number of representative feature points selected.

[0009] Preferably, step S300 includes the following process: for each grid area corresponding to the perception area of ​​the existing camera, multiple groups of image points and corresponding three-dimensional points as verification points are selected from the static image and the three-dimensional model; the Euclidean distance error ε between the image points in each group of verification points after being transformed by the homography matrix H and the three-dimensional points is calculated. j ; ; If the Euclidean distance error ε of multiple groups of verification points corresponding to the grid area j are all less than the set threshold, the grid area is marked as the accurate perception area; where P j Represents the coordinates of the 3D point in the verification point.

[0010] Preferably, step S400 includes the following process: according to the usage functions of different locations in the area, the inaccurately covered areas are divided into areas that need to be accurately monitored and areas that do not need to be accurately monitored; the grid areas corresponding to the areas that need to be accurately monitored are identified; if continuous inaccurately covered grid areas are identified, the continuous inaccurately covered grid areas are aggregated to form blind areas to be filled.

[0011] Preferably, in step S500, the calculation of the average perception range of the existing cameras includes the following process: solving the projection matrix of each existing camera based on the mapping relationship between the acquisition data of the existing camera and the three-dimensional model; estimating the approximate position of the existing camera in the spatial coordinate system from the projection matrix to obtain the approximate spatial center point of the existing camera; calculating the maximum horizontal distance from the approximate spatial center point to the boundary of the grid area with the farthest perception range in the grid coordinate system as the horizontal effective perception radius of the current existing camera; and taking the average value of the horizontal effective perception radius of all existing cameras as the radius of the average perception range of the existing cameras.

[0012] Preferably, solving the projection matrix includes the following process: selecting N groups of corresponding image points and three-dimensional points from the collected data and the three-dimensional model, and constructing the expression of the projection matrix V by direct linear transformation method as follows: ; By minimizing the projection error: ; Use SVD decomposition method to fit multiple groups of points and solve the projection matrix ; Among them, (u i , v i ) represents the coordinates of the image point in the data collected by the existing camera, (X j , Y j , Z j ) represents the 3D point coordinates of the corresponding image point in the 3D model, s represents the scale factor, 、 as well as Represents the projection matrix V and the homogeneous coordinates of the three-dimensional point respectively The first, second, and third components of the multiplication result, M represents the rotation and intrinsic coupling matrix, and p4 represents the translation information.

[0013] Preferably, the process of calculating and deploying the minimum number of camera points in step S500 is as follows: obtaining the horizontal span and vertical span of the blind spot area to be filled, and calculating the ratio N of the horizontal span and the average perception range of the existing cameras respectively. kw and N kh ; For N kw and N kh For blind areas where all the parameters are less than or equal to 1, single-point simulation control is used to fill the blind spots. kw and / or N kh For blind areas with a value greater than 1, multi-point coordinated control is used to fill the blind spots.

[0014] Preferably, the specific method for single-point simulation control is as follows: select each outer edge grid of the blind area to be filled as a candidate installation point; according to the theoretical effective perception range of the existing camera, screen out the points from the candidate installation points whose theoretical effective perception range can completely cover the blind area to be filled and have the largest overlapping area with the surrounding accurate perception area as the final control point.

[0015] Preferably, the theoretical effective perception range of the existing camera is calculated through internal parameters, installation height and pitch angle range. The acquisition of the pitch angle range of the existing camera includes the following process: after completing the calculation of the accurate perception area in step S300, extract all pixel point sets corresponding to the accurate perception area in the image captured by the existing camera; based on the resolution information of the existing camera, estimate the distribution position of the pixel point set in the image, obtain the proportion and angle range of the pixel point set in the image, and then infer the pitch angle range of the existing camera.

[0016] Preferably, the specific method for multi-point collaborative control is as follows: determine the main control direction based on the proportional relationship between the horizontal span and the vertical span of the blind spot area to be filled; starting from the boundaries at both ends of the main direction, simulate the combination of cameras to obtain multiple groups of candidate points for layout; obtain the coverage range of each group of candidate points for layout based on the theoretical effective perception range of existing cameras; select a group of candidate points that can completely cover the blind spot area to be filled and have the largest overlapping area with the surrounding accurate perception area as the final control point.

[0017] Compared with the prior art, the present invention has the following advantages: This application provides a mechanism that infers existing camera installation parameters and reconstructs the viewport mapping area. This allows the relationship between the camera's viewport and spatial range to be effectively restored, even in the absence of detailed external parameters, to determine the extent of blind spots. This mechanism eliminates the need to replace existing surveillance system hardware; instead, supplementary modeling and analysis are performed on the image side to complete the determination and completion of visual blind spots. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a schematic diagram of the overall working steps of this application. DETAILED DESCRIPTION

[0019] Below, the present application is further described in conjunction with specific implementation methods. It should be noted that, in the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" and the like are intended to mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms should not be understood as necessarily referring to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification.

[0020] In the description of this application, it should be noted that for directional words, such as the terms "center", "horizontal", "longitudinal", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and so on, indicating the orientation and position relationship are based on the orientation or position relationship shown in the accompanying drawings, which is only for the convenience of describing this application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and cannot be understood as limiting the specific scope of protection of this application.

[0021] It should be noted that the terms "first", "second", etc. in the description and claims of this application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0022] In this application, unless otherwise expressly specified or limited, terms such as "mounted," "connected," "connect," and "fixed" should be understood in a broad sense. For example, they may refer to connection, detachable connection, or integration; mechanical connection or electrical connection; direct connection or indirect connection through an intermediate medium; and internal communication between two components or interaction between two components. Those skilled in the art will understand the specific meanings of the above terms in this application based on specific circumstances.

[0023] In this application, unless otherwise expressly specified or limited, a first feature being "above" or "below" a second feature may include the first and second features being in direct contact, or may include the first and second features being in contact not directly but through another feature between them. Moreover, a first feature being "above," "above," and "above" a second feature may include the first feature being directly above or obliquely above the second feature, or may simply mean that the first feature is higher in level than the second feature. A first feature being "below," "below," and "below" a second feature may include the first feature being directly below or obliquely below the second feature, or may simply mean that the first feature is lower in level than the second feature.

[0024] The terms "comprises" and "having" and any variations thereof in the specification and claims of this application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus that includes a series of steps or units is not necessarily limited to those steps or units expressly listed, but may include other steps or units not expressly listed or inherent to such process, method, product or apparatus.

[0025] One of the preferred embodiments of this application is as follows: Figure 1 As shown in FIG, a camera point filling method for global tracking includes the following steps: S100: constructing a three-dimensional model of the region and performing projection to generate a two-dimensional base map, dividing the two-dimensional base map into grids and marking the grids to obtain a grid coordinate system for identifying each grid area.

[0026] S200: solving a homography matrix for estimating extrinsic parameters of existing cameras in the region based on a mapping relationship between collected data of existing cameras in the region and the three-dimensional model.

[0027] It is understandable that since most of the existing cameras in the area were deployed early, there are problems such as inconsistent models and offset installation positions. Their original intrinsic parameter matrix K and extrinsic parameter matrix R may not be accurately restored; therefore, an inverse method is needed to reconstruct the spatial extrinsic parameters of the existing cameras.

[0028] S300: Calculating an accurate perception area based on an existing camera in a grid coordinate system according to the obtained homography matrix, and combining grid areas with a continuous relationship in the accurate perception area to form a closed perception area.

[0029] It is understandable that after reverse engineering the extrinsic parameters of the existing camera, the area that the current camera can accurately identify can be obtained. There are three situations for the obtained accurate perception area: one is that the grid areas corresponding to the obtained accurate perception area are all continuous; another is that the grid areas corresponding to the obtained accurate perception area are partially continuous and partially discrete; and another is that the grid areas corresponding to the obtained accurate perception area are all discrete. For grid areas with discrete relationships, the vehicle is prone to track loss after passing through the area. Therefore, only grid areas with continuous relationships are combined to form a closed perception area, and grid areas with discrete relationships can be perceived and reinforced through subsequent point filling.

[0030] The continuity between grid regions can be determined by the distance between them. Specifically, if the distance between the centers of any two grid regions within the accurate perception area is less than a set distance threshold, then the two grid regions can be determined to be continuous. The specific value of the distance threshold is preferably between 5m and 15m, and can be determined based on the actual needs of those skilled in the art. For ease of understanding, a specific example will be used below to illustrate this.

[0031] Taking the service area as an example, the specific value of the distance threshold is 10m, the size of the grid area is 3m×3m, and the length of an ordinary car is generally 2.8m. Then when the target vehicle passes through two grid areas with a continuous relationship, the distance length that is not recognized by the camera is 4.8m. It can be assumed that the target vehicle travels at a speed of 15km / h in the service area, so the time the vehicle is not recognized by the camera is about 1.15s. Generally speaking, within the recognition range of a single camera, if the vehicle disappears for more than 1.5s, it will be judged that the vehicle track is lost; that is, in this example, when the target vehicle passes through the two grid areas with a continuous relationship mentioned above, its vehicle track can be recognized, and the track of the unrecognized area can be automatically completed by interpolation and the like. The specific track completion method is a well-known technology to those skilled in the art, so it will not be elaborated here.

[0032] S400: inferring the inaccurately covered area in the grid coordinate system based on the obtained closed sensing area; classifying the inaccurately covered area to obtain a plurality of blind areas to be monitored.

[0033] It's understandable that different locations within an area correspond to different functions. For example, a service area can be divided into parking areas, road areas, green areas, and built-up areas. Parking and road areas are areas that require accurate monitoring, while green areas and built-up areas are less so. Therefore, only parking and road areas need to be considered when calculating the blind areas to be filled.

[0034] S500: Calculate the average perception range of existing cameras, calculate the minimum number of camera points required for blind spot filling based on the horizontal and vertical spans of each blind spot filling area, and perform control.

[0035] It is understandable that, based on the above technical solution, this application provides a mechanism that infers the installation parameters of existing cameras and reconstructs the viewing angle mapping area. This allows, even in the absence of detailed external parameters of existing cameras, to effectively restore the relationship between existing cameras and the spatial range, thereby determining the blind spot range within the area. This mechanism eliminates the need to replace the existing monitoring system hardware, requiring only supplementary modeling and analysis on the image side to complete the determination and completion of visual blind spots. It is a very cost-effective system expansion solution.

[0036] By mapping the accurately perceived area identified in the image to the real-world BEV perspective, a precise connection can be established from the image plane to the spatial coordinate system. This process provides a reliable coordinate basis for subsequent camera positioning, radar connection, point cloud registration, and other processes, effectively improving the spatial consistency of multi-sensor collaborative perception and the accuracy of target completion. Based on the mapping between the pixel's accurately perceived area and its BEV spatial projection range, information extraction and matching are restricted to the accurately perceived area, avoiding erroneous associations introduced by inaccurate pixels and improving the accuracy of multi-source sensor data fusion.

[0037] In this embodiment, the construction of the three-dimensional model in step S100 can be achieved by combining photogrammetry technology with GNSS control point data measured on the spot; commonly used photogrammetry technologies include multi-view stereo reconstruction and SfM, etc., which can be selected according to the actual needs of those skilled in the art. Among them, the three-dimensional world coordinates (X w , Y w and Z w ) can be used to transform the original 3D model from the model coordinate system to the real geographic coordinate system of the region. The obtained 3D model can be projected using the BEV perspective to generate a 2D basemap.

[0038] For the grid division of the two-dimensional base map, the size of the unit grid can be selected according to the actual needs of technicians in this field. For example, the unit grid size can be 2m×2m or 3m×3m. The divided grids can be coded using a combination of letters and numbers. For example, the horizontal grids can be numbered according to the letters A to Z. If the number of horizontal grids exceeds the number of single letters, the letters can be repeated, such as AA or AB. The vertical grids can be numbered according to the numbers 1 to N, and finally a grid coordinate system with a unique identifier such as A1 and B3 is formed for each grid area.

[0039] In this embodiment, the specific process of reconstructing the existing camera external parameters by the inverse method in step S200 is as follows: select a static picture with an empty frame in the data collected by each existing camera; taking the service area as an example, the static picture is a picture with no vehicles or few vehicles blocking it. Select several image points of representative feature points in the static picture, and determine the plane points corresponding to each representative feature point in the two-dimensional base map; for the representative feature points, corner points or edge points can be selected. and plane point coordinates In the case of , the matrix with the minimum sum of square errors between image points and plane points is calculated by least squares method as the homography matrix H. The specific expression is as follows: .

[0040] It can be understood that the above expression means that among all possible matrices H, find a matrix that can minimize the sum of square errors between the projection point and the real image point. This optimal matrix is ​​the required homography matrix H. represents the homogeneous coordinates of the image points, represents the homogeneous coordinates of a plane point, Represents the third component of the matrix multiplication result, and n represents the number of representative feature points selected.

[0041] It should be noted that in order to ensure the stability and accuracy of the solved homography matrix H, multiple sets of new image points and corresponding three-dimensional points can be selected from the captured data of each existing camera to perform error evaluation on the solution of the homography matrix H. Specifically, the additionally selected image points can be mapped into real space using the solved homography matrix H to obtain the mapping point positions of the image points in the three-dimensional space coordinate system. The distance error between the obtained mapping point positions and the actual corresponding three-dimensional points is analyzed. If the error is less than the set threshold, it means that the accuracy of the solved homography matrix H meets the requirements. Otherwise, the solution of the homography matrix H is updated using more additional sets of image points and corresponding three-dimensional points.

[0042] In this embodiment, after obtaining a relatively accurate homography matrix H, the accurate perception range of the existing camera can be identified by the accuracy of the homography matrix H; for step S300, the specific calculation process of the accurate perception area corresponding to each existing camera is as follows: for the grid area corresponding to the perception area of ​​each existing camera, multiple groups of image points and corresponding three-dimensional points are selected from the static picture and the three-dimensional model as verification points; the Euclidean distance error ε between the image points in each group of verification points after being transformed by the homography matrix H and the three-dimensional points is calculated. j , Euclidean distance error ε j The expression is as follows: Among them, P j Represents the coordinates of the 3D point in the verification point.

[0043] According to the above expression, if the Euclidean distance error ε of multiple groups of verification points corresponding to the grid area is j If both values ​​are smaller than the set threshold, the grid area is marked as an accurate perception area.

[0044] To facilitate understanding, a detailed description will be given below using specific examples.

[0045] The sensing area of ​​an existing camera can be defined as A1 to A3, B1 to B3, and C1 to C3, meaning that the camera's sensing area covers nine grid areas. Multiple sets of image points and corresponding 3D points can then be selected from each grid area as verification points. The image points are derived from the camera's captured data, while the 3D points are derived from the 3D model. The number of verification point sets corresponding to each grid area can be selected based on the needs of those skilled in the art, for example, 20 or 30 sets can be selected.

[0046] For each set of verification point data, the image points can be converted from two-dimensional coordinate points to three-dimensional space coordinate points through the homography matrix H, and then the obtained three-dimensional space coordinate points are compared with the three-dimensional points in the verification points by the Euclidean distance error ε j For the calculation of Euclidean distance error ε j The specific value of the threshold can be determined according to the actual needs of those skilled in the art. For example, the threshold is 1m. Then when the Euclidean distance error ε corresponding to the verification points of all groups in the grid area is j When all the distances are less than 1m, the grid area can be considered as an accurate perception area; if there is a corresponding Euclidean distance error ε in multiple groups of verification points in the grid area j If the grid area is greater than or equal to 1m, the grid area can be marked as a conversion failure area, and the corresponding frame area is not included in the effective perception range.

[0047] After accurately identifying the perception ranges of all grid areas across multiple existing cameras in the area, a closed perception region can be derived based on the continuity of the grid areas. Specifically, convex hull analysis can be used to determine the continuity of the ametrines within the grid areas. The resulting closed perception region possesses the following properties: it continuously covers most of the accurate perception area; its shape and orientation reflect the primary perception direction of the current camera posture; and it can serve as a baseline for high-quality coverage areas in subsequent point-filling optimization.

[0048] Specifically, based on the obtained closed perception areas, the frame area of ​​each existing camera is labeled and classified in the grid coordinate system under the BEV perspective, ultimately dividing the entire grid coordinate system into two categories. One category consists of closed perception areas that can be accurately recognized by all existing cameras and have continuous grid areas. The other category includes accurate recognition areas that can be accurately recognized by all existing cameras but are in a discrete state, conversion failure areas corresponding to each existing camera, and areas not covered by all existing cameras. These areas can be marked as not accurately covered areas.

[0049] In this embodiment, as can be seen from the foregoing, when performing step S400, the inaccurately covered area can be divided into an area requiring accurate monitoring and an area not requiring accurate monitoring based on the usage functions of different locations in the area. The grid areas corresponding to the area requiring accurate monitoring are identified, and if continuous inaccurately covered grid areas are identified, the continuous inaccurately covered grid areas are aggregated to form a blind area to be filled.

[0050] It is understandable that for discrete grid areas that are not accurately covered in the area that needs to be accurately monitored, they are generally located near the closed perception area, that is, this part of the grid area belongs to the transition area of ​​the adjacent closed perception area, then the missing range of this part of the grid area is very small, and the time it takes for the vehicle to pass through is relatively short; that is, after the vehicle passes through the discrete grid area that is not accurately covered, it can be quickly captured by the surrounding cameras in time and the trajectory is completed, so there is no need to perform perception completion on this part of the grid area. To put it more simply, for a discrete single grid area that is not accurately covered, its area is very small relative to the perception range of the entire camera, and the area accounts for generally less than 1%. When the surrounding cameras can achieve rapid trajectory recognition and completion, setting up another camera for perception coverage is extremely cost-effective.

[0051] It should be noted that when filling in camera points, the camera used for filling in the points can be a new model camera or a spare camera of the same model as the original camera. For the new model camera used for filling in the points, its internal and external parameters are clear, and its perception coverage range can be accurately known; however, due to the difference in model, the new model camera and the original camera in the area may cause differences in internal parameters, imaging, time alignment, etc., thereby affecting the subsequent data fusion process of the regional monitoring system. Using a spare camera of the same model as the original camera to fill in the points can ensure the stability of the data fusion of the subsequent regional monitoring system; at the same time, based on the above content, the effective perception range of the existing cameras in the area can be inferred. Therefore, in this embodiment, it is preferred to use a spare camera of the same model as the original camera to fill in the points.

[0052] In this embodiment, the calculation of the average perception range of the existing cameras during step S500 includes the following process: Based on the mapping relationship between the existing camera's captured data and the three-dimensional model, the projection matrix of each existing camera is solved. The approximate position of the existing camera in the spatial coordinate system is estimated from the projection matrix to obtain the approximate spatial center point of the existing camera. The maximum horizontal distance from the approximate spatial center point to the boundary of the grid area with the farthest perception range in the grid coordinate system is calculated as the horizontal effective perception radius of the current existing camera. The average of the horizontal effective perception radiuses of all existing cameras is used as the radius of the average perception range of the existing cameras.

[0053] Specifically, solving the projection matrix includes the following process: Based on the correspondence between image points and three-dimensional points in the world coordinate system, the expression of the projection matrix V constructed by the direct linear transformation method (DLT) is as follows: .

[0054] Among them, (u i , v i ) represents the coordinates of the image point in the data collected by the existing camera, (X j , Y j , Z j ) represents the 3D point coordinates of the corresponding image point in the 3D model, and s represents the scale factor.

[0055] By minimizing the projection error: .

[0056] Use SVD decomposition method to fit multiple groups of points and solve the projection matrix .

[0057] in, 、 as well as Represents the projection matrix V and the homogeneous coordinates of the three-dimensional point respectively the first, second, and third components of the multiplication result, represents the rotation and intrinsic parameter coupling matrix, Represents translation information, and R represents the rotation matrix of the external parameter.

[0058] Specifically, after solving the projection matrix, the approximate position of the existing camera in the world coordinate system of the 3D model can be estimated based on the solved projection matrix V, that is, the center point O of the existing camera. It should be noted that due to the influence of image point selection errors, internal parameter loss, etc., the center point O is only an approximate estimate, but it can still be used as the basic point for subsequent spatial analysis.

[0059] After obtaining the approximate spatial center point O of all existing cameras, the geometric coverage capability of the existing cameras can be further analyzed based on the marked accurate perception area, that is, the current perception range of the existing cameras. The boundary point of the grid area farthest from the center point O in the accurate perception area of ​​the existing camera can be set as G j , the point G j The three-dimensional coordinates in the three-dimensional model are (X j , Y j , Z j ); then the point G j To the existing camera center point O(X o , Y o , Z o )'s Euclidean distance D j for: .

[0060] In order to simplify the analysis of the perception range of existing cameras, we can directly calculate the grid coordinate system under the BEV perspective, then the distance from the center point O to G j The maximum horizontal distance R max It can be regarded as the horizontal effective perception radius of the current camera.

[0061] .

[0062] It should be noted that due to the different installation heights of existing cameras and the different pitch angles of the current deployment, the effective perception ranges of the existing cameras are different. Therefore, when simulating the addition of points, the average value of the effective perception ranges of all existing cameras can be used as the standard value for simulation.

[0063] In this embodiment, the specific process of calculating and deploying the minimum number of camera points in step S500 is as follows: obtaining the horizontal span and vertical span of the blind spot area to be filled, and calculating the ratio N of each of them to the average perception range of the existing cameras. kw and N kh ; For N kw and N kh For blind areas where all the parameters are less than or equal to 1, single-point simulation control is used to fill the blind spots. kw and / or N kh For blind areas with a value greater than 1, multi-point coordinated control is used to fill the blind spots.

[0064] It is understandable that for N kw and N kh For blind areas where N is less than or equal to 1, the effective sensing range of a single camera can achieve full coverage, so only one additional camera is needed. kw and / or N kh For blind spots greater than 1, the effective sensing range of a single camera cannot fully cover them, so multiple camera deployment points are required. To maximize the monitoring system's tracking capabilities within the area, when deploying additional cameras, whether using single-point simulation or multi-point coordinated control, while ensuring full coverage of the blind spot, other areas should also be covered as much as possible.

[0065] Specifically, the specific method for single-point simulation control is as follows: select each outer edge grid of the blind area to be filled as a candidate installation point; according to the theoretical effective perception range of the existing camera, screen out the candidate installation points whose theoretical effective perception range can completely cover the blind area to be filled and have the largest overlapping area with the surrounding accurate perception area as the final control point.

[0066] The specific method for multi-point coordinated control is as follows: determine the main control direction based on the proportional relationship between the horizontal span and the vertical span of the blind spot area to be filled; starting from the boundaries at both ends of the main direction, simulate the combination of cameras to obtain multiple sets of candidate points for deployment; obtain the coverage range of each set of candidate points based on the theoretical effective perception range of existing cameras; select a set of candidate points that can completely cover the blind spot area to be filled and have the largest overlapping area with the surrounding accurate perception area as the final control point.

[0067] It is understandable that the perception range of the existing camera calculated in the above content is based on the current fixed pitch angle. However, the pitch angle of the camera is a range value, and when the camera is supplemented, its pitch angle can be adjusted arbitrarily within the pitch angle range according to actual needs. Therefore, before supplementing the camera points, it is necessary to calculate the theoretical effective perception range based on the internal parameters, installation height, and pitch angle range of the existing camera. Among them, since the camera model of the supplementary point is the same as the original existing camera, the internal parameters have not changed, and the installation height can be set according to actual needs. The calculation of the pitch angle range can be obtained through the data collected by the existing camera. For ease of understanding, a detailed description will be given below.

[0068] Specifically, obtaining the pitch angle range of an existing camera includes the following process: after completing the calculation of the accurate perception area in step S300, extracting all pixel point sets corresponding to the accurate perception area in the image captured by the existing camera; based on the resolution information of the existing camera, estimating the distribution position of the pixel point set in the image, obtaining the proportion and angle range of the pixel point set in the image, and then inferring the pitch angle range of the existing camera.

[0069] In layman's terms, if the pixels corresponding to the accurate perception area in the existing camera's captured data are concentrated in the lower half of the image, it indicates that the existing camera's pitch angle is large and can be considered the upper limit of the pitch angle range. The lower limit of the pitch angle range can then be determined, and the entire pitch angle range of the existing camera can be determined accordingly. When patching points, the installation height can be selected based on the actual patch point location. The theoretical effective perception range of the camera can then be determined based on the theoretical effective perception range that covers the blind area to be patched and maximizes overlap with the surrounding accurate perception area.

[0070] The above describes the basic principles, main features, and advantages of the present application. Those skilled in the art should understand that the present application is not limited to the above-described embodiments. The above-described embodiments and the specification merely illustrate the principles of the present application. Various changes and improvements may be made to the present application without departing from the spirit and scope of the present application. These changes and improvements fall within the scope of the present application for which protection is sought. The scope of protection claimed by the present application is defined by the appended claims and their equivalents.

Claims

1. A camera point filling method for global tracking, characterized in that: The steps include: S100: constructing a three-dimensional model of the region and projecting it to generate a two-dimensional base map, dividing the two-dimensional base map into grids and marking them to obtain a grid coordinate system that identifies each grid area; S200: solving a homography matrix for estimating extrinsic parameters of existing cameras in the region based on a mapping relationship between collected data of existing cameras in the region and the three-dimensional model; S3 00: Calculate the accurate perception area based on the existing camera in the grid coordinate system according to the obtained homography matrix, and combine the grid areas with continuous relationships in the accurate perception area to form a closed perception area; S400: inferring the inaccurately covered area in the grid coordinate system based on the obtained closed sensing area; classifying the inaccurately covered area to obtain a plurality of blind areas to be monitored; S500: Calculate the average perception range of existing cameras, calculate the minimum number of camera points required for blind spot filling based on the horizontal and vertical spans of each blind spot filling area, and perform control.

2. The camera point filling method for global tracking according to claim 1, wherein: Step S200 includes the following process: Select a static frame with an empty space in the data collected by each existing camera; Selecting a number of image points representing characteristic points in a static image, and determining a plane point corresponding to each representative characteristic point in a two-dimensional base map; When the coordinates of the image points and the plane points are known, the matrix with the minimum sum of square errors between the image points and the plane points is calculated by the least squares method as the homography matrix H. The specific expression is as follows: ； in, represents the homogeneous coordinates of the image points, represents the homogeneous coordinates of a plane point, Represents the result of matrix multiplication The third component, n, represents the number of representative feature points selected.

3. The camera point filling method for global tracking according to claim 2, wherein: Step S300 includes the following process: For each grid area corresponding to the perception area of ​​an existing camera, multiple groups of image points and corresponding 3D points are selected from the static image and the 3D model as verification points; Calculate the Euclidean distance error ε between the image point in each set of verification points after transformation by the homography matrix H and the three-dimensional point j ; ； Among them, P j Represents the coordinates of the three-dimensional point in the verification point; If the Euclidean distance error ε of multiple groups of verification points corresponding to the grid area j If both values ​​are smaller than the set threshold, the grid area is marked as an accurate perception area.

4. The camera point filling method for global tracking according to claim 1, wherein: Step S400 includes the following process: According to the usage functions of different locations in the area, the areas that are not accurately covered are divided into areas that need to be accurately monitored and areas that do not need to be accurately monitored; The grid areas corresponding to the areas that need to be accurately monitored are identified. If continuous grid areas that are not accurately covered are identified, the continuous grid areas that are not accurately covered are aggregated to form blind areas to be filled.

5. The camera point filling method for global tracking according to claim 1, wherein: In step S500, the calculation of the average perception range of the existing camera includes the following process: According to the mapping relationship between the data collected by the existing cameras and the 3D model, the projection matrix of each existing camera is solved; Estimate the approximate position of the existing camera in the spatial coordinate system from the projection matrix to obtain the approximate spatial center point of the existing camera; Calculate the maximum horizontal distance from the center point of the approximate space to the boundary of the grid area with the farthest perception range in the grid coordinate system as the horizontal effective perception radius of the current existing camera; The average value of the horizontal effective perception radius of all existing cameras is taken as the radius of the average perception range of the existing cameras.

6. The camera point filling method for global tracking according to claim 5, wherein: Solving the projection matrix includes the following process: Select N groups of corresponding image points and 3D points from the collected data and 3D model, and construct the projection matrix V by direct linear transformation as follows: ； By minimizing the projection error: ； Use SVD decomposition method to fit multiple groups of points and solve the projection matrix ; Among them, (u i , v i ) represents the coordinates of the image point in the data collected by the existing camera, (X j , Y j , Z j ) represents the 3D point coordinates of the corresponding image point in the 3D model, s represents the scale factor, 、 as well as Represents the projection matrix V and the homogeneous coordinates of the three-dimensional point respectively The first, second, and third components of the multiplication result, M represents the rotation and intrinsic coupling matrix, and p4 represents the translation information.

7. The camera point filling method for global tracking according to any one of claims 1 to 6, wherein: The process of calculating and deploying the minimum number of camera points in step S500 is as follows: Get the horizontal and vertical spans of the blind area to be filled, and calculate the ratio N of each to the average perception range of the existing cameras. kw and N kh ; For satisfying N kw and N kh For blind areas where the values ​​are all less than or equal to 1, single-point simulation control is used to fill the blind spots. For satisfying N kw and / or N kh For blind areas with a value greater than 1, multi-point coordinated control is used to fill the blind spots.

8. The camera point filling method for global tracking according to claim 7, wherein: The specific methods for single-point simulation control are as follows: Select each peripheral edge grid of the blind area to be filled as a candidate installation point; Based on the theoretical effective perception range of existing cameras, the candidate installation points are screened out, and the points whose theoretical effective perception range can completely cover the blind area to be filled and have the largest overlap area with the surrounding accurate perception area are selected as the final control points.

9. The camera point filling method for global tracking according to claim 8, wherein: The theoretical effective sensing range of an existing camera is calculated using internal parameters, installation height, and pitch angle range. The process of obtaining the pitch angle range is as follows: After the calculation of the accurate perception area in step S300 is completed, a set of pixels corresponding to the accurate perception area is extracted from the image captured by the existing camera; Based on the resolution information of the existing camera, the distribution position of the pixel point set in the image is estimated, the proportion and angle range of the pixel point set in the image are obtained, and then the pitch angle range of the existing camera is calculated.

10. The camera point filling method for global tracking according to claim 7, wherein: The specific methods for multi-point coordinated control are as follows: Determine the main control direction based on the proportional relationship between the horizontal span and the vertical span of the blind spot area to be filled; Starting from the boundaries at both ends of the main direction, simulate the camera combination layout to obtain multiple sets of layout candidate points; According to the theoretical effective sensing range of existing cameras, the coverage range of each set of candidate points is obtained; A set of candidate points that can completely cover the blind area to be filled and have the largest overlapping area with the surrounding accurate perception area are selected as the final control points.

Images