A video monitoring camera preset position determination method

Abstract

The application discloses a kind of video monitoring camera preset position determination method.Cameras preset position usually needs artificial setting one by one, time-consuming and laborious.The method of the present application is used in video monitoring system, using a set of scene images and the image of target to be monitored realizes camera preset position automation setting, so as to greatly improve efficiency.The present application adopts computer vision technology, obtains feature points based on scene image set, generates the three-dimensional model of scene by using motion recovery structure technology, surface reconstruction algorithm, matches the feature points of target to be monitored image with scene image set feature points, obtains the position of target to be monitored in three-dimensional space, so as to select appropriate camera, and calculate corresponding preset position, finally corrects preset position according to actual monitoring effect.

Description

Technical Field

[0001] The present invention relates to the technical field of video surveillance, and in particular to a method for determining a preset position of a video surveillance camera. Background Art

[0002] With the rapid development of the national economy and the growing demand for electricity, higher safety requirements for power equipment are being placed on the market. Consequently, a more reliable substation video surveillance system is needed. Due to the complex environments within substations and the large number of different types of equipment in operation, the optimal setting of video surveillance camera presets directly determines the impact of video surveillance technology on power grid operations. Other application scenarios also involve the issue of video surveillance camera preset settings, such as oil depots, warehouses for flammable and explosive hazardous materials, and industrial plants.

[0003] Currently, the determination of video surveillance camera preset positions relies primarily on manual labor, resulting in high workload and low efficiency. Especially in the context of using drones and other means to capture scene images and use them to assist in the design of video surveillance systems, automatically determining camera preset positions based on scene images and the monitored objects would greatly improve work efficiency. Therefore, researching a method for determining video surveillance camera preset positions is particularly important. Summary of the Invention

[0004] Based on the above background, the present invention proposes a method for determining the preset position of a video surveillance camera. This method uses a set of scene images and a set of images of the target to be monitored to automatically set the camera preset position, thereby significantly improving efficiency. The specific technical solutions adopted are as follows:

[0005] A method for determining a preset position of a video surveillance camera comprises the following steps:

[0006] Using a camera to capture a set of scene images;

[0007] Calculating feature points and feature description vectors of each scene image in the scene image set;

[0008] Based on the scene image set and its feature points, the motion recovery structure technology is used to obtain the camera posture parameters corresponding to the scene image and the three-dimensional coordinates of the scene image feature points;

[0009] According to the scene image set, camera intrinsic parameter matrix and camera pose parameters, the scene three-dimensional space is represented as a set of radiation fields;

[0010] A truncated signed distance field is created from the radiation field based on a signed surface approximation network, and a surface reconstruction algorithm is used to generate a three-dimensional model.

[0011] Use a camera to shoot the target to be monitored in the scene, and calculate the feature points and feature description vectors of the target image to be monitored;

[0012] An image feature point matching method is used to find scene image feature points that match the target image to be monitored;

[0013] Using the scene image feature points that match the target image to be monitored, the position of the target to be monitored in the 3D model is obtained according to the 3D coordinates of the scene image feature points, and the monitoring target area and its normal direction are calculated;

[0014] Calculate the azimuth angle and distance of the camera in the normal direction area based on the position of the camera in three-dimensional space, the monitoring target area and its normal direction, select the camera that meets the preset conditions, calculate the target's orientation in the camera coordinates and the focal length of the lens, and set it as the preset position;

[0015] Capture an image at a preset camera position, obtain the image offset through image matching, and correct the preset position according to the camera parameters.

[0016] Furthermore, when a camera is used to capture a set of scene images, the Zhang Zhengyou calibration method is used to determine the camera's intrinsic parameter matrix.

[0017] Furthermore, the Superpoint method is used to calculate the feature points and feature description vectors of each scene image in the scene image set, as well as the feature points and feature description vectors of the target image to be monitored.

[0018] Furthermore, the method of using structure-from-motion technology to obtain camera pose parameters corresponding to a scene image and three-dimensional coordinates of feature points of the scene image includes:

[0019] Use scene image feature points and feature description vectors to perform two-to-two matching between images to find image pairs with overlapping scenes;

[0020] Calculate the fundamental matrix between the image pairs and determine whether the image pairs are related based on whether a preset number of matching feature points satisfy the mapping relationship;

[0021] Calculate the intrinsic matrix by combining the camera’s intrinsic matrix between the relevant image pairs;

[0022] Decomposing the essential matrix by singular value decomposition to obtain the extrinsic parameter matrix of the camera, and obtaining the camera posture corresponding to the scene image;

[0023] Based on the acquired camera pose and matched feature points, triangulation is used to solve the three-dimensional coordinates of the scene image feature points in the three-dimensional scene;

[0024] The bundle adjustment method is used to simultaneously optimize the camera pose and the 3D coordinates of the scene image feature points.

[0025] Further, the method of representing a scene three-dimensional space as a set of radiance fields according to a set of scene images, camera intrinsic matrix and camera poses comprises:

[0026] The neural radiance field model is constructed based on a multi-layer perceptron neural network oriented to three-dimensional implicit space modeling.

[0027] A spatial point in the scene three-dimensional space is represented as a 5D vector, including a 3D coordinate position (x, y, z) of the spatial point and a viewing direction As the input of the neural radiance field model, the neural radiance field model outputs the density and the viewing direction corresponding to the 3D coordinate position (x, y, z) The color related to the 3D coordinate position (x, y, z);

[0028] According to the scene image set, the camera intrinsic matrix and the camera pose, the neural radiance field model is rendered in a volume rendering manner from the viewing angle determined by the camera pose, the generated color and density value are rendered into an image, and the parameters of the neural network are optimized based on the error between the rendered image and the real scene image in the scene image set.

[0029] The above training is performed using all images in the scene image set, the error between the rendered image and the scene image in the scene image set is minimized, and a trained neural radiance field model is obtained.

[0030] The scene three-dimensional space is represented as a set of learnable and continuous radiance fields based on the trained neural radiance field model.

[0031] Further, the method of creating a truncated signed distance field from a radiance field based on a signed surface approximation network comprises:

[0032] The signed surface approximation network is constructed based on a multi-layer perceptron.

[0033] The signed surface approximation network is trained based on the occupied space information along the ray aggregation of the radiance field and the depth map obtained by rendering.

[0034] The trained signed surface approximation network is used to obtain a truncated signed distance function, and a truncated signed distance field is created based on the signed distance function.

[0035] The truncated signed distance function determines the distance from a point to the boundary of a region in a limited region in space, and defines the sign of the distance at the same time. The point is positive inside the boundary of the region, negative outside the boundary of the region, and 0 on the boundary.

[0036] Further, the surface reconstruction algorithm adopts a Marching cubes algorithm, which is a method of creating a polygonal surface of an isosurface of a three-dimensional scalar field.

[0037] Furthermore, the method of finding scene image feature points that match the target image to be monitored by using the image feature point matching method includes:

[0038] Constructing a feature matching network, the feature matching network includes sequentially connected attention GNN network units and an optimal matching layer, and using manually annotated real matches as supervision, and training the network by minimizing the negative log-likelihood loss function to obtain a trained network;

[0039] The feature points of the target image to be monitored and its feature description vector, and the feature points of each scene image in the scene image set and its feature description vector are input into the feature matching network to obtain the scene image feature points that match the target image to be monitored.

[0040] Furthermore, the calculating the orientation of the target in the camera coordinates and the focal length of the lens and setting them as preset positions includes:

[0041] Determine the position of the target in the camera coordinates based on the positions of the target and the camera in the 3D model and the installation method of the camera;

[0042] Calculate the target's position in camera coordinates and its distance from the camera;

[0043] The focal length of the camera lens is determined based on the distance between the target and the camera and the pinhole imaging principle, so that the target is clearly imaged on the camera imaging surface;

[0044] The orientation of the target in the camera coordinates and the focal length of the camera lens are set as a preset position of the camera.

[0045] Furthermore, the method of obtaining the image offset by image matching and correcting the preset position according to the camera parameters includes:

[0046] Match the image captured at each preset position with the image of the target to be monitored, and calculate the angle at which the center of the target to be monitored in the captured image deviates from the optical axis of the camera;

[0047] Correct the preset position so that the center of the target to be monitored is close to the optical axis of the camera;

[0048] The camera focal length is scaled according to the size of the monitored target on the image at the current camera focal length.

[0049] The beneficial effects of the present invention are as follows:

[0050] The application adopts computer vision technology, obtains feature points based on a scene image set, generates a three-dimensional model of the scene by adopting motion recovery structure technology and surface reconstruction algorithm, obtains the position of the target to be monitored in three-dimensional space through matching the feature points of the target image to be monitored with the feature points of the scene image set, selects a suitable camera, calculates the corresponding preset position, and finally corrects the preset position according to the actual monitoring effect. Through the above automatic preset point determination process, the preset points of each camera can be calculated in advance, thereby greatly reducing the time of the preset point setting process, reducing the labor input, and effectively accelerating the project progress. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 A flowchart of a video monitoring camera preset position determination method provided by the embodiment of the application is shown.

[0052] Figure 2 A flowchart of an incremental motion recovery structure technology shown in the embodiment of the application is shown. DETAILED DESCRIPTION

[0053] To make the objectives, technical solutions and advantages of the embodiments of the application clearer, the technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are part of the embodiments of the application, rather than all the embodiments of the application. The components of the embodiments of the application described and shown in the drawings can be arranged and designed in various different configurations.

[0054] Therefore, the following detailed description of the embodiments of the application provided in the drawings is not intended to limit the scope of the claimed application, but only represents selected embodiments of the application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the application without creative labor are within the scope of protection of the application.

[0055] The application can be used for camera layout and preset position determination in scenarios such as substations and industrial plants. Without loss of generality, the embodiments of the application assume that the application scenario is a substation.

[0056] Reference Figure 1 The embodiments of the application show a video monitoring camera preset position determination method in a specified scenario, including the following steps:

[0057] S1: A camera is used to shoot a scene image set.

[0058] Specifically, an image set of the scene is captured by a camera, the images in the image set of the scene should cover the entire scene, the scene is captured from different angles by using the same camera without changing the internal parameters, and the capturing is completed in an environment where the light is bright and the light condition changes not drastically. The internal parameters can be expressed in the form of an internal matrix. Of course, if different cameras are used for capturing or the internal parameters change during the capturing, the internal parameter matrix at the time of capturing needs to be determined. It should be noted that the camera is only used to capture the images of the scene and is not directly related to the camera to be arranged.

[0059] The camera internal parameter matrix can be determined by using the parameters when the camera is capturing, such as capturing resolution, sensor target surface size, focal length, etc., but the accuracy is usually not high. Therefore, the internal parameter matrix of the camera is generally obtained by camera calibration. The traditional camera calibration method can be used to calibrate by using a calibration object, such as Tsai two-step method, Zhang Zhengyou calibration method, etc., or the active vision camera calibration method can be used to calibrate by actively controlling the camera to make specific movements. These are all prior art.

[0060] In this embodiment, the Zhang Zhengyou calibration method is used, a calibration board composed of a two-dimensional chessboard is used for calibration, and 10-20 pictures of different poses of the calibration board are collected, which can be realized by moving the camera or by moving the calibration board. The SIFT algorithm is used to extract the corner points in the calibration board picture, that is, the black and white chessboard intersection points. The internal parameter matrix K of the camera is obtained by calibration.

[0061] S2: Calculate the feature points and feature description vectors of each scene image in the image set of the scene.

[0062] Specifically, the commonly used algorithms such as Harris, FAST, SURF, and SIFT can be used to calculate the image feature points, and the feature points and corresponding descriptors of the scene image are obtained.

[0063] In this embodiment, the Superpoint (see arXiv.1712.07629.Superpoint: Self-Supervised Interest Point Detection and Description) algorithm is used to calculate the feature points of each scene image in the image set of the scene. The algorithm can stably and repeatedly detect rich feature points under different light conditions and viewpoints, and calculate descriptors for these points for image matching and other advanced tasks. The scene image is input, and Superpoint calculates the pixel-level interest point position and related descriptors in a single forward propagation by using a fully convolutional neural network.

[0064] First, the network uses a shared VGG-style encoder to reduce the dimensionality of the input image, then processes the image through two decoder heads, one for keypoint detection and the other for descriptor computation. The keypoint decoder computes a "pointness" probability for each pixel, while the descriptor decoder outputs L2-normalized fixed-length descriptors. This process involves self-supervised training, which includes a synthetic data pre-training and a homography adaptation step. Finally, Superpoint outputs the feature points and corresponding feature descriptor vectors for the scene image.

[0065] S3: Based on the set of scene images and their feature points, the camera pose parameters corresponding to the scene images and the three-dimensional coordinates of the scene image feature points are obtained by using the motion recovery structure technology.

[0066] Specifically, the motion recovery structure technology (SFM, Structure From Motion) is an algorithm for estimating three-dimensional structure from a sequence of multiple two-dimensional image sequences containing visual motion information. The camera pose can be recovered from multiple scene images, and a sparse point cloud structure can be reconstructed. The motion recovery structure technology can be global or incremental. The incremental method first estimates the camera pose and reconstructs the three-dimensional coordinate points based on a set of two images of the scene. After this initial process, new scene images are added for calculation, and the above steps are repeated until all images are processed. Global motion recovery structure adds all images to the reconstruction queue at once, and matches all image feature points at once. The incremental method is high in accuracy and low in efficiency, while the global method is high in efficiency and low in accuracy.

[0067] The embodiment adopts the incremental motion recovery structure technology, as shown in Figure 2 The specific processing flow is as follows:

[0068] S301: Image-to-image two-to-two matching is performed using image feature points and feature descriptor vectors to find image pairs with scene overlap.

[0069] The image feature points and feature descriptor vectors extracted in step S2 are used to judge the similarity of the feature points of the two images using the Euclidean distance, and the KD tree algorithm is used for feature point matching.

[0070] S302: The fundamental matrix between the image pair is calculated to determine whether there are enough feature matching points to satisfy the mapping relationship, so as to determine whether the image pair is related.

[0071] Specifically, based on computer vision theory, the constraint relationship between two images is expressed algebraically as the fundamental matrix F. Algorithms such as Random Sample Consensus (RANSAC) are used to estimate the fundamental matrix between the image pairs. While calculating the fundamental matrix, the RANSAC algorithm also determines whether there are sufficient feature matching points to satisfy the mapping relationship. For example, if there are more than eight pairs of feature matching points, the image pairs have scene overlap and are considered correlated.

[0072] S303: Calculate the intrinsic matrix E=K between the relevant image pairs and the camera's intrinsic parameter matrix -T FK, where K is the camera intrinsic matrix, E is the essential matrix, and F is the fundamental matrix.

[0073] S304: The intrinsic matrix is ​​decomposed into the camera's extrinsic parameter matrix by singular value decomposition to obtain the camera pose corresponding to the image.

[0074] Specifically, by decomposing the essential matrix E, E = t ^ R, obtain the camera's rotation matrix R and translation matrix t, that is, obtain the camera's external parameter matrix and obtain the camera pose corresponding to the image.

[0075] S305: According to the camera pose and the matched feature points, the three-dimensional coordinates of the feature points in the three-dimensional scene are solved by triangulation.

[0076] Specifically, using matching feature points from two or more camera perspectives, whose camera poses are known, in triangulation, the three-dimensional point p is mapped to the point x on the plane through the projection equation i , for each view j, the projection equation is expressed as x j =P j p, where P j is the projection matrix of the jth camera, which contains the combination of the extrinsic matrix and the camera posture parameters, specifically expressed as P j =K j [R j |t j ]; point x on the image plane i Expressed as homogeneous coordinates and normalized to make them directly related to the camera coordinate system; the projection equations of the 3D points of each view to the points on the image plane are solved using the least squares method to minimize the reprojection error of all projection points. Obtain the best three-dimensional coordinate estimate of point p, and then calculate the coordinates of the matching feature points in three-dimensional space.

[0077] S306: Use bundle adjustment to simultaneously optimize the camera posture and the three-dimensional coordinates of the feature points.

[0078] Specifically, nonlinear optimization iteration is performed by minimizing the reprojection error. The objective function is the sum of the distances between all matched feature points and their projection points. The optimization variables include the camera projection matrix P. j And the three-dimensional point space coordinates, find a set of optimal projection matrices and three-dimensional coordinates of the space points, and obtain the camera posture and three-dimensional coordinates of the image feature points corresponding to the final optimized image.

[0079] S4: The scene 3D space is represented as a set of radiation fields according to the scene image set, the camera intrinsic parameter matrix and the camera pose parameters.

[0080] Specifically, based on the scene image set, the camera intrinsic parameter matrix and the camera pose parameters, a multi-layer perceptron neural network for 3D implicit space modeling is used to construct the neural radiation field. The network input is a 5D vector, including the 3D coordinate position x = (x, y, z) of a spatial point and the viewing direction. The output is the density of the corresponding 3D position and the voxel density σ and color value c = (r, g, b) related to the view angle. This neural network can be written as: F Θ :(x,d)→(c,σ). In the specific implementation, the vector x is first input into the MLP network, which outputs σ and intermediate features. The intermediate features and d are then input into an additional fully connected layer to predict color. A batch of camera rays are randomly sampled from the scene images in the scene image set. A camera ray originates from the camera origin and passes through each pixel in the corresponding scene image. Volume rendering is used to render the color and density information in 3D space onto a 2D image. Hierarchical volume rendering is used to improve rendering efficiency. First, the rays are uniformly sampled to obtain a weight for each sampling point. The weight is determined by the volume density and the proximity of the location. The obtained weights are normalized on the rays, and the sum of the weights is set to 1 to obtain a probability density function. Based on the probability density function, fine sampling is performed, that is, more sampling points are allocated to more important segments. Based on the voxel density σ and color value c at the 3D locations of all sampling points along the camera ray, the color value of the 2D pixel corresponding to the camera ray is calculated to obtain a rendered image. The error between the rendered image and the real scene image is calculated. The loss function is minimized using optimization algorithms such as gradient descent to train the neural network and obtain a neural radiation field model. Finally, the trained model is used to represent the three-dimensional space of the scene as a set of continuous radiation fields.

[0081] S5: Create a truncated signed distance field from the radiation field based on a signed surface approximation network and generate a 3D model using a surface reconstruction algorithm.

[0082] The voxel density σ of a 3D point in the radiation field represents its density or absorptivity, so it can be used to determine the surface of the object and expressed using a signed distance field to generate a 3D model. This embodiment uses the following method to generate the 3D model:

[0083] S51. First create a truncated signed distance field.

[0084] The 3D coordinates are fed into the Signed Surface Approximation Network (SSAN) module, which takes the 3D coordinates as input and outputs three main pieces of information: a truncated signed distance function (TSDF), a normal vector, and an appearance feature vector, resulting in a truncated signed distance field representation of the scene. The SSAN module is divided into two independent branches: a geometry branch that outputs a TSDF approximation and normal predictions, and an appearance branch that outputs color features.

[0085] The truncated signed distance field is represented by a truncated signed distance function, which determines the distance from a point to the boundary of a finite region in space and defines the sign of the distance: positive if the point is inside the region boundary, negative if it is outside, and zero if it is on the boundary. The normal vector is a unit vector that represents the direction of the surface normal at the input point and is used for rendering and smoothing regularization. The appearance feature vector is an 8-dimensional vector containing color and material information for the appearance network.

[0086] The SSAN network consists of a multi-layer perceptron and is trained based on the radiance field obtained in step S4. The radiance field model aggregates spatial occupancy information along rays and the rendered depth map to guide SSAN training. Training involves three loss functions: a signed distance loss for interior and exterior points, a normal regularization loss, and an appearance loss. The signed distance loss for interior and exterior points ensures that the network correctly predicts the signed distances between interior and exterior points. The normal regularization loss ensures that the learned TSDF function has smooth normals near surfaces. The appearance loss optimizes appearance features by comparing predicted colors with those in the actual training images. The SSAN network weights are randomly initialized, and the loss is calculated based on the predicted results and the real data. The gradient of the loss function is backpropagated to update the SSAN network weights until training is complete, resulting in a TSDF representing the scene and a truncated signed distance field representation of the scene.

[0087] S52. Then, a surface reconstruction algorithm is used to generate a three-dimensional model from the signed distance field.

[0088] The surface reconstruction method can adopt common methods such as Poisson surface reconstruction method and surface reconstruction based on Delaunay triangulation. This embodiment uses Marching cubes algorithm, a method for creating polygonal surfaces of three-dimensional scalar field isosurfaces, to perform surface reconstruction:

[0089] After the TSDF of the expression scene is preprocessed, the TSDF is read into a specific array or octree; a hexahedron is extracted from the grid data body, and all information of the hexahedron is obtained, for example, values of 8 vertices, coordinate positions, and the like; function values of 8 vertices of the current hexahedron are compared with a given isosurface value, to obtain a state table of the hexahedron; according to an index of the state table of the current hexahedron, a hexahedron edge intersecting the isosurface is found, and a linear interpolation method is used to calculate position coordinates of each intersection point; a central difference method is used to obtain normal vectors of 8 vertices of the current hexahedron, and a linear interpolation method is used to obtain normals of each vertex of a triangular facet; according to coordinates of each vertex of the triangular facet and the vertex normal vector, the triangular facet is connected, to obtain a three-dimensional model of the scene represented by a triangular mesh.

[0090] S6: An image of a to-be-monitored target in a scene is captured by using a camera, and feature points of the image of the to-be-monitored target are calculated.

[0091] Specifically, the same algorithm as in step S2 is used to obtain feature points and corresponding descriptors of the image of the scene. In this embodiment, the Superpoint algorithm is used, an image of the to-be-monitored target is input, a fully convolutional neural network is used to output feature points and corresponding descriptors of the image of the to-be-monitored target.

[0092] S7: Image feature point matching is used to find feature points of the image of the scene that match the image of the to-be-monitored target.

[0093] Specifically, a feature matching network is first constructed, including sequentially connected attention GNN network units and optimal matching layers. The feature points of the target image to be monitored and its feature description vector, the feature points of each scene image in the scene image set and its feature description vector are input into the feature matching network, and sequentially pass through the attention GNN network unit and the optimal matching layer. The attention GNN network unit consists of a key point encoder, a multi-way graph neural network and an attention weight module. The key point encoder encodes the feature point position p and the feature description vector d into a vector f. The multi-way graph neural network considers a complete graph whose nodes are the key points of the two images. There are two types of edges in the graph: self-graph edges (connecting key points in the same image) and cross-graph edges (connecting key points between two images). Using a message passing mechanism, each node updates its representation based on the information of all related nodes. This process involves information aggregation of self-graph edges and cross-graph edges. In the attention aggregation module, self-attention and cross-attention are used to enhance the feature matching performance of vector f. The self-attention mechanism enables keypoints to focus on other keypoints within the same image, helping to capture their spatial relationships and visual appearance. The cross-attention mechanism enables keypoints to focus on potential matching points between two images, which helps compare keypoints in different images and find the correct match. The attention enhancement is repeated L times, and the representation of each keypoint is updated to form the final matching descriptor. Then, the optimal matching layer is entered, and the matching score matrix is ​​obtained by calculating the inner product of the feature matching vectors. An extra row and column are added to the matching score matrix as a trash can to handle unmatched key points. Then, the Sinkhorn algorithm is used to iterate T times, and the trash can nodes are discarded to solve the optimal feature allocation matrix. The above-mentioned feature matching network uses manually annotated real matches as supervision and is trained by minimizing the negative log-likelihood loss function. This loss function takes into account the confidence of correct matches and the confidence of unmatched key points. The optimal feature allocation matrix represents the N possibilities of matching feature points in the two images. The largest value in each row and column is used as the matching feature point between the target image to be monitored and the scene image, that is, the feature point of the scene image that matches the target image to be monitored is found.

[0094] S8: Using the scene image feature points that match the target image to be monitored, according to the positions of the scene image feature points in the three-dimensional model obtained in step S3, the position of the target to be monitored in the three-dimensional model is obtained, and the monitoring target area and its normal direction are calculated.

[0095] Since the feature points on the target to be monitored in step S7 have found matching feature points in the scene image, and these matching feature points have obtained three-dimensional coordinates in step S3, that is, their positions in the three-dimensional model have been determined, the position of the target to be monitored in the three-dimensional model can be determined accordingly.

[0096] The calculation of the normal direction can be performed by searching for neighboring points and using covariance analysis to calculate the principal axis of the neighboring points, calculating the gradient of the volume density function, etc. According to the positions of the matched feature points found in the scene image in the three-dimensional space, the embodiment performs plane fitting on the set of points in the three-dimensional space as the surface approximation of the target to be monitored, so as to determine the normal direction of the surface as the normal direction of the target region to be monitored.

[0097] S9: According to the positions of the cameras in the normal direction region (i.e. within the set solid angle) calculated based on the positions of the cameras in the three-dimensional space and the target region to be monitored and the normal direction thereof, the azimuth and distance of the cameras are selected, and the cameras meeting the preset conditions are selected, and the azimuth of the target in the camera coordinate and the focal length of the lens are calculated, and are set as the preset position.

[0098] Specifically, the positions of the cameras are determined in the design or have been installed, i.e. the positions of the cameras in the three-dimensional model are known. According to the positions of the target and the cameras in the three-dimensional model and the installation mode of the cameras (the installation mode determines the coordinate axes and the coordinate origin of the camera coordinate), the position of the target in the camera coordinate is determined.

[0099] The azimuth of the target in the camera coordinate and the distance between the target and the camera are calculated.

[0100] According to the distance between the target and the camera and the pinhole imaging principle, the focal length of the camera lens (referred to as the camera focal length) is determined, so that the target is clearly imaged on the imaging surface of the camera, i.e. the target is within a specified size range on the image, such as 1 / 4-3 / 4 of the image size. According to the pinhole imaging principle, i.e. the similar triangle relationship, when the size of the target and the object distance are known, the image distance is proportional to the size of the imaging (target image), and the image distance of the target image within the specified size is easily calculated, so as to calculate the focal length according to the convex lens imaging formula. Since the object distance is usually much larger than the image distance in a video monitoring system, the image distance can be used as an approximation of the focal length.

[0101] If the calculated focal length is within the range of the camera, it indicates that the camera can be used to monitor the target, and the azimuth of the target in the camera coordinate and the focal length of the camera lens are set as a preset position of the camera.

[0102] The embodiment uses the peripheral points of the target region to be monitored, such as the four vertices of the four-sided enclosing frame, to calculate the distance and azimuth of the camera in the camera space, i.e. the camera imaging external parameters, according to the camera internal parameters such as the predetermined focal length and the imaging surface size, to calculate the position and region of the target region to be monitored on the imaging surface of the camera, and to adjust the focal length of the camera in proportion to the size of the region. If the focal length of the camera is within the actual adjustable range, the camera is suitable.

[0103] S10: The image is taken at the preset position, the image offset is obtained through image matching, and the preset position is corrected according to the camera parameters.

[0104] Since the preset positions obtained by the above calculation can be deviated from the actual requirement, if the target to be monitored is too small (not clear) or too large (out of the image boundary) on the image, or part of the target to be monitored is out of the boundary, the image taken at each preset position is matched with the image of the target to be monitored, the degree of the center of the target to be monitored deviating from the center of the image, i.e. the angle of the center of the target to be monitored deviating from the optical axis of the camera, is calculated, the orientation of the preset position is corrected so that the center of the target to be monitored is close to the optical axis of the camera, i.e. the center of the target to be monitored is moved to the center of the image; whether the size of the target to be monitored on the image at the focal length of the camera is appropriate is calculated, such as whether the width and / or height is within 1 / 4-3 / 4 of the image, if the size of the target is out of the range, the focal length needs to be scaled according to the scaling factor.

[0105] Generally, there are multiple targets to be monitored, and correspondingly multiple images of the targets to be monitored, thus steps S6 to S10 need to be calculated for each target one by one. Multiple preset positions of the camera can be found for some targets to be monitored, the redundant preset positions can be manually deleted, or the monitoring effect can be calculated, such as the zoom of the camera should not be too large (which will affect the definition), the angle between the normal direction of the surface of the target to be monitored and the optical axis of the camera passing through the center of the target to be monitored should not be too large (too large will also not be clear), and the appropriate preset position is selected.

[0106] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the acts or steps recited in the claims can be performed in a different order and still accomplish desirable results. Also, the process depicted in the figures does not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some implementations, multitasking and parallel processing can be advantageous.

[0107] Those skilled in the art will understand that the above discussion of any of the embodiments is merely exemplary and is not intended to restrict the scope of the application (including claims) to these examples; the above embodiments or technical features among different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes to the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of brevity.

[0108] The embodiments of the present application are intended to cover all such alternatives, modifications, and variations of the present application falling within the broad scope of the appended claims. Accordingly, any and all such modifications, variations, or equivalents that fall within the spirit and scope of the embodiments of the present application are intended to be included within the scope of the present application.

Claims

1. A method for determining a preset position of a video surveillance camera, characterized in that: The steps include: Using a camera to capture a set of scene images; Calculating feature points and feature description vectors of each scene image in the scene image set; Based on the scene image set and its feature points, the motion recovery structure technology is used to obtain the camera posture parameters corresponding to the scene image and the three-dimensional coordinates of the scene image feature points; According to the scene image set, camera intrinsic parameter matrix and camera pose parameters, the scene three-dimensional space is represented as a set of radiation fields; A truncated signed distance field is created from the radiation field based on a signed surface approximation network, and a surface reconstruction algorithm is used to generate a three-dimensional model. Use a camera to shoot the target to be monitored in the scene, and calculate the feature points and feature description vectors of the target image to be monitored; An image feature point matching method is used to find scene image feature points that match the target image to be monitored; Using the scene image feature points that match the target image to be monitored, the position of the target to be monitored in the 3D model is obtained according to the 3D coordinates of the scene image feature points, and the monitoring target area and its normal direction are calculated; Calculate the azimuth angle and distance of the camera in the normal direction area based on the position of the camera in three-dimensional space and the monitored target area and its normal direction. Select the camera that meets the preset conditions, calculate the target's orientation in the camera coordinates and the focal length of the lens, and set it as the preset position. Capture an image at a preset camera position, obtain the image offset through image matching, and correct the preset position according to the camera parameters.

2. A method for determining a preset position of a video surveillance camera according to claim 1, characterized in that: When a camera is used to capture a set of scene images, the Zhang Zhengyou calibration method is used to determine the camera's intrinsic parameter matrix.

3. The method for determining a preset position of a video surveillance camera according to claim 1, wherein: The Superpoint method is used to calculate the feature points and feature description vectors of each scene image in the scene image set, as well as the feature points and feature description vectors of the target image to be monitored.

4. The method for determining a preset position of a video surveillance camera according to claim 1, wherein: The method of using the structure-from-motion technology to obtain the camera posture parameters corresponding to the scene image and the three-dimensional coordinates of the scene image feature points includes: Use scene image feature points and feature description vectors to perform two-to-two matching between images to find image pairs with overlapping scenes; Calculate the fundamental matrix between the image pairs and determine whether the image pairs are related based on whether a preset number of matching feature points satisfy the mapping relationship; Calculate the intrinsic matrix by combining the camera’s intrinsic matrix between the relevant image pairs; Decomposing the essential matrix by singular value decomposition to obtain the extrinsic parameter matrix of the camera, and obtaining the camera posture corresponding to the scene image; Based on the acquired camera pose and matched feature points, triangulation is used to solve the three-dimensional coordinates of the scene image feature points in the three-dimensional scene; The bundle adjustment method is used to simultaneously optimize the camera pose and the 3D coordinates of the scene image feature points.

5. The method for determining a preset position of a video surveillance camera according to claim 1, wherein: The method of expressing the scene three-dimensional space as a set of radiation fields according to the scene image set, the camera intrinsic parameter matrix and the camera posture parameters includes: A neural radiation field model is constructed based on a multi-layer perceptron neural network for three-dimensional implicit space modeling; Represent a spatial point in the three-dimensional space of the scene as a 5D vector, including the 3D coordinate position of a spatial point and viewing direction , as the input of the neural radiation field model, the neural radiation field model outputs the corresponding 3D coordinate position Density and viewing direction The relevant 3D coordinate position color; Based on the scene image set, the camera intrinsic parameter matrix and the camera pose parameters, the neural radiation field model is rendered using volume rendering from the perspective determined by the camera pose parameters, the generated color and density values ​​are rendered into an image, and the parameters of the neural network are optimized based on the error between the rendered image and the real image of the scene in the scene image set; Optimize the parameters of the neural network using all images in the scene image set, minimize the error between the rendered image and the scene image in the scene image set, and obtain a trained neural radiation field model; Based on the trained neural radiance field model, the three-dimensional space of the scene is represented as a set of learnable and continuous radiance fields.

6. The method for determining a preset position of a video surveillance camera according to claim 1, wherein: The method of creating a truncated signed distance field from a radiation field based on a signed surface approximation network includes: Construct a signed surface approximation network based on multi-layer perceptron; Training the signed surface approximation network based on the occupied space information of the radiation field aggregated along the rays and the rendered depth map; Using the trained signed surface approximation network to obtain a truncated signed distance function, and creating a truncated signed distance field based on the signed distance function; The truncated signed distance function determines the distance from a point to the boundary of a finite region in space and defines the sign of the distance at the same time: the sign is positive when the point is inside the boundary of the region, negative when the point is outside the boundary, and 0 when the point is on the boundary.

7. The method for determining a preset position of a video surveillance camera according to claim 1, wherein: The surface reconstruction algorithm uses the Marching cubes algorithm, a method for creating polygonal surfaces of three-dimensional scalar field isosurfaces.

8. The method for determining a preset position of a video surveillance camera according to claim 1, wherein: The method of using the image feature point matching method to find the scene image feature points that match the target image to be monitored includes: Constructing a feature matching network, the feature matching network includes sequentially connected attention GNN network units and an optimal matching layer, and using manually annotated real matches as supervision, and training the network by minimizing the negative log-likelihood loss function to obtain a trained network; The feature points of the target image to be monitored and its feature description vector, and the feature points of each scene image in the scene image set and its feature description vector are input into the feature matching network to obtain the scene image feature points that match the target image to be monitored.

9. The method for determining a preset position of a video surveillance camera according to claim 1, wherein: The calculation of the orientation of the target in the camera coordinates and the focal length of the lens and setting them as preset positions includes: Determine the position of the target in the camera coordinates based on the positions of the target and the camera in the 3D model and the installation method of the camera; Calculate the target's position in camera coordinates and its distance from the camera; The focal length of the camera lens is determined based on the distance between the target and the camera and the pinhole imaging principle, so that the target is clearly imaged on the camera imaging surface; The orientation of the target in the camera coordinates and the focal length of the camera lens are set as a preset position of the camera.

10. The method for determining a preset position of a video surveillance camera according to claim 1, wherein: The method of obtaining the image offset by image matching and correcting the preset position according to the camera parameters includes: Match the image captured at each preset position with the image of the target to be monitored, and calculate the angle at which the center of the target to be monitored in the captured image deviates from the optical axis of the camera; Correct the preset position so that the center of the target to be monitored is close to the optical axis of the camera; The camera focal length is scaled according to the size of the monitored target on the image at the current camera focal length.

Images