Method for determining preset position of video monitoring camera

Abstract

The invention discloses a method for determining a preset position of a video monitoring camera. Camera presetting positions usually need to be manually set one by one, and time and labor are wasted. The method is used for a video monitoring system, and automatic setting of the preset position of the camera is realized by utilizing a group of scene images and the to-be-monitored target image, so that the efficiency is greatly improved. The method comprises the following steps: acquiring feature points based on a scene image set by adopting a computer vision technology, generating a three-dimensional model of a scene by adopting a motion recovery structure technology and a surface reconstruction algorithm, and obtaining the position of a to-be-monitored target in a three-dimensional space by matching the feature points of a to-be-monitored target image with the feature points of the scene image set. Therefore, a proper camera is selected, a corresponding preset position is calculated, and finally the preset position is corrected according to an actual monitoring effect.

Description

Technical Field

[0001] The invention relates to the technical field of video monitoring, and in particular to a method for determining a preset position of a video monitoring camera. Background Art

[0002] With the rapid development of the national economy, the demand for electricity is growing, which puts higher requirements on the safety of power equipment. Therefore, a more reasonable and reliable substation video surveillance system is needed. Due to the complex scenes in the substation and the large number of types and equipment in production and operation, how to reasonably set the preset points of video surveillance cameras will directly determine the role that video surveillance technology can play in the production and operation of the power grid. Other application scenarios also involve the problem of setting preset positions for video surveillance cameras, such as oil depots, warehouses for flammable and explosive dangerous goods, industrial plants, etc.

[0003] At present, the determination of the preset position of video surveillance cameras mainly relies on manual work, which is labor-intensive and inefficient. Especially when drones and other means are used to obtain scene images and use scene images to assist in the design of video surveillance systems, if the camera preset position can be automatically determined based on the scene image and the target to be monitored, the work efficiency can be greatly improved. Therefore, it is particularly important to study a method for determining the preset position of video surveillance cameras. Summary of the invention

[0004] Based on the above background, the present invention proposes a method for determining the preset position of a video surveillance camera, which uses a set of scene images and a target image to be monitored to realize automatic setting of the camera preset position, thereby greatly improving efficiency. The following technical solutions are specifically adopted:

[0005] A method for determining a preset position of a video surveillance camera comprises the following steps:

[0006] Using a camera to capture a set of scene images;

[0007] Calculate the feature points and feature description vectors of each scene image in the scene image set;

[0008] Based on the scene image set and its feature points, the structure-from-motion technology is used to obtain the camera posture parameters corresponding to the scene image and the three-dimensional coordinates of the scene image feature points.

[0009] According to the scene image set, the camera intrinsic parameter matrix and the camera posture parameters, the scene three-dimensional space is represented as a set of radiation fields;

[0010] A truncated signed distance field is created from the radiation field based on a signed surface approximation network, and a surface reconstruction algorithm is used to generate a three-dimensional model;

[0011] Use a camera to shoot the target to be monitored in the scene, and calculate the feature points and feature description vectors of the image of the target to be monitored;

[0012] An image feature point matching method is used to find scene image feature points that match the target image to be monitored;

[0013] Using the scene image feature points that match the target image to be monitored, the position of the target to be monitored in the three-dimensional model is obtained according to the three-dimensional coordinates of the scene image feature points, and the monitoring target area and its normal direction are calculated;

[0014] According to the position of the camera in the three-dimensional space and the monitoring target area and its normal direction, the azimuth angle and distance of the camera in the normal direction area are calculated, and the camera that meets the preset conditions is selected, and the azimuth of the target in the camera coordinates and the focal length of the lens are calculated and set as the preset position;

[0015] Capture an image at the preset position of the camera, obtain the image offset through image matching, and correct the preset position according to the camera parameters.

[0016] Furthermore, when a camera is used to capture a set of scene images, the Zhang Zhengyou calibration method is used to determine the camera's intrinsic parameter matrix.

[0017] Furthermore, the Superpoint method is used to calculate the feature points and feature description vectors of each scene image in the scene image set, as well as the feature points and feature description vectors of the target image to be monitored.

[0018] Furthermore, the method of using the structure-from-motion technology to obtain the camera posture parameters corresponding to the scene image and the three-dimensional coordinates of the feature points of the scene image includes:

[0019] Use scene image feature points and feature description vectors to perform two-to-two matching between images to find image pairs with overlapping scenes;

[0020] Calculate the basic matrix between the image pairs, and determine whether the image pairs are related based on whether there are a preset number of matching feature points that satisfy the mapping relationship;

[0021] The intrinsic matrix is ​​calculated by combining the camera's intrinsic matrix between the relevant image pairs;

[0022] Decomposing the essential matrix by singular value decomposition to obtain the extrinsic parameter matrix of the camera, and obtaining the camera posture corresponding to the scene image;

[0023] According to the acquired camera posture and matched feature points, the three-dimensional coordinates of the scene image feature points in the three-dimensional scene are solved by triangulation;

[0024] The bundle adjustment method is used to simultaneously optimize the camera pose and the 3D coordinates of the scene image feature points.

[0025] Furthermore, the step of representing the scene three-dimensional space as a set of radiation fields according to the scene image set, the camera intrinsic parameter matrix and the camera posture includes:

[0026] A neural radiation field model is constructed based on a multi-layer perceptron neural network for three-dimensional implicit space modeling;

[0027] Represent a spatial point in the three-dimensional space of the scene as a 5D vector, including the 3D coordinate position (x, y, z) of a spatial point and the viewing direction As input to the neural radiation field model, the neural radiation field model outputs the density and viewing direction corresponding to the 3D coordinate position (x, y, z) The color of the associated 3D coordinate position (x, y, z);

[0028] According to the scene image set, the camera intrinsic parameter matrix and the camera pose, the neural radiation field model is rendered using volume rendering from the perspective determined by the camera pose, the generated color and density values ​​are rendered into an image, and the parameters of the neural network are optimized based on the error between the rendered image and the real image of the scene in the scene image set;

[0029] Use all images in the scene image set to perform the above training, minimize the error between the rendered image and the scene image in the scene image set, and obtain a trained neural radiation field model;

[0030] Based on the trained neural radiance field model, the three-dimensional space of the scene is represented as a set of learnable and continuous radiance fields.

[0031] Furthermore, the step of creating a truncated signed distance field from the radiation field based on the signed surface approximation network includes:

[0032] Construct a signed surface approximation network based on multi-layer perceptron;

[0033] Training the signed surface approximation network based on the occupied space information of the radiation field aggregated along the rays and the rendered depth map;

[0034] Using the trained signed surface approximation network to obtain a truncated signed distance function, and creating a truncated signed distance field based on the signed distance function;

[0035] The truncated signed distance function determines the distance from a point to the boundary of a finite region in space, and defines the sign of the distance at the same time, where the point is positive inside the boundary of the region, negative outside, and 0 when located on the boundary.

[0036] Furthermore, the surface reconstruction algorithm adopts Marching cubes algorithm, a method for creating polygonal surfaces of three-dimensional scalar field isosurfaces.

[0037] Furthermore, the method of finding scene image feature points that match the target image to be monitored by using an image feature point matching method includes:

[0038] Constructing a feature matching network, the feature matching network includes sequentially connected attention GNN network units and an optimal matching layer, and using manually annotated real matches as supervision, training the network by minimizing a negative log-likelihood loss function to obtain a trained network;

[0039] The feature points of the target image to be monitored and its feature description vector, and the feature points of each scene image in the scene image set and its feature description vector are input into the feature matching network to obtain the scene image feature points that match the target image to be monitored.

[0040] Furthermore, the calculation of the orientation of the target in the camera coordinates and the focal length of the lens and setting them as preset positions includes:

[0041] Determine the position of the target in the camera coordinates according to the positions of the target and the camera in the three-dimensional model and the installation method of the camera;

[0042] Calculate the target's position in the camera coordinates and the distance from the camera;

[0043] The focal length of the camera lens is determined according to the distance between the target and the camera and the pinhole imaging principle, so that the target can be clearly imaged on the camera imaging surface;

[0044] The position of the target in the camera coordinates and the focal length of the camera lens are set as a preset position of the camera.

[0045] Furthermore, the step of obtaining the image offset by image matching and correcting the preset position according to the camera parameters includes:

[0046] Match the image captured at each preset position with the image of the target to be monitored, and calculate the angle at which the center of the target to be monitored in the captured image deviates from the optical axis of the camera;

[0047] Correct the preset position so that the center of the target to be monitored is close to the optical axis of the camera;

[0048] The camera focal length is scaled according to the size of the monitored target on the image at the current camera focal length.

[0049] The beneficial effects of the present invention are as follows:

[0050] The present invention adopts computer vision technology, obtains feature points based on a scene image set, generates a three-dimensional model of the scene by using motion recovery structure technology and surface reconstruction algorithm, and obtains the position of the target to be monitored in three-dimensional space by matching the feature points of the target image to be monitored with the feature points of the scene image set, thereby selecting a suitable camera, calculating the corresponding preset position, and finally correcting the preset position according to the actual monitoring effect. Through the above-mentioned automated preset point determination process, the preset points of each camera can be calculated in advance, thereby greatly reducing the time of the preset point setting process, reducing manpower investment, and effectively speeding up the project progress. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 The present invention provides a simplified flowchart of a method for determining preset positions of video surveillance cameras according to an embodiment of the present invention.

[0052] Figure 2 The figure is a simplified schematic diagram of the incremental motion recovery structure technology process shown in an embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0054] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] The present invention can be used for camera deployment and preset position determination in scenes such as substations and industrial plants. Without loss of generality, the embodiment of the present invention assumes that the application scene is a substation.

[0056] refer to Figure 1 The embodiment of the present invention shows a method for determining a preset position of a video surveillance camera in a specified scene, comprising the following steps:

[0057] S1: Use a camera to capture a set of scene images.

[0058] Specifically, a set of images of a scene is captured by a camera. The images in the set of scene images should cover the entire scene. The same camera is used to capture the scene from different angles without changing the internal parameters. The shooting is completed in an environment where the lighting is bright and the lighting conditions do not change drastically. The internal parameters can be expressed in the form of an internal matrix. Of course, if different cameras are used for shooting, or the internal parameters change during the shooting process, it is necessary to determine the internal parameter matrix during shooting. It should be pointed out that the camera is only used to capture images of the scene and has no direct connection with the camera to be deployed.

[0059] The camera intrinsic parameter matrix can be determined by the parameters of the camera during shooting, such as shooting resolution, sensor target size, focal length, etc., but the accuracy is usually not high. Therefore, the camera intrinsic parameter matrix is ​​generally obtained by calibrating the camera. The traditional camera calibration method can be used to calibrate the camera using calibration objects, such as Tsai's two-step method and Zhang Zhengyou's calibration method. The active visual camera calibration method can also be used to actively control the camera to make specific movements for calibration. These are all existing technologies.

[0060] This embodiment adopts Zhang Zhengyou calibration method, uses a calibration board composed of two-dimensional chessboard for calibration, collects 10-20 pictures of the calibration board in different positions, which can be achieved by moving the camera or the calibration board. The SIFT algorithm is used to extract the corner points in the calibration board picture, that is, the intersection points of the black and white chessboard. The camera's intrinsic parameter matrix K is obtained by calibration.

[0061] S2: Calculate the feature points and feature description vectors of each scene image in the scene image set.

[0062] Specifically, the image feature points may be calculated using commonly used algorithms such as Harris, FAST, SURF, and SIFT to obtain feature points and corresponding descriptors of the scene image.

[0063] This embodiment uses the Superpoint (see arXiv.1712.07629. Superpoint: Self-Supervised Interest Point Detection and Description) algorithm to calculate the feature points of each scene image in the scene image set, which can stably and repeatedly detect rich feature points under different lighting conditions and viewpoints, and calculate descriptors for these points for advanced tasks such as image matching. The scene image is input, and Superpoint uses a fully convolutional neural network to simultaneously calculate the image pixel-level interest point location and related descriptors in a single forward propagation.

[0064] First, the network uses a shared VGG-style encoder to reduce the dimensionality of the input image, and then processes the image through two decoder heads, one for interest point detection and the other for descriptor calculation. The interest point decoder calculates a "point-like" probability for each pixel, while the descriptor decoder outputs an L2-normalized fixed-length descriptor. This process involves self-supervised training, which includes synthetic data pre-training and homography adaptation steps. Finally, Superpoint outputs feature points and corresponding feature description vectors of the scene image.

[0065] S3: Based on the scene image set and its feature points, the motion recovery structure technology is used to obtain the camera posture parameters corresponding to the scene image and the three-dimensional coordinates of the scene image feature points.

[0066] Specifically, Structure From Motion (SFM) is an algorithm that estimates three-dimensional structure from a series of multiple two-dimensional image sequences containing visual motion information. It can restore the camera pose through multiple scene images and reconstruct the sparse point cloud structure. Structure From Motion can be used in a global or incremental manner. The incremental method first estimates the camera pose and reconstructs the three-dimensional coordinate points based on two images of a set of scenes. After this initial process, new scene images are added for measurement, and the above steps are repeated until all images are processed. The global Structure From Motion adds all images to the reconstruction queue at once and matches the feature points of all images at once. The incremental method has high precision and low efficiency, while the global method has high efficiency and low precision.

[0067] This embodiment uses incremental motion recovery structure technology, such as Figure 2 The specific processing flow is as follows:

[0068] S301: Use image feature points and feature description vectors to perform two-to-two matching between images to find image pairs with overlapping scenes.

[0069] The image feature points and feature description vectors extracted in step S2 are used to determine the similarity of the feature points of the two images using the Euclidean distance, and algorithms such as the KD tree are used to match the feature points.

[0070] S302: Calculate the basic matrix between the image pairs to determine whether there are enough feature matching points to satisfy the mapping relationship, thereby determining whether the image pairs are related.

[0071] Specifically, based on computer vision theory, the constraint relationship between two images is expressed in an algebraic way, which is the basic matrix F. The basic matrix between the image pairs is estimated using algorithms such as Random Sample Consensus (RANSAC). While calculating the basic matrix, the RANSAC algorithm determines whether there are enough feature matching points to satisfy the mapping relationship. For example, if there are more than 8 pairs of feature matching points, the image pairs have scene overlap, which is called a related image pair.

[0072] S303: Calculate the intrinsic matrix between the relevant image pairs by combining the camera's intrinsic parameter matrix, E = K -T FK, where K is the camera intrinsic matrix, E is the essential matrix, and F is the fundamental matrix.

[0073] S304: The intrinsic matrix is ​​decomposed by singular value to obtain the extrinsic parameter matrix of the camera, and the camera posture corresponding to the image is obtained.

[0074] Specifically, by decomposing the essential matrix E, E = t ^ R, obtain the camera's rotation matrix R and translation matrix t, that is, obtain the camera's external parameter matrix and obtain the camera posture corresponding to the image.

[0075] S305: According to the camera posture and the matched feature points, the three-dimensional coordinates of the feature points in the three-dimensional scene are solved by triangulation.

[0076] Specifically, using matching feature points from two or more camera perspectives, the camera pose is known, and in triangulation, the three-dimensional point p is mapped to a point x on the plane through the projection equation i , for each view j, the projection equation is expressed as x j =P j p, where P j is the projection matrix of the jth camera, which contains the combination of the extrinsic matrix and the camera attitude parameters, specifically expressed as P j =K j [R j |t j ]; point x on the image plane i Represented as homogeneous coordinates and normalized to make them directly related to the camera coordinate system; the projection equations of the 3D points of each view to the points on the image plane are solved using the least squares method to minimize the reprojection error of all projection points. The best three-dimensional coordinate estimate of point p is obtained, thereby calculating the coordinates of the matching feature points in three-dimensional space.

[0077] S306: Use bundle adjustment to simultaneously optimize the camera posture and the three-dimensional coordinates of the feature points.

[0078] Specifically, nonlinear optimization iteration is performed by minimizing the reprojection error. The objective function is the sum of the distances between all matched feature points and their projection points. The optimization variables include the camera projection matrix P j And the three-dimensional point spatial coordinates, find a set of optimal projection matrices and spatial point three-dimensional coordinates, and obtain the camera posture corresponding to the final optimized image and the three-dimensional coordinates of the image feature points.

[0079] S4: The scene three-dimensional space is represented as a set of radiation fields according to the scene image set, the camera intrinsic parameter matrix and the camera pose parameters.

[0080] Specifically, based on the scene image set, the camera intrinsic parameter matrix and the camera posture parameters, a multi-layer perceptron neural network for three-dimensional implicit space modeling is used to construct the neural radiation field. The network input is a 5D vector, including the 3D coordinate position x = (x, y, z) of a spatial point and the viewing direction The output is the density of the corresponding 3D position and the voxel density σ and color value c = (r, g, b) of the 3D point related to the viewing angle. This neural network can be written as: F Θ :(x,d)→(c,σ). In the specific implementation, the vector x is first input into the MLP network, and σ and intermediate features are output. The intermediate features and d are then input into an additional fully connected layer to predict the color. A batch of camera rays are randomly sampled from the scene images of the scene image set. The camera rays are rays starting from the camera origin and passing through each pixel in the corresponding scene image. The color and density information in the 3D space is rendered onto the 2D image using volume rendering technology. Hierarchical volume rendering is used to improve rendering efficiency. First, the light is uniformly sampled to obtain the weight of each sampling point. The weight is determined by the volume density and the distance of the position. A normalization is performed on the light according to the obtained weight, and the sum of the weights is set to 1 to obtain the probability density function. The probability density function is used for fine sampling, that is, more sampling points are given on the more important line segments. According to the voxel density σ and color value c of the 3D position of all sampling points on the camera ray, the color value of the two-dimensional pixel corresponding to the camera ray is calculated to obtain the rendered image. The error between the rendered image and the real scene image is calculated, and the loss function is minimized using optimization algorithms such as gradient descent, so as to train the neural network and obtain the neural radiation field model. Finally, the trained model is used to represent the three-dimensional space of the scene as a set of continuous radiation fields.

[0081] S5: A truncated signed distance field is created from the radiation field based on a signed surface approximation network, and a surface reconstruction algorithm is used to generate a three-dimensional model.

[0082] The voxel density σ of a 3D point in the radiation field represents its density or absorptivity, so it can be used to determine the surface of the object and expressed using a signed distance field to generate a three-dimensional model. This embodiment uses the following method to generate a three-dimensional model:

[0083] S51. First create a truncated signed distance field.

[0084] The 3D coordinates are provided to the Signed Surface Approximation Network module (SSAN), which takes the 3D coordinates as input and outputs three main pieces of information: the truncated signed distance function (TSDF), the normal vector, and the appearance feature vector, thereby obtaining a truncated signed distance field representation of the scene. The SSAN module is divided into two independent branches, a geometry branch for outputting TSDF approximations and normal predictions, and an appearance branch for outputting color features.

[0085] The truncated signed distance field is represented by a truncated signed distance function, which determines the distance from a point to the boundary of the region in a finite region in space and defines the sign of the distance at the same time. The point is positive inside the region boundary, negative outside, and 0 on the boundary. The normal vector is a unit vector that represents the direction of the surface normal at the input point and is used for rendering and smoothing regularization. The appearance feature vector is an 8-dimensional vector containing color and material information for the appearance network.

[0086] The SSAN network consists of a multi-layer perceptron and is trained based on the radiation field obtained in step S4. The occupied space information aggregated along the rays of the radiation field model and the depth map obtained by rendering guide the training of SSAN. The training involves three loss functions: the signed distance loss of internal and external points, the normal regularization loss, and the appearance loss. The signed distance loss of internal and external points ensures that the network can correctly predict the signed distance of internal and external points. The normal regularization loss ensures that the learned TSDF function has smooth normals near the surface. The appearance loss optimizes the appearance features by comparing the predicted colors with the colors in the actual training images. The weights of the SSAN network are randomly initialized, and the loss is calculated based on the predicted results and the real data. The gradient of the loss function is back-propagated to update the weights of the SSAN network until the training is completed, and the TSDF expressing the scene is obtained, and the truncated signed distance field expression of the scene is obtained.

[0087] S52. Then, a surface reconstruction algorithm is used to generate a three-dimensional model from the signed distance field.

[0088] The surface reconstruction method may adopt common methods such as Poisson surface reconstruction method and surface reconstruction based on Delaunay triangulation. This embodiment uses Marching cubes algorithm, a method for creating polygonal surfaces of three-dimensional scalar field isosurfaces, to perform surface reconstruction:

[0089] After preprocessing, the TSDF expressing the scene is read into a specific array or octree; a hexahedron is extracted from the mesh data volume, and all the information of the hexahedron, such as the values ​​of the 8 vertices, coordinate positions, etc., is obtained at the same time; the function values ​​of the 8 vertices of the current hexahedron are compared with the given isosurface values ​​to obtain the state table of the hexahedron; according to the state table index of the current hexahedron, the edges of the hexahedron intersecting with the isosurface are found, and the position coordinates of each intersection are calculated by linear interpolation; the normal vectors of the 8 vertices of the current hexahedron are calculated by the central difference method, and the normal direction of each vertex of the triangle patch is obtained by linear interpolation; according to the coordinates of the vertices of each triangle patch, the vertex normal vectors are used to connect the triangles to obtain the three-dimensional model of the scene represented by the triangular mesh.

[0090] S6: Use a camera to shoot the target to be monitored in the scene, and calculate feature points of the image of the target to be monitored.

[0091] Specifically, the same algorithm as in step S2 is used to obtain the feature points and corresponding descriptors of the scene image. This embodiment uses the Superpoint algorithm, inputs the target image to be monitored, and outputs the feature points and corresponding descriptors of the target image to be monitored through a fully convolutional neural network.

[0092] S7: Using an image feature point matching method to find scene image feature points that match the target image to be monitored.

[0093] Specifically, a feature matching network is first constructed, including sequentially connected attention GNN network units and optimal matching layers. The feature points of the target image to be monitored and its feature description vector, the feature points of each scene image in the scene image set and its feature description vector are input into the feature matching network, and sequentially pass through the attention GNN network unit and the optimal matching layer. The attention GNN network unit consists of a key point encoder, a multi-way graph neural network and an attention weight module. The key point encoder encodes the feature point position p and the feature description vector d into a vector f. The multi-way graph neural network considers a complete graph whose nodes are the key points of two images. There are two types of edges in the graph: self-graph edges (connecting key points in the same image) and cross-graph edges (connecting key points between two images). Using the message passing mechanism, each node updates its representation based on the information of all related nodes. This process involves information aggregation of self-graph edges and cross-graph edges. In the attention aggregation module, self-attention and cross-attention are used to enhance the feature matching performance of vector f. The self-attention mechanism enables key points to focus on other key points in the same image, which helps to capture the spatial relationship and visual appearance of key points; the cross-attention mechanism enables key points to focus on potential matching points between two images, which helps to compare key points in different images and find the correct match. The attention enhancement is repeated L times, and the representation of each key point is updated to form the final matching descriptor. Then, the optimal matching layer is entered, and the matching score matrix is ​​obtained by calculating the inner product of the feature matching vector, and an extra row and column are added to the matching score matrix as a trash can to handle unmatched key points; then, the Sinkhorn algorithm is used, and after T iterations, the trash can nodes are thrown away to solve the optimal feature allocation matrix; the above-mentioned feature matching network uses manually annotated real matches as supervision and is trained by minimizing the negative log-likelihood loss function, which takes into account the confidence of correct matches and the confidence of unmatched key points; the optimal feature allocation matrix represents the N possibilities of matching feature points in the two images, and the largest value in each row and column is used as the matching feature point of the target image to be monitored and the scene image, that is, the scene image feature point that matches the target image to be monitored is found.

[0094] S8: Using the scene image feature points that match the target image to be monitored, according to the positions of the scene image feature points in the three-dimensional model obtained in step S3, the position of the target to be monitored in the three-dimensional model is obtained, and the monitoring target area and its normal direction are calculated.

[0095] Since the feature points on the monitored target in step S7 have found matching feature points in the scene image, and these matching feature points have obtained three-dimensional coordinates in step S3, that is, their positions in the three-dimensional model have been determined, the position of the monitored target in the three-dimensional model can be determined accordingly.

[0096] The normal direction can be calculated by searching for adjacent points, using covariance analysis to calculate the principal axis of the adjacent points, calculating the gradient of the volume density function, etc. In this embodiment, according to the positions of the matching feature points found in the scene image in the three-dimensional space, the plane fitting is performed on the set of points in the three-dimensional space as the surface approximation of the target to be monitored, so that the normal direction of the surface can be determined as the normal direction of the monitored target area.

[0097] S9: Calculate the azimuth angle and distance of the camera within the normal direction area (i.e., within the set solid angle) based on the position of the camera in the three-dimensional space and the monitored target area and its normal direction, select the camera that meets the preset conditions, calculate the azimuth of the target in the camera coordinates and the focal length of the lens, and set it as the preset position.

[0098] Specifically, the position of the camera has been determined or installed during design, that is, its position in the three-dimensional model is known. According to the positions of the target and the camera in the three-dimensional model and the installation method of the camera (the installation method determines the coordinate axis and coordinate origin of the camera coordinates), the position of the target in the camera coordinates is determined;

[0099] Calculate the target's position in the camera coordinates and the distance from the camera;

[0100] The focal length of the camera lens (camera focal length for short) is determined based on the distance between the target and the camera and the pinhole imaging principle, so that the target is clearly imaged on the camera imaging surface, even if the target is within a specified size range on the image, such as 1 / 4 to 3 / 4 image size; based on the pinhole imaging principle, that is, the relationship of similar triangles, when the target size and object distance are known, the proportional relationship between the image distance and the image (target image) size is obtained, and it is easy to calculate the image distance of the target image at a specified size, thereby calculating the focal length based on the convex lens imaging formula; since the object distance in a video surveillance system is usually many times larger than the image distance, the image distance can be used as an approximation of the focal length;

[0101] If the calculated focal length is within the range of the camera, it means that the camera can be used to monitor the target, and the position of the target in the camera coordinates and the focal length of the camera lens are set as a preset point of the camera.

[0102] This implementation uses the outer points of the target area to be monitored, such as the four vertices of a four-sided bounding box, and their distance and orientation in the camera space, i.e., the external parameters of camera imaging. According to the internal parameters of the camera, such as the predetermined focal length and the size of the imaging surface, the position and area of ​​the target area to be monitored on the camera imaging surface are calculated, and the camera focal length is adjusted proportionally according to the size of the area. If the camera focal length is within the actual adjustable range, it is a suitable camera.

[0103] S10: capturing an image at a preset position and obtaining an image offset through image matching, and correcting the preset position according to camera parameters.

[0104] Since the preset positions calculated above may deviate from actual needs to a certain extent, such as the target to be monitored on the image is too small (not clear) or too large (outside the image boundary), or partially moved out of the boundary, the image captured at each preset position is matched with the image of the target to be monitored, and the degree to which the center of the target to be monitored in the captured image deviates from the center of the image, that is, the angle at which the center of the target to be monitored deviates from the optical axis of the camera, and the preset position orientation is corrected so that the center of the target to be monitored is close to the optical axis of the camera, that is, the center of the target to be monitored is moved to the center of the image; calculate whether the size of the target to be monitored on the image is appropriate under the focal length of the camera, such as the width and / or height is within the range of 1 / 4 to 3 / 4 on the image. If the target size exceeds this range, the focal length needs to be scaled according to the zoom factor.

[0105] Generally, there are multiple targets to be monitored, and correspondingly there are multiple images of the targets to be monitored, so steps S6 to S10 need to be calculated for each target one by one. Some targets to be monitored may have multiple camera preset points, and redundant preset points can be manually deleted, or the appropriate preset points can be selected by calculating the monitoring effect, such as the camera zoom should not be too large (affecting the clarity), the angle between the normal direction of the surface of the target to be monitored and the camera optical axis passing through the center of the target to be monitored should not be too large (too tilted will also make it unclear), and the appropriate preset points can be selected.

[0106] It should be noted that some embodiments of the present invention are described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0107] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present invention (including the claims) is limited to these examples. Under the concept of the present invention, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present invention as described above, which are not provided in detail for the sake of simplicity.

[0108] The embodiments of the present invention are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for determining a preset position of a video surveillance camera, characterized in that: The steps include: Using a camera to capture a set of scene images; Calculate the feature points and feature description vectors of each scene image in the scene image set; Based on the scene image set and its feature points, the structure-from-motion technology is used to obtain the camera posture parameters corresponding to the scene image and the three-dimensional coordinates of the scene image feature points. According to the scene image set, the camera intrinsic parameter matrix and the camera posture parameters, the scene three-dimensional space is represented as a set of radiation fields; A truncated signed distance field is created from the radiation field based on a signed surface approximation network, and a surface reconstruction algorithm is used to generate a three-dimensional model; Use a camera to shoot the target to be monitored in the scene, and calculate the feature points and feature description vectors of the image of the target to be monitored; An image feature point matching method is used to find scene image feature points that match the target image to be monitored; Using the scene image feature points that match the target image to be monitored, the position of the target to be monitored in the three-dimensional model is obtained according to the three-dimensional coordinates of the scene image feature points, and the monitoring target area and its normal direction are calculated; According to the position of the camera in the three-dimensional space and the monitoring target area and its normal direction, the azimuth angle and distance of the camera in the normal direction area are calculated, and the camera that meets the preset conditions is selected, and the azimuth of the target in the camera coordinates and the focal length of the lens are calculated and set as the preset position; Capture an image at the preset position of the camera, obtain the image offset through image matching, and correct the preset position according to the camera parameters.

2. A method for determining a preset position of a video surveillance camera according to claim 1, characterized in that: When a camera is used to capture a set of scene images, the Zhang Zhengyou calibration method is used to determine the camera's intrinsic parameter matrix.

3. A method for determining a preset position of a video surveillance camera according to claim 1, characterized in that: The Superpoint method is used to calculate the feature points and feature description vectors of each scene image in the scene image set, as well as the feature points and feature description vectors of the target image to be monitored.

4. A method for determining a preset position of a video surveillance camera according to claim 1, characterized in that: The method of using the structure-from-motion technology to obtain the camera posture parameters corresponding to the scene image and the three-dimensional coordinates of the feature points of the scene image includes: Use scene image feature points and feature description vectors to perform two-to-two matching between images to find image pairs with overlapping scenes; Calculate the basic matrix between the image pairs, and determine whether the image pairs are related based on whether there are a preset number of matching feature points that satisfy the mapping relationship; The intrinsic matrix is ​​calculated by combining the camera's intrinsic matrix between the relevant image pairs; Decomposing the essential matrix by singular value decomposition to obtain the extrinsic parameter matrix of the camera, and obtaining the camera posture corresponding to the scene image; According to the acquired camera posture and matched feature points, the three-dimensional coordinates of the scene image feature points in the three-dimensional scene are solved by triangulation; The bundle adjustment method is used to simultaneously optimize the camera pose and the 3D coordinates of the scene image feature points.

5. A method for determining a preset position of a video surveillance camera according to claim 1, characterized in that: The method of representing the scene three-dimensional space as a set of radiation fields according to the scene image set, the camera intrinsic parameter matrix and the camera posture includes: A neural radiation field model is constructed based on a multi-layer perceptron neural network for three-dimensional implicit space modeling; Represent a spatial point in the three-dimensional space of the scene as a 5D vector, including the 3D coordinate position (x, y, z) of a spatial point and the viewing direction As input to the neural radiation field model, the neural radiation field model outputs the density and viewing direction corresponding to the 3D coordinate position (x, y, z) The color of the associated 3D coordinate position (x, y, z); According to the scene image set, the camera intrinsic parameter matrix and the camera pose, the neural radiation field model is rendered using volume rendering from the perspective determined by the camera pose, the generated color and density values ​​are rendered into an image, and the parameters of the neural network are optimized based on the error between the rendered image and the real image of the scene in the scene image set; Use all images in the scene image set to perform the above training, minimize the error between the rendered image and the scene image in the scene image set, and obtain a trained neural radiation field model; Based on the trained neural radiance field model, the three-dimensional space of the scene is represented as a set of learnable and continuous radiance fields.

6. A method for determining a preset position of a video surveillance camera according to claim 1, characterized in that: The method of creating a truncated signed distance field from a radiation field based on a signed surface approximation network includes: Construct a signed surface approximation network based on multi-layer perceptron; Training the signed surface approximation network based on the occupied space information of the radiation field aggregated along the rays and the rendered depth map; Using the trained signed surface approximation network to obtain a truncated signed distance function, and creating a truncated signed distance field based on the signed distance function; The truncated signed distance function determines the distance from a point to the boundary of a finite region in space, and defines the sign of the distance at the same time, where the point is positive inside the boundary of the region, negative outside, and 0 when located on the boundary.

7. A method for determining a preset position of a video surveillance camera according to claim 1, characterized in that: The surface reconstruction algorithm uses the Marching cubes algorithm, a method for creating polygonal surfaces of three-dimensional scalar field isosurfaces.

8. A method for determining a preset position of a video surveillance camera according to claim 1, characterized in that: The method of using the image feature point matching method to find the scene image feature points that match the target image to be monitored includes: Constructing a feature matching network, the feature matching network includes sequentially connected attention GNN network units and an optimal matching layer, and using manually annotated real matches as supervision, training the network by minimizing a negative log-likelihood loss function to obtain a trained network; The feature points of the target image to be monitored and its feature description vector, and the feature points of each scene image in the scene image set and its feature description vector are input into the feature matching network to obtain the scene image feature points that match the target image to be monitored.

9. A method for determining a preset position of a video surveillance camera according to claim 1, characterized in that: The calculating the orientation of the target in the camera coordinates and the focal length of the lens and setting them as preset positions includes: Determine the position of the target in the camera coordinates according to the positions of the target and the camera in the three-dimensional model and the installation method of the camera; Calculate the target's position in the camera coordinates and the distance from the camera; The focal length of the camera lens is determined according to the distance between the target and the camera and the pinhole imaging principle, so that the target can be clearly imaged on the camera imaging surface; The position of the target in the camera coordinates and the focal length of the camera lens are set as a preset position of the camera.

10. A method for determining a preset position of a video surveillance camera according to claim 1, characterized in that: The method of obtaining the image offset by image matching and correcting the preset position according to the camera parameters includes: Match the image captured at each preset position with the image of the target to be monitored, and calculate the angle at which the center of the target to be monitored in the captured image deviates from the optical axis of the camera; Correct the preset position so that the center of the target to be monitored is close to the optical axis of the camera; The camera focal length is scaled according to the size of the monitored target on the image at the current camera focal length.

Images