Video spatialization precision optimization method, system and device and storage medium

Abstract

The invention provides a video spatialization precision optimization method, system and device and a storage medium, and relates to the technical field of video spatialization. The method comprises the steps of firstly accessing video data, then inputting first camera parameters including one or more of a horizontal field angle, a vertical field angle, a space coordinate X, a space coordinate Y and a height, carrying out control point marking on a video picture of the video data and a geographic space data picture, and finally carrying out point marking on the geographic space data picture on the basis of a particle swarm optimization (PSO) algorithm. And calculating an optimal camera parameter in combination with the first camera parameter and the control point so as to optimize the video spatialization precision. Therefore, the video spatialization is carried out based on the PSO algorithm, the precision of the video spatialization is improved, and the fusion of the video data and the geographic space data is realized.

Description

Technical Field

[0001] The present application relates to the field of video spatialization technology, and in particular to a method, system, device and storage medium for optimizing video spatialization accuracy. Background Art

[0002] With the widespread application of video surveillance technology in many key fields such as natural resource supervision, public safety, smart transportation, smart agriculture, and park planning and construction, we are faced with the problem of being unable to effectively utilize massive video data.

[0003] Traditional video processing methods have obvious limitations. They only obtain the geographic spatial coordinates of the camera when shooting the video, but cannot obtain the precise spatial coordinates corresponding to each pixel in the video. This limitation leads to low efficiency and insufficient precision of video spatialization. Especially when the terrain of the geographic spatial area corresponding to the video screen is complex and undulating, the problem is particularly prominent. Users need to invest a lot of time and energy to spatially process the video. However, even so, the accuracy obtained often still cannot meet actual needs, and it is difficult to accurately integrate video data with geographic spatial data. This technical bottleneck not only limits the in-depth application of video surveillance data in various fields, but also hinders the advancement of intelligent and refined management. Therefore, there is an urgent need for a more efficient and accurate video spatial processing method to achieve effective utilization of massive video data and provide strong support for intelligent management and decision-making in various fields. Summary of the invention

[0004] In view of the above problems, the present application provides a method for optimizing video spatialization accuracy, including the following contents:

[0005] In a first aspect, the present application provides a method for optimizing video spatialization accuracy, the method comprising:

[0006] Accessing video data, wherein the video data includes a real-time video stream or an offline video file;

[0007] Inputting first camera parameters, wherein the first camera parameters include a horizontal field of view angle, a vertical field of view angle, a spatial coordinate X, a spatial coordinate Y, and a height, wherein inputting the first camera parameters includes inputting one or more of the first camera parameters;

[0008] Marking control points on the video screen and the geographic space data screen of the video data;

[0009] Based on the particle swarm optimization PSO algorithm, the optimal camera parameters are calculated in combination with the first camera parameters and control points to optimize the video spatialization accuracy.

[0010] Optionally, the input first camera parameter includes:

[0011] Enter an exact value or a range value as the first camera parameter.

[0012] Optionally, when the video data is multi-angle video data, marking control points on a video screen of the video data includes:

[0013] Receiving video data shot at multiple angles, wherein the video data shot at multiple angles is obtained by adjusting a pan value, a tilt value, and a zoom value of a camera;

[0014] First, control points are marked on the video image and the geospatial data image at the first angle, and based on the PSO algorithm, the marked control points and the first camera parameters, the optimal camera parameters are calculated for the video data at each angle;

[0015] If the spatialization accuracy of the video at the second angle is less than the preset video accuracy threshold, re-mark the control points for the video data of the second angle, and when the Pan value, Tilt value and Zoom value of the first angle have not been input, input the Pan value, Tilt value and Zoom value of the camera corresponding to the first angle and the second angle respectively;

[0016] Based on the PSO algorithm, the newly added or modified control points and the first camera parameters, the optimal camera parameters are calculated for the video data at the second angle.

[0017] Optionally, the calculating the optimal camera parameters based on the particle swarm optimization (PSO) algorithm and combining the first camera parameters and control points includes:

[0018] Initialize the number of particle swarms, each particle in the particle swarm comprises a set of camera parameters, and the camera parameters comprise a first camera parameter and a second camera parameter;

[0019] Assigning parameter values ​​and speeds to the parameters of the first camera parameters of each particle for which precise values ​​are not defined and to the second camera parameters;

[0020] Calculate the fitness of each particle;

[0021] Iteratively update the particle parameters according to the fitness of the particle until the preset number of iterations is reached to obtain the optimal camera parameters, wherein the particle parameters include the individual extreme value pbest, the group extreme value gbest, the parameters and speed of the first camera parameter value of the particle whose exact value is not defined, and the second camera parameter value and speed.

[0022] Optionally, calculating the fitness of each particle includes:

[0023] Constructing a mapping model between video pixel coordinates and geographic space coordinates according to the first camera parameter value and the second camera parameter value of each particle;

[0024] Converting pixel coordinates in a configuration point pair into geospatial coordinates, or converting geospatial coordinates in a configuration point pair into pixel coordinates, wherein the configuration point pair includes pixel coordinates and geospatial coordinates;

[0025] The fitness is calculated by calculating the sum of squared distances between the transformed coordinates and the original coordinates.

[0026] Optionally, after obtaining the optimal camera parameters, the method further includes:

[0027] converting pixel coordinates in all configured point pairs into geospatial coordinates based on the optimal camera parameters;

[0028] Calculate the distance between the converted geospatial coordinates and the original geospatial coordinates in the configuration point pair respectively;

[0029] The sum of the distances between the converted geographic space coordinates of each configuration point pair and the original geographic space coordinates is taken as the error sum, and the average error value of each configuration point pair is obtained according to the error sum.

[0030] Optionally, the method further includes:

[0031] Extract the geospatial coordinates of all configured point pairs to generate source datasets;

[0032] Convert pixel coordinates in the configuration point pairs into geospatial coordinates based on the optimal camera parameters to generate a target data set;

[0033] The errors of the control points are evaluated according to the source data set and the target data set to optimize the control point marking process.

[0034] In a second aspect, the present application provides a video spatialization accuracy optimization system, the system comprising:

[0035] A data access unit, used to access video data, wherein the video data includes a real-time video stream or an offline video file;

[0036] a data input unit, used for inputting first camera parameters, wherein the first camera parameters include a horizontal field of view angle, a vertical field of view angle, a spatial coordinate X, a spatial coordinate Y, and a height, wherein inputting the first camera parameters includes inputting one or more of the first camera parameters;

[0037] A marking unit, used for marking control points on the video screen and the geographic space data screen of the video data;

[0038] The calculation unit is used to calculate the optimal camera parameters based on the particle swarm optimization PSO algorithm in combination with the first camera parameters and control points to optimize the video spatialization accuracy.

[0039] Optionally, the input first camera parameter includes:

[0040] Enter an exact value or a range value as the first camera parameter.

[0041] Optionally, when the video data is multi-angle video data, the marking unit marking the control points of the video picture of the video data includes:

[0042] Receiving video data shot at multiple angles, wherein the video data shot at multiple angles is obtained by adjusting a pan value, a tilt value, and a zoom value of a camera;

[0043] First, control points are marked on the video image and the geospatial data image at the first angle, and based on the PSO algorithm, the marked control points and the first camera parameters, the optimal camera parameters are calculated for the video data at each angle;

[0044] If the spatialization accuracy of the video at the second angle is less than the preset video accuracy threshold, re-mark the control points for the video data of the second angle, and when the Pan value, Tilt value and Zoom value of the first angle have not been input, input the Pan value, Tilt value and Zoom value of the camera corresponding to the first angle and the second angle respectively;

[0045] Based on the PSO algorithm, the newly added or modified control points and the first camera parameters, the optimal camera parameters are calculated for the video data at the second angle.

[0046] Optionally, the calculating unit calculates the optimal camera parameters based on a particle swarm optimization (PSO) algorithm in combination with the first camera parameters and control points, including:

[0047] Initialize the number of particle swarms, each particle in the particle swarm comprises a set of camera parameters, and the camera parameters comprise a first camera parameter and a second camera parameter;

[0048] Assigning parameter values ​​and speeds to the parameters of the first camera parameters of each particle for which precise values ​​are not defined and to the second camera parameters;

[0049] Calculate the fitness of each particle;

[0050] Iteratively update the particle parameters according to the fitness of the particle until the preset number of iterations is reached to obtain the optimal camera parameters, wherein the particle parameters include the individual extreme value pbest, the group extreme value gbest, the parameters and speed of the first camera parameter value of the particle whose exact value is not defined, and the second camera parameter value and speed.

[0051] Optionally, calculating the fitness of each particle includes:

[0052] Constructing a mapping model between video pixel coordinates and geographic space coordinates according to the first camera parameter value and the second camera parameter value of each particle;

[0053] Converting pixel coordinates in a configuration point pair into geospatial coordinates, or converting geospatial coordinates in a configuration point pair into pixel coordinates, wherein the configuration point pair includes pixel coordinates and geospatial coordinates;

[0054] The fitness is calculated by calculating the sum of squared distances between the transformed coordinates and the original coordinates.

[0055] Optionally, after obtaining the optimal camera parameters, the calculation unit is further used to:

[0056] converting pixel coordinates in all configured point pairs into geospatial coordinates based on the optimal camera parameters;

[0057] Calculate the distance between the converted geospatial coordinates and the original geospatial coordinates in the configuration point pair respectively;

[0058] The sum of the distances between the converted geographic space coordinates of each configuration point pair and the original geographic space coordinates is taken as the error sum, and the average error value of each configuration point pair is obtained according to the error sum.

[0059] Optionally, the computing unit is further used for:

[0060] Extract the geospatial coordinates of all configured point pairs to generate source datasets;

[0061] Convert pixel coordinates in the configuration point pairs into geospatial coordinates based on the optimal camera parameters to generate a target data set;

[0062] The errors of the control points are evaluated according to the source data set and the target data set to optimize the control point marking process.

[0063] In a third aspect, the present application provides a device, comprising a memory and a processor, the memory being used to store instructions or codes, and the processor being used to execute the instructions or codes, so that the device executes the video spatialization accuracy optimization method introduced in any implementation manner of the first aspect.

[0064] In a fourth aspect, the present application provides a computer-readable storage medium, in which codes are stored. When the codes are executed, a device executing the codes implements the video spatialization accuracy optimization method introduced in any implementation of the first aspect.

[0065] The present application provides a method for optimizing the accuracy of video spatialization. When executing the method, first access the video data, the video data includes a real-time video stream or an offline video file, and then input the first camera parameters, the first camera parameters include the horizontal field of view angle, the vertical field of view angle, the spatial coordinate X, the spatial coordinate Y and the height, and the input of the first camera parameters includes inputting one or more of the first camera parameters, and then marking the control points of the video screen and the geospatial data screen of the video data, and finally based on the particle swarm optimization PSO algorithm, the optimal camera parameters are calculated in combination with the first camera parameters and the control points to optimize the video spatialization accuracy. In this way, by performing video spatialization based on the PSO algorithm, the accuracy of video spatialization is improved, and the fusion of video data and geospatial data is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0067] Figure 1 A flowchart of a method for optimizing video spatialization accuracy provided in an embodiment of the present application;

[0068] Figure 2 A schematic diagram of first camera parameter settings provided in an embodiment of the present application;

[0069] Figure 3 A schematic diagram of a control point marking provided in an embodiment of the present application;

[0070] Figure 4 A schematic diagram of setting Pan value, Tilt value and Zoom value provided in an embodiment of the present application;

[0071] Figure 5 A video spatialization and geographic spatial data superimposed effect diagram provided in an embodiment of the present application;

[0072] Figure 6 A schematic diagram of the structure of a video spatialization accuracy optimization system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0073] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0074] Figure 1 A flowchart of a method for optimizing video spatialization accuracy provided in an embodiment of the present application. Figure 1 As shown, the video spatialization accuracy optimization method provided in the embodiment of the present application may include:

[0075] S101. Access video data, where the video data includes a real-time video stream or an offline video file.

[0076] Video datasets are created by using SuperMap iDesktopX software, which is a cross-platform 2D and 3D integrated desktop GIS software platform. SuperMap iDesktopX software supports access to video data, which can be real-time video streams or offline video files. Through this software, users can create video datasets and add the real-time video stream address of the camera or the storage path of the offline video file to this dataset to access video data. For real-time video streams, the software supports multiple protocols, such as RTSP, RTMP, HLS (m3u8), HTTP-FLV and HTTP-MP4; for offline video files, it supports multiple formats such as MP4, AVI, RMVB, FLV, MOV, etc. The accessed video dataset will be saved in the SuperMap data source for subsequent management and application.

[0077] S102: Input first camera parameters.

[0078] After the video dataset is created, open the video dataset to the video map window, and find the area for entering camera parameters in the corresponding settings or property panels of the video map window. Enter the first camera parameters, including horizontal field of view, vertical field of view, spatial coordinate X, spatial coordinate Y, and height. If one of the first camera parameters is unknown, do not enter the corresponding value. Among them, the horizontal field of view is also called HFOV, FovX, horizontal field of view = 2×arctan (w / 2f), w is the width of the image, which is the range of the scene that the camera or optical instrument can capture in the horizontal direction of the video, and f is the focal length of the lens. The vertical field of view is also called VFOV, FovY, vertical field of view = 2×arctan (h / 2f), where h is the height of the image, f is the focal length of the lens, and h is the range of the scene that the camera or optical instrument can capture in the vertical direction of the video. Spatial coordinates X and spatial coordinates Y are the coordinates of the geographical space where the camera is located. The height value can be the altitude or the relative height of the camera relative to the ground in the video screen. For each first camera parameter, you can enter a precise value or a range value. For example, if you don't know the exact value of the height, but know that it is approximately 20 to 30 meters, you can enter a value range of 20 to 30 meters.

[0079] like Figure 2 As shown, Figure 2 A schematic diagram of the first camera parameter setting is provided for an embodiment of the present application, wherein the first camera parameters include a horizontal field of view angle, a vertical field of view angle, a spatial coordinate X, a spatial coordinate Y and a height. Accurate values ​​are entered for the spatial coordinate X and the spatial coordinate Y in the first camera parameters, and a range value is entered for the height Z, and the range value corresponding to the height is reflected by entering a maximum value and a minimum value. If the values ​​of the horizontal field of view angle FovX and the vertical field of view angle FovY are unknown, no specific values ​​are entered. In addition, while configuring the first camera parameters, iterative parameters can also be configured, including the number of iterations and the number of samples. Whether the height of the geographic spatial coordinates in the control point pairs to be marked in the following method participates in the registration can also be set at the same time.

[0080] S103: marking control points on the video frame of the video data and the geographic space data frame.

[0081] Enable the registration mode for the video data, add the geospatial data as reference data and complete the marking of control points in the registration mode. Specifically, mark the control points in the video screen and the geospatial data screen respectively. The points marked in the video screen are called video pixels, and the points marked in the geospatial data screen are called geospatial coordinate points. The geospatial coordinate points can have or not have a height value. Before registration, you can set whether the geospatial coordinate height is involved in the registration. Figure 3 As shown, Figure 3 A control point marking schematic diagram provided for an embodiment of the present application, wherein the video point x-coordinate, the video point y-coordinate, the map point x-coordinate, and the map point y-coordinate constitute a set of registration point pairs, and the height may or may not participate in the registration according to the previous settings. The video point x-coordinate is the horizontal position (x-axis coordinate) of a feature point in the video screen. The video point y-coordinate is the vertical position (y-axis coordinate) of a feature point in the video screen. These two coordinates (x and y) together determine a specific point in the video screen. The map point x-coordinate is the horizontal position (x-axis coordinate) in the geospatial data (such as a map or GIS layer) corresponding to the video point. The map point y-coordinate is the vertical position (y-axis coordinate) in the geospatial data corresponding to the video point. These two coordinates (x and y) determine a specific point in the geospatial data corresponding to the video point.

[0082] When the video data is multi-angle video data, marking the control points of the video pictures of the video data includes:

[0083] Receive video data shot from multiple angles. These video data are obtained by adjusting the camera's pan, tilt, and zoom values ​​to cover different viewing angles and ranges. Pan, Tilt, Zoom, generally referred to as PTZ. It represents the camera's pan-tilt all-round (up and down, left and right) movement and lens zoom control, where P represents the camera's horizontal rotation angle, T corresponds to the vertical rotation angle, and Z is the camera's current focal length multiple.

[0084] Next, control points are marked for the video screen and geospatial data screen at one of the angles, referred to here as the first angle. Control points are corresponding points on the video screen and the geospatial data screen, which are used in subsequent calculations and alignment processes. Then, based on the particle swarm optimization (PSO) algorithm, the marked control points, and the initial camera parameters (referred to as the first camera parameters), the optimal camera parameters are calculated for the video data at each angle. These optimal camera parameters will make the alignment between the video data and the geospatial data more accurate. During the calculation process, if it is found that the video spatialization accuracy of a certain angle (referred to as the second angle) does not meet the preset accuracy requirements, for example, the accuracy value is less than a preset accuracy threshold, it is necessary to re-mark the control points for the video data at that angle. At the same time, enter the Pan value, Tilt value, and Zoom value of the camera corresponding to the angle, such as Figure 4 As shown, Figure 4A schematic diagram of setting Pan value, Tilt value and Zoom value (PTZ value for short) is provided for the embodiment of the present application to more accurately reflect the actual position and posture of the camera. When the spatialization accuracy of the second angle does not meet the requirements, when marking the control point for the second angle and entering the PTZ value, the PTZ value corresponding to the first angle also needs to be entered for the first angle for which the PTZ value is not entered.

[0085] Finally, based on the PSO algorithm, the newly added control points or the modified control points and the first camera parameters (or the camera parameters reinitialized as needed), the optimal camera parameters are calculated for the video data of the second angle. This process may be iterated multiple times until the preset accuracy requirements are met. By continuously adjusting the control points and camera parameters, the alignment between multi-angle video data and geospatial data is more accurate and reliable. For example, for an offline video with 10 angles, first mark the control points for any angle, combine PSO and camera parameters to calculate the best camera parameters, and then rotate to the 6th angle. It is found that the video accuracy at the 6th angle is very poor. Then, the control points can be marked for the 6th angle to improve the spatial accuracy of this angle. The other 8 angles are fine and there is no need to improve the accuracy.

[0086] When video data is presented in the form of online real-time video stream, it can be deeply connected with SuperMap iDesktopX software. Specifically, the interface provided by the software allows users to obtain and set the three key parameters of the camera: Pan, Tilt and Zoom. Through this interface, SuperMap iDesktopX software can directly control the rotation and zoom of the camera, eliminating the need for users to manually enter these parameter values.

[0087] In addition, in order to ensure all-round and accurate control of video data, at least 4 pairs of configuration points need to be set, which cover all possible angle marks of the video. These control points play a vital role in the registration process of video and geospatial data. They help the software to understand and respond to the dynamic changes of the camera more accurately, thereby achieving smoother and more accurate video control. By connecting to the interface of SuperMap iDesktopX software, online real-time video streaming can realize automatic control of the camera, while ensuring the accuracy and reliability of video control through a sufficient number of control points.

[0088] S104 . Based on a particle swarm optimization (PSO) algorithm, optimal camera parameters are calculated in combination with the first camera parameters and control points to optimize the spatialization accuracy of the video.

[0089] In the PSO algorithm, a group of particles is first initialized. Each particle represents a potential solution, that is, a set of camera parameters, which include the first camera parameters and the second camera parameters. The first camera parameters are the known parameters of the horizontal field of view angle, the vertical field of view angle, the spatial coordinate X, the spatial coordinate Y and the height, which can be an accurate value or a range value. The second camera parameters are unknown parameters, including the yaw angle, the pitch angle, the side slip angle, CenterX, and CenterY. The yaw angle is also called Yaw, which is the angle of rotation around the Y axis. It is analogous to the yaw of an airplane, that is, the left and right rotation of the head. The pitch angle is also called Pitch, which is the angle of rotation around the X axis. It is analogous to the pitch of an airplane, that is, the upward or downward rotation of the head. The side slip angle is also called Roll, which is the angle of rotation around the Z axis. It is analogous to the roll of an airplane, that is, the head rotates left or right; CenterX represents the horizontal position of the camera optical axis in the image coordinate system, in pixels; CenterY represents the vertical position of the camera optical axis in the image coordinate system, in pixels.

[0090] For each particle, the parameters of the first camera parameters that have not yet been precisely defined and the second camera parameters are initialized and assigned initial velocities. These velocities and parameter values ​​will be used as the position and movement direction of the particle in the search space. According to the preset fitness function, the fitness value of each particle is calculated. This value reflects the accuracy of the camera parameters represented by the particle when aligning video data with geospatial data, and is usually calculated based on the reprojection error of the control points.

[0091] When the particle swarm is iterated once or a particle is iterated, the fitness is calculated and then the individual extreme value pbest, the group extreme value gbest, the value and speed of the particle are updated. The results obtained by the two methods are basically the same. In the embodiment of the present application, when a particle is iterated, if the particle fitness is less than the fitness of the previous particle, the pbest, gbest, the value and speed of the particle are updated. The individual extreme value represents the best historical position of the particle itself, and the group extreme value represents the best position in the entire particle swarm. When the particle swarm is iterated once, the dynamic inertia weight is updated, and the formula is w = (wini-wend) * (Gmax-g) / Gmax + wend, where wini is the initial inertia weight, wend is the inertia weight iterated to the maximum number of times, Gmax is the maximum number of iterations, and g is the current number of iterations. According to the pre-set number of iterations, the particle swarm is iterated, and the iterative process will continue until the preset number of iterations is reached or other stop conditions are met. After the iteration is completed, the particle with the highest fitness value is selected from the particle swarm, and the camera parameters represented by it are the optimal camera parameters required. By optimizing the camera parameters, the spatial accuracy of the video data is optimized, and the final spatial effect is as follows: Figure 5 As shown, Figure 5 An embodiment of the present application provides an effect diagram of a video being spatialized and superimposed with geographic spatial data.

[0092] Through the above method, the PSO algorithm can be used to efficiently search for the optimal camera parameters in a complex parameter space, thereby optimizing the spatial accuracy of video data. This method is not only applicable to static video data, but also to online real-time video streams, providing strong technical support for video surveillance, remote sensing image processing and other fields.

[0093] In the above embodiment, calculating the fitness of each particle includes: constructing a mapping model from video pixel coordinates to geographic space coordinates according to the first camera parameter value and the second camera parameter value of each particle. This model can describe how any point in the video screen is mapped to the corresponding position in the geographic space. Using the mapping model, the pixel coordinates in the configuration point pair are converted to the geographic space coordinates, or the geographic space coordinates are converted to pixel coordinates. The configuration point pair is a set of known corresponding points, which contains the pixel coordinates in the video screen and the coordinates in the geographic space. It should be noted that, according to the actual application scenario and requirements, it may be necessary to perform a conversion from pixels to geographic space, or an inverse conversion from geographic space to pixels. The sum of the squares of the distances between the converted coordinates and the original coordinates is calculated as the fitness value of the particle. This sum of the squares of the distances reflects the accuracy of the mapping model, that is, the accuracy of the camera parameters represented by the particle in aligning the video data with the geographic space data. The smaller the fitness value, the closer the camera parameters of the particle are to the optimal solution, that is, the more accurate the mapping model is.

[0094] Through this process, a fitness value can be calculated for each particle to evaluate the accuracy of its camera parameters in the registration task. During the iteration of the PSO algorithm, the parameter values ​​and speeds of the particles will be updated according to these fitness values ​​to guide the particles to move to a better solution. Specifically, it includes updating the individual extreme value pbest of the particle, the group extreme value gbest, the parameters and speeds of the particles whose exact values ​​are not defined in the first camera parameter value, and the second camera parameter value and speed. For the camera parameters in the first camera parameter, if the exact value is not specified when the first camera parameter is initially entered, but a range value is specified, then when updating the particle parameters, the parameter value and speed will be updated like the second camera parameter. If the exact value is specified, the camera parameters will not be updated. Finally, the particle with the smallest fitness value can be selected from the particle swarm, and the camera parameters it represents are the optimal camera parameters required.

[0095] In an implementation method of an embodiment of the present application, after obtaining the optimal camera parameters, the method further includes evaluating the accuracy of the registration result through subsequent operations.

[0096] First, the pixel coordinates in all configuration point pairs are converted into geospatial coordinates using the obtained optimal camera parameters. This step is the application of the mapping model, which can accurately map the points in the video screen into the geographic space. Next, the distance between the converted geospatial coordinates and the original geospatial coordinates in each configuration point pair is calculated respectively. This distance reflects the accuracy of the mapping model on the point pair, that is, the deviation between the converted coordinates and the original coordinates. Then, the distances between the converted geospatial coordinates and the original geospatial coordinates of all configuration point pairs are added together to obtain the error sum. This error sum reflects the overall accuracy of the entire mapping model on all configuration point pairs. The error sum is divided by the number of configuration point pairs to obtain the average error value of each configuration point pair. This average error value is a more intuitive indicator, which represents the average deviation of the mapping model on each point pair.

[0097] Through these subsequent operations, the accuracy of the registration results can be quantitatively evaluated. The smaller the average error value, the more accurate the mapping model and the higher the quality of the registration results. This method provides an effective verification method for the precise registration of video data and geospatial data, which helps to ensure the reliability and accuracy of the registration results. At the same time, it also provides a basis for further optimization of camera parameters and mapping models, which can guide more in-depth research and improvement.

[0098] In an implementation method of an embodiment of the present application, the control point marking process can be optimized by further extending the registration process, including:

[0099] The geospatial coordinates of all configuration point pairs are extracted, and these coordinates serve as real-world reference points to form the source dataset. The source dataset is the basis for evaluating the accuracy of registration and optimizing the control point marking. Using the optimal camera parameters obtained, the pixel coordinates in the configuration point pairs are converted into geospatial coordinates to generate the target dataset. The target dataset is the geospatial coordinates obtained from the video screen by the mapping model, which reflects the accuracy of the mapping model. The error of the control points is evaluated based on the source dataset and the target dataset. This usually involves calculating the deviation between the coordinates in the target dataset and the corresponding coordinates in the source dataset, that is, the error. This error reflects the accuracy of the mapping model on the control points. Based on the error evaluation of the control points, the marking process of the control points can be optimized. For example, if the errors of some control points are large, it may be necessary to reselect these control points or adjust their marking positions to improve the accuracy of the mapping model. In addition, the error distribution can be used to guide the subsequent control point selection to ensure that they can evenly cover the entire video screen and geospatial area.

[0100] Through these extended steps, not only can the accuracy of the registration results be quantitatively evaluated, but the control point marking process can also be optimized based on the evaluation results. This optimization helps to improve the accuracy of the mapping model, thereby obtaining more accurate registration results. At the same time, it also provides a more reliable foundation for subsequent video data processing and geospatial analysis.

[0101] The above are some specific implementations of a method for optimizing video spatialization accuracy provided in the embodiment of the present application. Based on this, the present application also provides a corresponding system. The system provided in the embodiment of the present application will be introduced from the perspective of functional modularization.

[0102] Figure 6 A schematic diagram of the structure of a video spatialization precision optimization system provided in an embodiment of the present application. Figure 6 As shown, the video spatialization accuracy optimization system provided in the embodiment of the present application includes:

[0103] A data access unit 610 is used to access video data, wherein the video data includes a real-time video stream or an offline video file;

[0104] A data input unit 620 is used to input first camera parameters, wherein the first camera parameters include a horizontal field of view angle, a vertical field of view angle, a spatial coordinate X, a spatial coordinate Y, and a height, wherein inputting the first camera parameters includes inputting one or more of the first camera parameters;

[0105] A marking unit 630, used for marking control points on the video frame of the video data and the geographic space data frame;

[0106] The calculation unit 640 is used to calculate the optimal camera parameters based on the particle swarm optimization PSO algorithm in combination with the first camera parameters and control points to optimize the video spatialization accuracy.

[0107] In an implementation of the embodiment of the present application, the inputting the first camera parameter includes:

[0108] Enter an exact value or a range value as the first camera parameter.

[0109] In an implementation of the embodiment of the present application, when the video data is multi-angle video data, the marking unit marking the control points of the video screen of the video data includes:

[0110] Receiving video data shot at multiple angles, wherein the video data shot at multiple angles is obtained by adjusting a pan value, a tilt value, and a zoom value of a camera;

[0111] First, control points are marked on the video image and the geospatial data image at the first angle, and based on the PSO algorithm, the marked control points and the first camera parameters, the optimal camera parameters are calculated for the video data at each angle;

[0112] If the spatialization accuracy of the video at the second angle does not meet the preset video accuracy threshold, re-mark the control points for the video data of the second angle, and when the Pan value, Tilt value and Zoom value of the first angle have not been input, input the Pan value, Tilt value and Zoom value of the camera corresponding to the first angle and the second angle respectively;

[0113] Based on the PSO algorithm, the newly added or modified control points and the first camera parameters, the optimal camera parameters are calculated for the video data at the second angle.

[0114] In an implementation of the embodiment of the present application, the calculation unit calculates the optimal camera parameters based on a particle swarm optimization PSO algorithm in combination with the first camera parameters and the control points, including:

[0115] Initialize the number of particle swarms, each particle in the particle swarm comprises a set of camera parameters, and the camera parameters comprise a first camera parameter and a second camera parameter;

[0116] Assigning parameter values ​​and speeds to the parameters of the first camera parameters of each particle for which precise values ​​are not defined and to the second camera parameters;

[0117] Calculate the fitness of each particle;

[0118] Iteratively update the particle parameters according to the fitness of the particle until the preset number of iterations is reached to obtain the optimal camera parameters, wherein the particle parameters include the individual extreme value pbest, the group extreme value gbest, the parameters and speed of the first camera parameter value of the particle whose exact value is not defined, and the second camera parameter value and speed.

[0119] In one implementation of the embodiment of the present application, calculating the fitness of each particle includes:

[0120] Constructing a mapping model between video pixel coordinates and geographic space coordinates according to the first camera parameter value and the second camera parameter value of each particle;

[0121] Converting pixel coordinates in a configuration point pair into geospatial coordinates, or converting geospatial coordinates in a configuration point pair into pixel coordinates, wherein the configuration point pair includes pixel coordinates and geospatial coordinates;

[0122] The fitness is calculated by calculating the sum of squared distances between the transformed coordinates and the original coordinates.

[0123] In an implementation manner of the embodiment of the present application, after obtaining the optimal camera parameters, the calculation unit is further used to:

[0124] converting pixel coordinates in all configured point pairs into geospatial coordinates based on the optimal camera parameters;

[0125] Calculate the distance between the converted geospatial coordinates and the original geospatial coordinates in the configuration point pair respectively;

[0126] The sum of the distances between the converted geographic space coordinates of each configuration point pair and the original geographic space coordinates is taken as the error sum, and the average error value of each configuration point pair is obtained according to the error sum.

[0127] In one implementation of the embodiment of the present application, the computing unit is further configured to:

[0128] Extract the geospatial coordinates of all configured point pairs to generate source datasets;

[0129] Convert pixel coordinates in the configuration point pairs into geospatial coordinates based on the optimal camera parameters to generate a target data set;

[0130] The errors of the control points are evaluated according to the source data set and the target data set to optimize the control point marking process.

[0131] The embodiments of the present application also provide corresponding devices and computer storage media for implementing the solutions provided by the embodiments of the present application.

[0132] The device includes a memory and a processor, the memory is used to store instructions or codes, and the processor is used to execute the instructions or codes so that the device executes the method described in any embodiment of the present application.

[0133] The computer storage medium stores codes, and when the codes are executed, a device executing the codes implements the method described in any embodiment of the present application.

[0134] Through the description of the above implementation methods, it can be known that those skilled in the art can clearly understand that all or part of the steps in the above-mentioned embodiment method can be implemented by means of software plus a general hardware platform. Based on such an understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a storage medium, such as a read-only memory (ROM) / RAM, a magnetic disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network communication device such as a router) to execute the methods described in each embodiment of the present application or some parts of the embodiments.

[0135] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0136] It should also be noted that the various embodiments in this specification are described in a progressive manner, and the same and similar parts between the various embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiments. The device and system embodiments described above are merely schematic, in which the units described as separate components may or may not be physically separated, and the components indicated as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0137] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for optimizing video spatialization accuracy, characterized in that: The method comprises: Accessing video data, wherein the video data includes a real-time video stream or an offline video file; Inputting first camera parameters, wherein the first camera parameters include a horizontal field of view angle, a vertical field of view angle, a spatial coordinate X, a spatial coordinate Y, and a height, wherein inputting the first camera parameters includes inputting one or more of the first camera parameters; Marking control points on the video screen and the geographic space data screen of the video data; Based on the particle swarm optimization PSO algorithm, the optimal camera parameters are calculated in combination with the first camera parameters and control points to optimize the video spatialization accuracy.

2. The method according to claim 1, characterized in that The input first camera parameter includes: Enter an exact value or a range value as the first camera parameter.

3. The method according to claim 1, characterized in that When the video data is multi-angle video data, marking the control points of the video pictures of the video data includes: Receiving video data shot at multiple angles, wherein the video data shot at multiple angles is obtained by adjusting a pan value, a tilt value, and a zoom value of a camera; First, control points are marked on the video image and the geospatial data image at the first angle, and based on the PSO algorithm, the marked control points and the first camera parameters, the optimal camera parameters are calculated for the video data at each angle; If the spatialization accuracy of the video at the second angle is less than the preset video accuracy threshold, re-mark the control points for the video data of the second angle, and when the Pan value, Tilt value and Zoom value of the first angle have not been input, input the Pan value, Tilt value and Zoom value of the camera corresponding to the first angle and the second angle respectively; Based on the PSO algorithm, the newly added or modified control points and the first camera parameters, the optimal camera parameters are calculated for the video data at the second angle.

4. The method according to claim 1, characterized in that: The calculating of the optimal camera parameters based on the particle swarm optimization (PSO) algorithm and combining the first camera parameters and the control points comprises: Initialize the number of particle swarms, each particle in the particle swarm comprises a set of camera parameters, and the camera parameters comprise a first camera parameter and a second camera parameter; Assigning parameter values ​​and speeds to the parameters of the first camera parameters of each particle for which precise values ​​are not defined and to the second camera parameters; Calculate the fitness of each particle; Iteratively update the particle parameters according to the fitness of the particle until the preset number of iterations is reached to obtain the optimal camera parameters, wherein the particle parameters include the individual extreme value pbest, the group extreme value gbest, the parameters and speed of the first camera parameter value of the particle whose exact value is not defined, and the second camera parameter value and speed.

5. The method according to claim 4, characterized in that The calculation of the fitness of each particle includes: Constructing a mapping model between video pixel coordinates and geographic space coordinates according to the first camera parameter value and the second camera parameter value of each particle; Converting pixel coordinates in a configuration point pair into geospatial coordinates, or converting geospatial coordinates in a configuration point pair into pixel coordinates, wherein the configuration point pair includes pixel coordinates and geospatial coordinates; The fitness is calculated by calculating the sum of squared distances between the transformed coordinates and the original coordinates.

6. The method according to claim 5, characterized in that After obtaining the optimal camera parameters, the method further includes: converting pixel coordinates in all configured point pairs into geospatial coordinates based on the optimal camera parameters; Calculate the distance between the converted geospatial coordinates and the original geospatial coordinates in the configuration point pair respectively; The sum of the distances between the converted geographic space coordinates of each configuration point pair and the original geographic space coordinates is taken as the error sum, and the average error value of each configuration point pair is obtained according to the error sum.

7. The method according to claim 6, characterized in that The method further comprises: Extract the geospatial coordinates of all configured point pairs to generate source datasets; Convert pixel coordinates in the configuration point pairs into geospatial coordinates based on the optimal camera parameters to generate a target data set; The errors of the control points are evaluated according to the source data set and the target data set to optimize the control point marking process.

8. A video spatialization accuracy optimization system, characterized in that: The system comprises: A data access unit, used to access video data, wherein the video data includes a real-time video stream or an offline video file; a data input unit, used for inputting first camera parameters, wherein the first camera parameters include a horizontal field of view angle, a vertical field of view angle, a spatial coordinate X, a spatial coordinate Y, and a height, wherein inputting the first camera parameters includes inputting one or more of the first camera parameters; A marking unit, used for marking control points on the video screen and the geographic space data screen of the video data; The calculation unit is used to calculate the optimal camera parameters based on the particle swarm optimization PSO algorithm in combination with the first camera parameters and control points to optimize the video spatialization accuracy.

9. A computing device, characterized in that The computing device includes: a memory and a processor; The memory is used to store computer programs; The processor is configured to implement the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images