A resource scheduling method for image reconstruction algorithm
By breaking down the image reconstruction task into subtasks and analyzing resource requirements, and combining this with an intelligent scheduling model to optimize resource allocation, the problem of unbalanced resource scheduling in image reconstruction algorithms is solved, thereby improving reconstruction efficiency and resource utilization.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- WUHAN UNIV
- Filing Date
- 2023-03-13
- Publication Date
- 2026-05-19
AI Technical Summary
Existing image reconstruction algorithms suffer from problems such as computational complexity, varying resource requirements, and numerous limitations of computing platforms, resulting in low reconstruction efficiency.
The image reconstruction task is divided into three major sub-tasks: dense reconstruction, surface reconstruction, and texture reconstruction. The resource requirements of each sub-task are calculated, and intelligent scheduling is performed through a resource scheduling model. Heuristic and deep learning algorithms are combined to optimize resource allocation.
It improves the efficiency and resource utilization of image reconstruction, is applicable to various resource types and computing platforms, and ensures the accuracy and stability of reconstruction results.
Smart Images
Figure CN116401044B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of 3D reconstruction technology, specifically relating to a resource scheduling method for image reconstruction algorithms. Background Technology
[0002] With the development of information technology, the era of big data has arrived, and the demand for high-performance algorithms is increasing daily due to the massive amount of data. Image reconstruction, which uses multi-view image matching algorithms to reconstruct 3D models of experimental areas, has been widely used in fields such as cultural relic preservation. However, due to the continuous growth in the scale of image data and the huge computational load of reconstruction algorithms, data processing time has increased significantly. How to reconstruct images efficiently has become a major concern in the industry.
[0003] Resource scheduling, or the rational allocation of computing resources, is one of the effective means to improve data processing efficiency. In the field of cloud computing, there are already scheduling algorithms applicable to various computing resources, including resource scheduling algorithms based on classical heuristics and resource scheduling algorithms based on deep reinforcement learning. Currently, resource scheduling for image reconstruction algorithms still faces the following challenges: 1) The image reconstruction processing flow is complex, and the computational requirements of each step are different; 2) Data scenarios are diverse and varied, with different resource requirements for different scenarios; 3) There are many types of computing resources, and their actual combinations are diverse. To address these issues, there is currently a lack of a resource scheduling method suitable for image reconstruction algorithms to improve the efficiency of 3D reconstruction. Summary of the Invention
[0004] To address the shortcomings of existing technologies, this invention provides a resource scheduling method for image reconstruction algorithms, which solves the problem of uneven allocation of computational resources in image reconstruction algorithms and improves image reconstruction efficiency.
[0005] To achieve the above objectives, the present invention provides a resource scheduling method for image reconstruction algorithms, comprising the following steps:
[0006] Step 1: Decompose the image reconstruction task into three categories: dense reconstruction subtask, surface reconstruction subtask, and texture reconstruction subtask, and calculate the resource requirements of each type of subtask.
[0007] Step 1.1: Divide the reconstruction area into spatial tiles, and then complete the dense matching of each tile area by filtering matching image pairs, calculating the depth information of matching image pairs, and fusing the depth information. Calculate the resource requirements of the dense reconstruction subtask based on the tile area, the number of image pairs within the tile, the depth information calculation resources, the depth information fusion resource requirements, and the algorithm attributes.
[0008] Step 1.2: Surface reconstruction is performed using the tiles divided in the dense reconstruction subtask of Step 1.1 as units. First, the dense point cloud in each tile is divided into blocks. Then, the point cloud in each block is constructed using the visual graph cut triangular mesh construction method. After the mesh is constructed, the meshes are merged. Finally, the resource requirements of the surface reconstruction subtask are calculated based on the tile size, the number of dense point cloud blocks in the tile, the number of point clouds to be constructed, and the algorithm attributes.
[0009] Step 1.3: Cluster the triangular mesh surfaces constructed by the surface reconstruction subtask in Step 1.2, and then select the corresponding images for texture generation. Calculate the resource requirements of the texture reconstruction subtask based on the number of triangular mesh points, the number of triangular mesh surfaces, the number of selected images, and the image size.
[0010] Step 2: Based on the resource requirements of various sub-tasks obtained in Step 1, and in conjunction with the existing resource types and quantities... The system calculates the resource requirements of various sub-tasks under different resource types and operating environments. Based on the urgency of the image reconstruction task, as well as the actual energy consumption and estimated completion time, it constructs a resource scheduling model to intelligently schedule various resources.
[0011] Furthermore, the dense reconstruction subtask in step 1.1 Resource demand analysis is conducted, and the location information of the area to be reconstructed is obtained based on the connection point information of aerial triangulation or other location information. Utilizing the segmented area Use uniform grid or quadtree partitioning method to Divide the space into tiles. Due to the different sizes... Received The different division results lead to different numbers of tiles being reconstructed. k Different points have different computational resource requirements, therefore different sizes of [aspects / scales] are used before dense point cloud matching. right Different tile sizes are segmented and resource requirements are calculated to select the appropriate tile segmentation scale based on hardware performance.
[0012] use right Perform segmentation to obtain the segmentation results. Then, by filtering matching image pairs, calculating the depth information of the matching image pairs, and fusing the depth information, a dense matching process for each segmented region is completed. This is done sequentially. Number of densely matched image pairs required for the region And the resource requirements for depth information computation for each matched image pair. Calculate the maximum resource requirements for depth information computation for each image pair. You can get Resource requirements for calculating depth information when matching image pairs in a region. The depth information calculation attribute is set to GPU, based on the image width of each image pair. ,high Estimate the size of the image depth information and calculate the sum of the depth information of all images in the region to obtain the resources required for depth information fusion in the region. .when , When kept constant, statistics When a value significantly larger than the average distribution appears in the numerical distribution, it indicates a large amount of image data in that tile area. The area needs to be spatially subdivided again, and the resource requirements of the subdivided tiles need to be calculated until the resource requirements are close to the average resource requirements in the numerical distribution. Finally, based on the tile size... Number of images within the tiles Deep information computing resources Deep information fusion resource requirements Computationally intensive reconstruction subtasks based on algorithm attributes The resource demand, namely: .
[0013] Furthermore, step 1.2 uses tiles divided before dense reconstruction. For surface reconstruction of individual tiles, the densely reconstructed point cloud undergoes filtering and downsampling preprocessing, followed by the generation of triangular meshes using a visual graph cut triangular mesh construction method. Due to the large volume of dense point cloud data in each tile, it is necessary to first divide the dense point cloud into blocks, then construct a mesh for each block, and finally merge the meshes. Therefore, the resource requirements for the mesh construction process are calculated based on the number of dense point clouds and the number of blocks, and subsequently, the resource requirements for merging the meshes are also calculated. The resource requirements for the surface reconstruction subtask are determined based on the tile size. Number of dense point clouds within a tile Number of point clouds to be constructed And algorithm attribute calculation, that is: ...
[0014] Furthermore, in step 1.3, texture mapping involves clustering the flatness of the triangular mesh surfaces constructed in step 1.2 through surface reconstruction, and then selecting the corresponding images for texture generation. Therefore, while considering the resource requirements of the triangular mesh, it is necessary to incorporate image resource requirements by selecting the number and size of images, thereby calculating the total resource requirements. .
[0015] Furthermore, in step 2, based on the image reconstruction task Resource requirements for various sub-tasks Combining existing resource types and the number of resources ,calculate Different types of resources Resource requirements in the operating environment According to the task Based on the urgency level, actual energy consumption, and estimated completion time, a resource scheduling model is constructed to ensure that the usage of each type of resource is less than the total resource amount, i.e.:
[0016] (1)
[0017] In the formula, This indicates the maximum resource utilization rate, with a value less than 1.
[0018] The resource scheduling model is constructed and optimized based on the characteristics of the reconstruction algorithm and combined with heuristic algorithms or deep learning reinforcement algorithms, so as to intelligently allocate various resources according to the adaptive data characteristics and resource attributes of the reconstruction task.
[0019] Compared with the prior art, the present invention has the following advantages:
[0020] 1) The method proposed in this invention can intelligently split the reconstruction task and adapt different resources to the split sub-tasks, avoiding the uniform allocation of resources for the entire task, and can make reasonable use of resources to improve reconstruction efficiency and resource utilization.
[0021] 2) The method proposed in this invention is applicable to a combination of various resource types. The algorithm strategy can be adjusted according to different resource configurations, reducing the limitations of the computing platform on the reconstruction task and improving the universality of the reconstruction algorithm on the computing platform.
[0022] 3) The method proposed in this invention is applicable to image sets acquired by multiple platforms and multiple sensors, and can adaptively divide tasks for various scenes to ensure the correctness and stability of reconstruction results. Attached Figure Description
[0023] Figure 1 This is a flowchart illustrating the resource scheduling process according to an embodiment of the present invention. Detailed Implementation
[0024] This invention provides a resource scheduling method for image reconstruction algorithms. This method breaks down the image reconstruction task into subtasks, calculates the resource requirements of each subtask, and intelligently allocates computing resources based on the number of subtasks and resource requirements, combined with algorithm characteristics and hardware performance. This optimizes the utilization of computing resources, changes the strategy of simply increasing the number of hardware devices to improve computing power, and improves reconstruction efficiency.
[0025] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.
[0026] like Figure 1As shown, this invention provides a resource scheduling method for image reconstruction algorithms, comprising the following steps:
[0027] Step 1: Divide the image reconstruction task into subtasks and calculate the resource requirements of each subtask.
[0028] Input task ,in This is a reconstruction task based on an independent image dataset, given the existing image dataset and corresponding aerial triangulation results. The task requirement is to reconstruct a 3D model of the dataset. The reconstruction task is divided into three subtasks: dense reconstruction, surface reconstruction, and texture mapping. The dependencies between subtasks can be represented by a directed acyclic graph. Indicated. Utilizing the reconstruction subtask. Image attributes Calculation of aerial triangulation results resource requirements Image attributes It includes attributes such as the number of images and the size of the images.
[0029] Step 1.1: Perform resource requirement analysis on the dense reconstruction subtask.
[0030] First of all Subtask intensive reconstruction Resource requirements analysis is performed. Since dense reconstruction demands massive computational resources, the reconstruction area needs to be divided into multiple sub-regions to avoid excessive resource requirements leading to reconstruction task failure. The location information of the area to be reconstructed is obtained based on the connection point information from aerial triangulation or other location information. Utilizing the segmented area Elements such as uniform grids or quadtrees are used for partitioning. Divide the space into different sizes of tiles. Received The different division results lead to different numbers of tiles being reconstructed. k Different points have different computational resource requirements, therefore different sizes of [specific parameters] need to be used before dense point cloud matching. right Different tile sizes are segmented and resource requirements are calculated to select the appropriate tile segmentation scale based on hardware performance.
[0031] use right Perform segmentation to obtain the segmentation results. Then, by filtering matching image pairs, calculating the depth information of the matching image pairs, and fusing the depth information, a dense matching process for each segmented region is completed. This is done sequentially. Number of densely matched image pairs required for the region And the resource requirements for calculating depth information for each matched image pair, such as using PatchMatch and its variants, based on the image width. ,high Parallax propagation method and the number of adjacent images Calculate the resource requirements for depth information computation for each matched image pair. Determine the maximum resource requirements for depth information computation for each image pair. You can get Resource requirements for calculating depth information of matched image pairs in a region. Since calculating depth information for matched image pairs would significantly improve efficiency on a GPU, the depth information calculation attribute is set to GPU. This is based on the image width of each image pair. ,high Estimate the size of the image depth information and calculate the sum of the depth information of all images in the region to obtain the resources required for depth information fusion in the region. .when , When kept constant, statistics When a value is much larger than the average distribution, it indicates that the image data volume of the tile area is large. For example, this problem may occur when processing aerial-ground fusion reshoot data. It is necessary to divide the area into spatial subdivisions again and calculate the resource demand of the tile after subdivision until the resource demand is close to the average resource demand in the numerical distribution.
[0032] After the above operations, the resource requirements for dense matching have been calculated. The resource requirements for the dense reconstruction subtask are based on the tile size. Number of images within the tiles Deep information computing resources Deep information fusion resource requirements And algorithm attribute calculation, that is: .
[0033] Step 1.2: Perform resource requirement analysis for the surface reconstruction subtask.
[0034] Tiles divided in the dense reconstruction subtask For surface reconstruction of individual units, the densely reconstructed point cloud undergoes preprocessing operations such as filtering and downsampling, and then a triangular mesh is generated using the visual graph cut triangular mesh construction method. Due to the large amount of dense point cloud data within each tile, it is necessary to first divide the dense point cloud into blocks, then construct a mesh for each block, and finally merge the meshes. Therefore, the resource requirements for the mesh construction process can be calculated by considering the number of dense point clouds within each block and the number of blocks, and thus the resource requirements for merging the meshes. Surface reconstruction resources are determined based on the tile size. Number of dense point clouds within a tile Number of point clouds to be constructed And algorithm attribute calculation: .
[0035] Step 1.3: Perform resource requirement analysis for the texture reconstruction subtask.
[0036] Texture mapping involves clustering the flatness of the triangular mesh surfaces constructed in step 1.2 through surface reconstruction, and then selecting the corresponding images for texture generation. Therefore, while considering the resource requirements of the triangular mesh, it is necessary to incorporate image resource requirements by selecting the number and size of images, and then calculate the resource requirements. .
[0037] Step 2: Construct a resource scheduling model to make intelligent decisions and allocate various resources.
[0038] Based on image reconstruction task Subtask resource requirements Combining existing resource types and the number of resources ,calculate Different types of resources Resource requirements in the operating environment According to the task Based on the urgency level, actual energy consumption, and estimated completion time, a resource scheduling model is constructed, ensuring that the usage of each type of resource is less than the total resource amount, i.e.:
[0039] (1)
[0040] In the formula, This indicates the maximum utilization rate of resources, and the value is less than 1.
[0041] The resource scheduling model is constructed and optimized based on the characteristics of the reconstruction algorithm and combined with heuristic algorithms or deep learning reinforcement algorithms. In this way, it can adapt to the data characteristics and resource attributes of the reconstruction task to achieve the purpose of intelligently allocating resources, improving resource utilization, and increasing reconstruction efficiency.
[0042] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to replace them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.
Claims
1. A resource scheduling method for image reconstruction algorithms, characterized in that, Includes the following steps: Step 1, Image Reconstruction Task The task is broken down into three subtasks: dense reconstruction, surface reconstruction, and texture reconstruction. The resource requirements for each subtask are then calculated. ; Step 1.1: Divide the reconstructed region into spatial tiles. Then, complete the dense matching of each tile region by selecting matching image pairs, calculating the depth information of the matching image pairs, and fusing the depth information. Calculate the resource requirements of the dense reconstruction subtask based on the tile area, the number of image pairs within the tile, the depth information computation resources, the depth information fusion resource requirements, and the algorithm attributes. ; Step 1.2: Surface reconstruction is performed using the tiles divided in the dense reconstruction subtask of Step 1.1 as units. First, the dense point cloud within each tile is divided into blocks. Then, a mesh is constructed for each point cloud block using the visual graph cut triangular mesh construction method. After the mesh is constructed, the meshes are merged. Finally, the resource requirements of the surface reconstruction subtask are calculated based on the tile size, the number of dense point cloud blocks within the tile, the number of point clouds to be meshed, and the algorithm attributes. ; Step 1.3: Cluster the triangular mesh faces constructed in the surface reconstruction subtask of Step 1.2 based on their flatness, then select the corresponding images for texture generation. Calculate the resource requirements of the texture reconstruction subtask based on the number of triangular mesh points, the number of triangular mesh faces, the number of selected images, and their sizes. ; Step 2: Based on the resource requirements of various sub-tasks obtained in Step 1, and in conjunction with the existing resource types and quantities... The system calculates the resource requirements of various sub-tasks under different resource types and operating environments. Based on the urgency of the image reconstruction task, as well as the actual energy consumption and estimated completion time, it constructs a resource scheduling model to intelligently schedule various resources.
2. The resource scheduling method for image reconstruction algorithms as described in claim 1, characterized in that: Step 1.1 involves the dense reconstruction subtask. Resource demand analysis is conducted, and the location information of the area to be reconstructed is obtained based on the connection point information of aerial triangulation or other location information. Utilizing the segmented area Use uniform grid or quadtree partitioning method to Dividing the space into tiles, due to different sizes Received The different division results lead to different numbers of tiles being reconstructed. k Different points have different computational resource requirements, therefore different sizes of [aspects / scales] are used before dense point cloud matching. right Different tile sizes are segmented and resource requirements are calculated to select the appropriate tile segmentation scale based on hardware performance.
3. The resource scheduling method for image reconstruction algorithms as described in claim 2, characterized in that: Step 1.1 utilizes right Perform segmentation to obtain the segmentation results. Then, by filtering matching image pairs, calculating the depth information of matching image pairs, and fusing depth information, the dense matching process of each segmented region is completed. Number of densely matched image pairs required for the region The resource requirements for depth information computation for each matched image pair are calculated, and the maximum resource requirements for depth information computation for each image pair are determined. You can get Resource requirements when calculating depth information for matching image pairs in a region.
4. The resource scheduling method for image reconstruction algorithms as described in claim 3, characterized in that: In step 1.1, the depth information calculation attribute is set to GPU, based on the image width of each image pair. ,high Estimate the size of the image depth information and calculate the sum of the depth information of all images in the region to obtain the resources required for depth information fusion in the region. ;when , When kept constant, statistics When a value significantly larger than the average distribution appears in the numerical distribution, it indicates a large amount of image data in that tile area. The area needs to be spatially subdivided again, and the resource requirements of the subdivided tiles need to be calculated until the resource requirements are close to the average resource requirements in the numerical distribution. Finally, based on the tile size... Number of images within the tiles Deep information computing resources Deep information fusion resource requirements Computationally intensive reconstruction subtasks based on algorithm attributes The resource demand, namely: .
5. The resource scheduling method for image reconstruction algorithms as described in claim 4, characterized in that: Step 1.2 uses the tiles divided before dense reconstruction For surface reconstruction of individual units, the densely reconstructed point cloud undergoes filtering and downsampling preprocessing, followed by the generation of triangular meshes using a visual graph cut triangular mesh construction method. Due to the large amount of dense point cloud data in each tile, it is necessary to first divide the dense point cloud into blocks, then construct a mesh for each block, and finally merge the meshes. Therefore, the resource requirements for the mesh construction process are calculated based on the number of dense point clouds and the number of blocks, and subsequently, the resource requirements for merging the meshes are also calculated. The resource requirements for the surface reconstruction subtask are based on the tile size. Number of dense point clouds within a tile Number of point clouds to be constructed And algorithm attribute calculation, that is: .
6. The resource scheduling method for image reconstruction algorithms as described in claim 1, characterized in that: In step 2, based on the image reconstruction task Subtask resource requirements Combining existing resource types and the number of resources ,calculate Different types of resources Resource requirements in the operating environment According to the task Based on the urgency level, actual energy consumption, and estimated completion time, a resource scheduling model is constructed to ensure that the usage of each type of resource is less than the total resource amount, i.e.: （1） In the formula, This indicates the maximum resource utilization rate, with a value less than 1. The resource scheduling model is constructed and optimized based on the characteristics of the reconstruction algorithm and combined with heuristic algorithms or deep learning reinforcement algorithms, so as to intelligently allocate various resources according to the adaptive data characteristics and resource attributes of the reconstruction task.