MaaS model dynamic deployment method based on GPU resources

Abstract

This invention relates to the field of GPU technology, specifically disclosing a method for dynamic deployment of MaaS models based on GPU resources, including the following steps: establishing recognition models for different types of road defects based on deep learning; acquiring sub-road segments and assigning priority levels to them; determining target images based on the priority levels of the sub-road segments; classifying the target images and obtaining the GPU memory usage of the recognition models; and allocating GPUs to the recognition models based on the GPU memory usage for them to identify road defects in the target images. This invention improves the flexibility of model deployment.

Description

Technical Field

[0001] This invention relates to the field of GPU technology, and more specifically to a method for dynamically deploying MaaS models based on GPU resources. Background Technology

[0002] In the road defect inspection business, road images are currently collected by inspection vehicles and uploaded to the data center. The data center needs to analyze and identify a large number of images to determine whether there are abnormalities such as potholes, cracks, or signs being contaminated.

[0003] Traditional model deployment methods have many problems when processing such large-scale image data. Fixed model deployments cannot flexibly adjust resource allocation according to fluctuations in actual business volume, resulting in resource waste during off-peak periods and task backlogs due to insufficient resources during peak periods. In addition, different types of road defect detection have different model requirements, and a single fixed model is difficult to adapt to complex and ever-changing detection scenarios. Summary of the Invention

[0004] The purpose of this invention is to provide a method for dynamically deploying MaaS models based on GPU resources, thereby solving the aforementioned technical problems.

[0005] The objective of this invention can be achieved through the following technical solutions:

[0006] The method for dynamically deploying MaaS models based on GPU resources includes the following steps:

[0007] Based on deep learning, identification models for different types of road defects are established. These models are used to determine whether the corresponding road defects exist.

[0008] Obtain all road intersections in the inspection area and mark the road segments between two adjacent road intersections as sub-segments;

[0009] Based on the road defect data of sub-segments and the degree of impact of different types of road defects on road safety, the priority levels of sub-segments are divided into high, medium and low priority levels.

[0010] The priority level of the sub-road segment is used as the priority level of the images acquired on the sub-road segment, and the images with the higher priority level are used as the target images;

[0011] The target image is classified, with one classification corresponding to one recognition model. The GPU memory usage of the recognition model when recognizing road defects in a target image of the corresponding classification is obtained.

[0012] A GPU resource pool is built to manage GPUs, and GPUs are allocated to the recognition model based on GPU memory usage so that the recognition model can identify road defects in the target image.

[0013] As a further aspect of the present invention: the priority levels for dividing sub-road segments include:

[0014] The impact value is obtained by normalizing the degree of impact.

[0015] Extract the number of times the same type of road defect occurred in the past n inspections from the road defect data, where n is a preset number. Multiply the number by the corresponding impact value to get the target number. Sum the target numbers for a sub-segment to get the sorted number.

[0016] The sub-segments are sorted in descending order according to their corresponding ranking numbers. Starting from the first position in the ranking, the highest level is used as the priority level for the top 20% of the sub-segments in the ranking, the medium level is used as the priority level for the top 20% to 50% of the sub-segments in the ranking, and the lowest level is used as the risk level for the remaining sub-segments in the ranking.

[0017] As a further aspect of the present invention: classifying the target image includes:

[0018] The target image is segmented into ROI regions, and the visual features of the ROI regions are extracted as real-time features. Target images with the same real-time features are classified as a single category.

[0019] Visual features of different types of road defects are preset and denoted as standard features. If the real-time feature corresponding to category b1 is the same as the standard feature b2, then the recognition model corresponding to the standard feature b2 is used as the recognition model corresponding to category b1.

[0020] As a further aspect of the present invention: allocating GPU resources to the recognition model based on GPU memory usage includes:

[0021] The GPUs in the GPU resource pool are sorted in descending order of total video memory to obtain the first sort; the GPUs are then sorted in descending order of video memory usage to obtain the second sort.

[0022] Obtain the total video memory DY1 of the GPU at the top of the first sort, and determine the stage position i in the second sort. Stage position i satisfies the following constraints:

[0023] D i+1 ≥0.8DY1 and D i ≤0.8DY1, where Di represents the sum of the GPU memory usage of the first i GPUs in the second sort, starting from the first position in the second sort;

[0024] Then, the GPU at the top of the first sort is designated as the target GPU for the recognition model corresponding to the GPU memory usage of the first to the i-th positions in the second sort, and is used by the recognition model corresponding to the GPU memory usage of the first to the i-th positions in the second sort.

[0025] Remove the top GPU from the first sort to get a new first sort, and remove the memory usage of the first i GPUs from the second sort to get a new second sort;

[0026] Identify the target GPU and its corresponding recognition model, and repeat the above steps until the sum of the memory usage of all GPUs in a new second ranking is less than the total memory usage of the first GPU in the corresponding new first ranking.

[0027] As a further aspect of the present invention: identifying road defects in a target image includes:

[0028] Based on the target GPU, the corresponding recognition model is loaded, the target image is input into the corresponding recognition model, and the output is whether the target image contains the road defects corresponding to the recognition model.

[0029] As a further aspect of the present invention, identifying road defects in the target image also includes:

[0030] The system monitors the utilization rate of the target GPU in real time. If the utilization rate is greater than 80%, the corresponding identification model is identified as an abnormal model, and the target GPU is marked as an abnormal GPU.

[0031] The occupancy rate of each abnormal model is obtained in real time, and the abnormal models are sorted in descending order according to the occupancy rate to obtain the third sort.

[0032] Sort the target GPUs in ascending order of utilization to obtain the fourth sort;

[0033] Obtain the first abnormal model and the first target GPU in the third and fourth sorts respectively. Load the first abnormal model based on the first target GPU. After loading, the first abnormal model will no longer run on the abnormal GPU.

[0034] As a further aspect of the present invention: if the utilization rate of the target GPU at the top of the fourth sorting is higher than 50%, a prompt message is sent to the preset administrator.

[0035] As a further aspect of the present invention: establishing identification models for different road defects includes:

[0036] For road defect a, a database is established, which stores images of road defect a based on manual annotations.

[0037] A recognition model is established based on deep learning, and the recognition model is trained and validated based on a database to obtain the recognition model of road defect a.

[0038] The beneficial effects of this invention are:

[0039] 1) This invention constructs a GPU resource pool and adopts a dynamic allocation strategy that sorts memory demand and total memory usage, enabling multiple models to be loaded and run on demand. Idle memory is automatically reclaimed during business downtime, and models are fully stacked in priority during business peaks to improve concurrent processing capabilities. This achieves a dynamic balance between resource utilization and task processing, while avoiding the maintenance burden and potential risks caused by frequent manual adjustments to equipment configuration.

[0040] 2) This invention sorts and assigns grades to sub-road segments based on historical disease frequency and safety impact value. First, only high-grade images are used as target images and quickly matched to the corresponding models according to visual features to complete the recognition. After the high-grade tasks are processed, medium-grade and low-grade images are taken over in turn according to the same rules. The loaded models are continuously reused and the use of video memory is dynamically adjusted. This ensures that high-risk areas are dealt with first and fully releases idle computing power to complete subsequent detection. Overall, it achieves precise coupling between risk classification and computing resources.

[0041] 3) This invention sets up a real-time GPU utilization monitoring and abnormal model migration mechanism. When the GPU load is detected to be too high, the model is automatically migrated to a low-load GPU and an alarm is triggered when necessary to avoid recognition delay and service interruption caused by single-point congestion. During the migration process, the model and data status are kept consistent to ensure continuous and accurate recognition results. This enables highly available and scalable road defect detection services under complex business fluctuations and equipment differences, further reducing manual troubleshooting time and improving the system's self-healing capabilities. Attached Figure Description

[0042] The invention will now be further described with reference to the accompanying drawings.

[0043] Figure 1 This is a flowchart illustrating the dynamic deployment method of the MaaS model based on GPU resources according to the present invention. Detailed Implementation

[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] Please see Figure 1 As shown, this invention is a method for dynamically deploying MaaS models based on GPU resources, including the following steps:

[0046] Step 1: Establish identification models for different types of road defects;

[0047] In a preferred embodiment of the present invention, establishing an identification model for different road defects includes:

[0048] For road defect a, a database is established, which stores images of road defect a based on manual annotations.

[0049] A recognition model is established based on deep learning, and the recognition model is trained and validated based on a database to obtain the recognition model of road disease a.

[0050] Step 2: Divide the road into sub-segments and determine the priority level of different sub-segments;

[0051] In a preferred embodiment of the present invention, the priority level for dividing sub-road segments includes:

[0052] First, take the intersections in the road network as natural dividing points, extract the coordinates of all intersections on the inspection map in sequence and connect them according to the road topology order. The continuous road segments between two adjacent intersections are marked as a sub-segment. For example, intersection A to B forms sub-segment AB.

[0053] The database of previous inspections retrieved the defect entries associated with each sub-section of the road. The impact of different defects on driving safety was converted to the same scale, i.e., normalized, so that the risk assessment values ​​with different sources, dimensions and evaluation criteria were mapped to a unified range, which is convenient for subsequent comparison.

[0054] By retrieving the past n inspection records for each sub-segment, the number of times the same disease appears in that sub-segment is counted. This frequency, which naturally reflects the activity level of the disease, is multiplied by the normalized impact value of the corresponding disease to obtain the target number of the disease in that sub-segment. This target number can be understood as a "frequency-harm" joint weight: the more frequently it appears and the greater the harm, the higher the product, thus highlighting the hidden dangers that need to be given priority.

[0055] The total number of defects in the same sub-segment is summed to obtain the comprehensive ranking number, which comprehensively reflects the recent risk accumulation of the sub-segment. The ranking numbers of all sub-segments are listed and sorted in descending order. The larger the value, the more concentrated the risk.

[0056] To keep the grading thresholds relatively stable without over-segmentation, a simple proportional segmentation method can be used: start from the top of the list and sequentially take out sub-segments until they accumulate to the top one-fifth of the total list, then mark these as high-level; continue taking out the next half of the total list and mark this interval as medium-level; the remaining sub-segments are classified as low-level.

[0057] Step 3: Use the priority level of the sub-segment as the priority level of the images acquired on the sub-segment, and use the images with the higher priority level as the target images;

[0058] The target image is classified, with one classification corresponding to one recognition model. The GPU memory usage of the recognition model when recognizing road defects in a target image of the corresponding classification is obtained.

[0059] It is important to note that only all high-level images are extracted as target images for the next processing stage, thus focusing on processing the content most likely to affect traffic safety and avoiding diluting computing resources. After the high-level images are processed, the medium-level and low-level images are processed in the same way, which will not be elaborated here.

[0060] If the inspection vehicle acquires new images between the time when high-level images are started and the time when low-level images are finished, then the high-level, medium-level, and low-level processing will be repeated on these images until the inspection is completed.

[0061] Another preferred embodiment of the present invention includes classifying target images as follows:

[0062] Locating potentially damaged areas at the pixel level involves segmenting the image into regions of interest (ROIs). Visual cues such as edge gradients, texture distribution, or color anomalies are used to crop the entire image into one or more local ROIs. For example, in a photo of a road surface containing potholes, rectangles or polygons are selected around the depressions and cracks. This reduces background interference and narrows the computational scope for subsequent feature extraction.

[0063] For each ROI, a visual descriptor is calculated, such as directional gradient distribution, crack morphology curve, or pit depth shadow morphology. These descriptors are combined into real-time features. ROIs with the same or highly similar real-time features (similarity greater than 80%) represent ROIs that may have the same type of disease.

[0064] To ensure that the classification has a clear basis, a standard feature library is prepared in advance for common diseases. It contains visual samples of typical diseases such as pits, cracks, subsidence, and oil surface damage under different lighting or viewing angles. For example, the features of typical pits include descriptions such as "prominent edge gradient, continuous central shadow block, and gray level decreasing stepwise from the periphery to the center".

[0065] When comparing real-time features with standard features, once the matching degree reaches the pre-set consistency threshold (similarity higher than 80%), it is considered that the classification b1 of the real-time feature and the standard feature b2 belong to the same disease type. Therefore, the recognition model trained with the standard feature b2 is directly called to process classification b1.

[0066] Step 4: Build a GPU resource pool to manage GPUs, and allocate GPUs to the recognition model based on GPU memory usage so that the recognition model can identify road defects in the target image;

[0067] In another preferred embodiment of the present invention, allocating GPU memory to the recognition model based on GPU memory usage includes:

[0068] Register the available GPUs in the computer room to a unified resource pool and read the maximum memory limit of each GPU. For example, devices G1, G2, and G3 have 48GB, 32GB, and 16GB of memory respectively. To facilitate the selection of the device with the strongest capacity, first make a list according to the total memory from largest to smallest. This step can ensure that the GPU with the largest remaining space is always given priority in the future, thereby reducing cross-card communication caused by the model being distributed among multiple devices.

[0069] At the same time, a list of recognition models that need to be deployed is also created. The models in the list are arranged from largest to smallest based on the memory consumption of a single target image during inference. For example, models M1, M2, and M3 consume 12GB, 10GB, and 8GB of memory respectively. This sorting allows the system to process the most "space-consuming" models first, avoiding them being rejected by GPUs with small memory.

[0070] Then, the first GPU in the list, G1, is retrieved, with a memory limit of DY1 = 48GB. The memory usage is then accumulated from top to bottom in the model list until the accumulated value Di does not exceed 80% of DY1. However, adding the next model Di+1 would exceed this threshold. For example, M1+M2+M3 (30GB) is still less than 38.4GB, but adding M4 would exceed it. Therefore, the stage position i is determined to be 3. Using 80% instead of 100% of the limit is to account for the additional overhead of model loading, such as weight buffers and temporary feature maps; leaving a margin reduces the risk of accidental overflow during inference.

[0071] Once i is determined, G1 is assigned to models M1 to M3 for joint use. During inference, the three models run in parallel or in rotation in the memory space of G1.

[0072] After the allocation is completed, G1 is removed from the GPU list, and M1 to M3 are deleted from the model list, forming two new lists. The algorithm then takes the first G2 of the new GPU list and repeats the above accumulation and comparison process. This cycle continues until, after a certain accumulation, the total memory requirement of the remaining models in the model list is less than the single-card memory limit of the first Gx in the current GPU list. At this point, it means that all the remaining models can be accommodated on this GPU, so they are allocated all at once and no further iterations are continued.

[0073] In another preferred embodiment of the present invention, identifying road defects in a target image includes:

[0074] When it is determined that a certain model needs to be run on a specific GPU, the weights and computation graph of the model on the disk are first copied to the video memory of the target GPU, similar to putting a tool on the workbench for easy access; after loading is completed, the target image is input into the corresponding recognition model, and the output is whether the target image contains the road defects corresponding to the recognition model.

[0075] In another preferred embodiment of the present invention, the identification of road defects in a target image further includes:

[0076] During the model execution phase, the hardware utilization and memory usage changes of each GPU are continuously read at millisecond intervals, and the average value is taken using a sliding window to eliminate instantaneous spikes. When the average utilization of a GPU is consistently higher than 80% and does not drop down after several sampling windows, the device is marked as an abnormal GPU. At the same time, the models that are executing inference on that card are registered as abnormal models one by one. For example, if models M5 and M6 running on GPU G2 have jointly pushed up the load in the past dozens of samplings, then G2 is recorded as an abnormal GPU, and M5 and M6 are recorded as abnormal models.

[0077] For each anomalous model, the average amount of display memory actually occupied by its current inference thread within a preset calculation cycle is calculated separately. This is used as a measure of migration urgency. All anomalous models are then sorted from high to low according to this measure to generate a third sort, so that the model most likely to cause congestion is placed at the front.

[0078] At the same time, the GPUs that are still available in the resource pool are sorted from low to high according to their real-time utilization to generate a fourth sort, so that the devices with the highest idle rate are at the front, making it easier to take on subsequent tasks.

[0079] Next, select the top-ranked abnormal model in the third ranking and the top-ranked target GPU in the fourth ranking. For example, select the M5 with the highest load and the G1 with the highest idle degree to form a migration pair. First, preload the weights required by M5 on G1 and allocate the video memory buffer. Then, let M5 freeze the input after completing the current inference batch on the original G2. Switch to G1 to continue processing subsequent images. During this period, the model state and temporary features are synchronized through shared memory writing or high-speed network transmission. After the switch is completed, the binding relationship between M5 and G2 is removed in the monitoring system, and the utilization rate of G2 also decreases.

[0080] After the migration is completed, the list is automatically updated: if the G1 utilization is still below the threshold and there are still abnormal models remaining in the model list, the new third and fourth sorting positions are taken and the operation is repeated; through continuous looping, high-load models will be migrated to idle GPUs in order, thereby gradually releasing high-pressure equipment to a safe range and keeping the inference service uninterrupted.

[0081] It is worth noting that if the utilization rate of the target GPU at the top of the fourth sort is higher than 50%, a prompt message will be sent to the preset administrator.

[0082] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the present invention should still fall within the scope of the present invention.

Claims

1. A method for dynamic deployment of a MaaS model based on GPU resources, characterized in that, Includes the following steps: Based on deep learning, identification models for different types of road defects are established. These models are used to determine whether the corresponding road defects exist. Obtain all road intersections in the inspection area and mark the road segments between two adjacent road intersections as sub-segments; Based on the road defect data of sub-segments and the degree of impact of different types of road defects on road safety, the priority levels of sub-segments are divided into high, medium and low priority levels. The priority level of the sub-road segment is used as the priority level of the images acquired on the sub-road segment, and the images with the higher priority level are used as the target images; The target image is classified, with one classification corresponding to one recognition model. The GPU memory usage of the recognition model when recognizing road defects in a target image of the corresponding classification is obtained. A GPU resource pool is built to manage GPUs, and GPUs are allocated to the recognition model based on GPU memory usage so that the recognition model can identify road defects in the target image; Allocating GPUs to the recognition model based on GPU memory usage includes: The GPUs in the GPU resource pool are sorted in descending order of total video memory to obtain the first sort; the GPUs are then sorted in descending order of video memory usage to obtain the second sort. Obtain the total video memory DY1 of the GPU at the top of the first sort, and determine the stage position i in the second sort. Stage position i satisfies the following constraints: D i+1 ≥0.8DY1 and D i ≤0.8DY1, where Di represents the sum of the GPU memory usage of the first i GPUs in the second sort, starting from the first position in the second sort; Then, the GPU at the top of the first sort is designated as the target GPU for the recognition model corresponding to the GPU memory usage of the first to the i-th positions in the second sort, and is used by the recognition model corresponding to the GPU memory usage of the first to the i-th positions in the second sort. Remove the top GPU from the first sort to get a new first sort, and remove the memory usage of the first i GPUs from the second sort to get a new second sort; Identify the target GPU and its corresponding recognition model, and repeat the above steps until the sum of the memory usage of all GPUs in a new second ranking is less than the total memory usage of the first GPU in the corresponding new first ranking.

2. The GPU resource-based MaaS model dynamic deployment method according to claim 1, characterized in that, The priority levels for dividing sub-road segments include: The impact value is obtained by normalizing the degree of impact. Extract the number of times the same type of road defect occurred in the past n inspections from the road defect data, where n is a preset number. Multiply the number by the corresponding impact value to get the target number. Sum the target numbers for a sub-segment to get the sorted number. The sub-segments are sorted in descending order according to their corresponding ranking numbers. Starting from the first position in the ranking, the highest level is used as the priority level for the top 20% of the sub-segments in the ranking, the medium level is used as the priority level for the top 20% to 50% of the sub-segments in the ranking, and the lowest level is used as the risk level for the remaining sub-segments in the ranking.

3. The method for dynamically deploying a MaaS model based on GPU resources according to claim 1, characterized in that, Classifying target images includes: The target image is segmented into ROI regions, and the visual features of the ROI regions are extracted as real-time features. Target images with the same real-time features are classified as a single category. Visual features of different types of road defects are preset and denoted as standard features. If the real-time feature corresponding to category b1 is the same as the standard feature b2, then the recognition model corresponding to the standard feature b2 is used as the recognition model corresponding to category b1.

4. The method for dynamically deploying a MaaS model based on GPU resources according to claim 1, characterized in that, Identifying road defects in target images includes: Based on the target GPU, the corresponding recognition model is loaded, the target image is input into the corresponding recognition model, and the output is whether the target image contains the road defects corresponding to the recognition model.

5. The method for dynamically deploying a MaaS model based on GPU resources according to claim 1, characterized in that, Identifying road defects in target images also includes: The system monitors the utilization rate of the target GPU in real time. If the utilization rate is greater than 80%, the corresponding identification model is identified as an abnormal model, and the target GPU is marked as an abnormal GPU. The occupancy rate of each abnormal model is obtained in real time, and the abnormal models are sorted in descending order according to the occupancy rate to obtain the third sort. Sort the target GPUs in ascending order of utilization to obtain the fourth sort; Obtain the first abnormal model and the first target GPU in the third and fourth sorts respectively. Load the first abnormal model based on the first target GPU. After loading, the first abnormal model will no longer run on the abnormal GPU.

6. The method for dynamic deployment of MaaS models based on GPU resources according to claim 5, characterized in that, If the utilization rate of the target GPU at the top of the fourth sort is higher than 50%, a notification message will be sent to the preset administrator.

7. The method for dynamically deploying a MaaS model based on GPU resources according to claim 1, characterized in that, Establishing identification models for different road defects includes: For road defect a, a database is established, which stores images of road defect a based on manual annotations. A recognition model is established based on deep learning, and the recognition model is trained and validated based on a database to obtain the recognition model of road defect a.

Images