Intelligent computing power configuration method and system based on large language model

Abstract

The application discloses an intelligent computing power configuration method and system based on a large language model. The method comprises the following steps: receiving natural language input of a user, analyzing the user's intention through a large language model, extracting key information and generating structured configuration parameters; automatically identifying the application scene type through the large language model according to the scene information, distributing video channels to each scene and recommending algorithm combinations, and generating a scene configuration scheme; calculating the required number of GPUs from multiple resource dimensions such as video memory, QPS and decoding capability, and taking the maximum value of each dimension as the final number of GPUs; and generating a computing power configuration scheme containing the number of GPU configurations and the algorithm deployment mapping relationship of each scene. The system comprises an intention understanding module, a scene distribution module, a computing power calculation module and a scheme generation module. The application realizes end-to-end automatic conversion from natural language requirements to executable configuration schemes, reduces the use threshold of computing power configuration, and improves the accuracy of multi-dimensional resource configuration.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and cloud computing technology, specifically to a method and system for intelligent allocation of GPU computing resources based on a large language model. Background Technology

[0002] With the widespread deployment of deep learning-based video analytics algorithms in fields such as intelligent surveillance and security, how to accurately determine the required GPU computing resources based on factors such as the number of cameras, algorithm type, and scenario requirements has become an urgent problem to be solved in practical engineering.

[0003] Existing technologies include several algorithm configuration solutions for video surveillance scenarios. For example, some solutions receive sampled data from edge nodes, execute various algorithms on the sampled data, and obtain algorithm evaluation values, thereby automatically determining the target algorithm suitable for data analysis on terminal devices. While these solutions have made some progress in the automatic evaluation and selection of algorithms, their core limitation lies in the fact that the algorithm configuration process is based on a data-driven evaluation system, requiring the pre-collection and uploading of sampled data. They cannot understand deployment requirements described by users in natural language, and the barrier to entry for use remains relatively high.

[0004] Furthermore, in the field of GPU resource management, existing solutions primarily focus on runtime task scheduling. For example, some GPU scheduling methods for deep learning training tasks allocate resources by analyzing job queue characteristics, while others for inference tasks dynamically adjust GPU resources by monitoring QoS metrics. However, these solutions focus on runtime scheduling and sharing of GPU resources, rather than the resource planning problem of calculating the required number of GPUs based on application scenario requirements in the pre-deployment stage.

[0005] In terms of resource computing dimensions, existing computing power evaluation methods usually only consider single factors such as video memory or inference throughput, lacking a systematic approach that comprehensively considers multiple resource constraints such as video memory usage, inference throughput, and video decoding capacity. This can easily lead to unreasonable configuration problems where resources in one dimension are insufficient while resources in other dimensions are wasted.

[0006] Therefore, there is a need for an intelligent computing power configuration method that can understand natural language requirements, automatically allocate scenarios and recommend algorithms, accurately calculate computing power requirements from multiple dimensions, and output configuration schemes that can be directly used for deployment. Summary of the Invention

[0007] The purpose of this invention is to provide an intelligent computing power configuration method and system based on a large language model, so as to solve the technical problems in the existing technology, such as the computing power configuration process relying on human experience, inability to understand natural language requirements, unintelligent scenario allocation, and single resource calculation dimension.

[0008] To achieve the above-mentioned objectives, the technical solution provided by this invention includes:

[0009] The intelligent computing power allocation method based on large language models includes:

[0010] The system receives natural language input from users, parses the user's intent in the natural language input using a large language model, extracts key information, and generates structured configuration parameters.

[0011] Based on the scene information in the structured configuration parameters, the application scene type is automatically identified through a large language model, the video channels are allocated to the various scenes corresponding to the application scene type, and a recommended algorithm combination is generated for each scene to generate a scene configuration scheme.

[0012] Based on the scenario configuration scheme, the required number of GPUs is calculated from multiple resource dimensions, and the maximum value among the calculation results of each resource dimension is taken as the final required number of GPUs.

[0013] Based on the final required number of GPUs and the scene configuration scheme, a computing power configuration scheme is generated, which includes the number of GPUs and the algorithm deployment mapping relationship for each scene.

[0014] Preferably, the multiple resource dimensions specifically include the following resource dimensions: a memory dimension for evaluating the GPU memory resources required for the algorithm model to run, a QPS dimension for evaluating the algorithm's inference throughput, and a decoding dimension for evaluating the video channel's decoding capacity.

[0015] Preferably, the calculation method for the number of GPUs required for the video memory dimension includes: querying the model list corresponding to each algorithm in the scene configuration scheme, and accumulating the video memory requirements of each model to obtain the model video memory; calculating the decoding video memory based on the number of video channels and the video memory usage of a single channel; taking the sum of the model video memory and the decoding video memory as the total video memory requirement; calculating the number of GPUs required for the video memory dimension based on the total video memory requirement and the video memory capacity of a single GPU, and setting a safety threshold to ensure that the video memory utilization does not exceed a preset ratio.

[0016] Preferably, the calculation method for the number of GPUs required for the QPS dimension includes: classifying algorithms into high-frequency algorithms and low-frequency algorithms based on their detection intervals, wherein high-frequency algorithms have detection intervals less than a preset interval threshold, and low-frequency algorithms have detection intervals not less than the preset interval threshold; calculating the QPS consumption of each algorithm; for multiple algorithms belonging to the same model, taking the maximum QPS consumption among the multiple algorithms as the QPS consumption of that model; summing the QPS consumption of all models to obtain the total QPS requirement; calculating the number of GPUs required for the QPS dimension based on the total QPS requirement and the maximum QPS of a single GPU, and setting a safety threshold to ensure that the QPS utilization does not exceed a preset ratio.

[0017] Preferably, the calculation method for the number of GPUs required for the decoding dimension includes: counting the number of video channels corresponding to the high-frequency algorithm and the number of video channels corresponding to the low-frequency algorithm; calculating the number of GPUs required for each according to the upper limit of the number of channels carried by a single GPU for the high-frequency algorithm and the upper limit of the number of channels carried by a single GPU for the low-frequency algorithm; and taking the sum of the number of GPUs required for the high-frequency algorithm and the number of GPUs required for the low-frequency algorithm as the number of GPUs required for the decoding dimension.

[0018] Preferably, the method further includes identifying composite algorithms in the scenario configuration scheme, automatically expanding the composite algorithm into multiple sub-models, and calculating resource requirements for each of the multiple sub-models.

[0019] Preferably, the key information includes the number of video channels, algorithm list, GPU model, and scene configuration information, and the structured configuration parameters organize the key information in a preset data format.

[0020] Preferably, the method also includes using a caching strategy to maintain a preset number of dialogue history for each user, and when receiving subsequent natural language input, combining the dialogue history to perform intent parsing, so as to support the optimization of the configuration scheme through multi-turn dialogue iteration.

[0021] Preferably, the large language model automatically identifies the scene type based on the scene description in the scene information, allocates video channels according to the scene type and the sum of the number of video channels in each scene is equal to the total number of video channels, and recommends no more than a preset number of algorithm combinations for each scene.

[0022] This invention also discloses an intelligent computing power configuration system based on a large language model, comprising:

[0023] The intent understanding module is used to receive the user's natural language input, parse the user's intent in the natural language input through a large language model, extract key information, and generate structured configuration parameters.

[0024] The scene allocation module is used to automatically identify the application scene type based on the scene information in the structured configuration parameters using a large language model, allocate video channels to the various scenes corresponding to the application scene type, and recommend algorithm combinations for each scene to generate a scene configuration scheme.

[0025] The computing power calculation module is used to calculate the required number of GPUs from multiple resource dimensions according to the scenario configuration scheme, and take the maximum value among the calculation results of each resource dimension as the final required number of GPUs.

[0026] The scheme generation module is used to generate a computing power configuration scheme that includes the number of GPUs and the algorithm deployment mapping relationship for each scenario, based on the final required number of GPUs and the scenario configuration scheme.

[0027] The present invention has the following beneficial effects:

[0028] 1. By receiving and parsing users' natural language input through a large language model and generating structured configuration parameters, non-technical users can express their computing power configuration requirements without needing to master GPU hardware parameters and algorithm characteristics, thus lowering the barrier to entry for computing power configuration.

[0029] 2. By using a large language model to automatically identify scene types based on scene descriptions and complete video channel allocation and algorithm recommendations, the traditional method of manually configuring scene by scene based on human experience is replaced, which improves the rationality and generation efficiency of scene configuration schemes.

[0030] 3. Calculate the required number of GPUs from three resource dimensions: video memory, QPS, and decoding capability, and take the maximum value to determine the final number. This ensures that there are no bottlenecks in each resource dimension and avoids the problem of insufficient or wasted resources due to evaluation of a single dimension.

[0031] 4. The final output of the method is a structured computing power configuration scheme containing the number of GPUs and the algorithm deployment mapping relationship. It can be directly used to guide actual deployment and realizes end-to-end automated conversion from natural language requirements to executable configuration schemes. Attached Figure Description

[0032] Figure 1 This is a flowchart illustrating the intelligent computing power configuration method based on a large language model provided in an embodiment of the present invention.

[0033] Figure 2 This is a schematic diagram of the structure of an intelligent computing power configuration system based on a large language model, provided in an embodiment of the present invention. Detailed Implementation

[0034] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0035] Example 1

[0036] like Figure 1 As shown, the intelligent computing power configuration method based on a large language model provided by the present invention includes the following steps.

[0037] Step S1: Receive the user's natural language input and generate structured configuration parameters.

[0038] In real-world deployment scenarios of video surveillance systems, users typically describe their computing power requirements in natural language, such as "installing 100 cameras at a construction site, using an RTX 3060 graphics card, requiring algorithms for safety helmet detection, personnel intrusion, and license plate recognition." This unstructured requirement description contains various heterogeneous information, and traditional solutions require technicians to manually analyze and convert this information, which is inefficient and prone to errors.

[0039] In this step, after receiving the user's natural language input, the system performs semantic parsing on the input using a large language model to identify the user's intent, extract key information, and generate structured configuration parameters. The key information includes the number of video channels, an algorithm list, GPU model, and scene configuration information. The structured configuration parameters organize this key information in a preset data format.

[0040] Specifically, the large language model uses preset system prompts to guide user input, perform intent parsing, and output standardized structured data. The preset system prompts embed an algorithm knowledge base and a GPU parameter database. The algorithm knowledge base contains standard identifiers, functional descriptions, applicable scenario types, detection intervals, and resource consumption parameters for each algorithm supported by the system. The GPU parameter database contains hardware specifications for each GPU model, such as memory capacity, maximum QPS processing capability, and video decoding capacity limit. When performing intent parsing, the large language model can complete tasks such as algorithm recognition, parameter completion, and GPU model matching based on the embedded knowledge base data. In one specific embodiment, the preset data format is JSON, containing the following fields: number of video channels (e.g., channel_num: 100), algorithm list (e.g., ability_list: ["WithoutHelmetOnSite", "HumanBreakIn", "SingleVehicleSnap"]), GPU model (e.g., computeUnit: "NVIDIA GeForce RTX 3060"), and scene description (e.g., desc: "construction site scene"). It should be understood that in other embodiments, the preset data format may also be XML, YAML or other structured data formats, and the present invention does not limit this.

[0041] During intent parsing, the large language model can handle ambiguous expressions and implicit information in user input. For example, when a user describes "needing license plate recognition," the large language model can semantically match this common expression with the functional descriptions of various algorithms in the algorithm knowledge base embedded in the system prompt, thus mapping it to the corresponding standard algorithm identifier (such as SingleVehicleSnap or LicenseVehiclePlateRecognize). When the user only describes the scene type but does not list specific algorithms, the large language model can retrieve algorithms matching the scene type based on the applicable scene type information recorded in the algorithm knowledge base and automatically supplement them as recommended algorithms. When the user does not specify the GPU model, the large language model can recommend a suitable GPU model by comparing the computing power requirements with the carrying capacity of each GPU model, based on the hardware specifications of various GPU models recorded in the GPU parameter database, combined with the resource consumption parameters of the algorithm to be deployed and the video channel scale. In addition, the large language model can also recognize quantitative modifiers in user input, such as distinguishing the semantic difference between "50 channels per scene" and "50 channels in total," ensuring that the extracted number of video channels is accurate. This parameter extraction method based on deep semantic understanding significantly lowers the barrier to entry for computing power configuration compared to the traditional structured form filling method, enabling non-technical personnel such as project managers and sales engineers to independently complete the assessment of computing power requirements.

[0042] In some preferred embodiments, a caching strategy is also included to maintain a preset number of dialogue rounds for each user. When subsequent natural language input is received, intent parsing is performed in conjunction with the dialogue history to support iterative optimization of the configuration scheme through multiple dialogue rounds. In existing single-turn interaction schemes, users need to fully describe all configuration requirements in a single input, and must re-enter all information if adjustments are needed. This scheme introduces a context management mechanism, allowing users to describe only the parts that need modification in subsequent dialogues. The system automatically merges the historical context with the current input, achieving incremental optimization of the configuration scheme. In one specific embodiment, the caching strategy adopts an LRU (Least Recently Used) strategy, maintaining a history of the most recent 20 dialogue rounds for each user. It should be understood that in other embodiments, the preset number of rounds can be adjusted according to actual business scenarios and system resources, and the caching strategy can also adopt a FIFO or time window-based eviction strategy.

[0043] Step S2: Assign scenarios and generate scenario configuration schemes based on structured configuration parameters.

[0044] After obtaining the structured configuration parameters, the system automatically identifies the application scenario type based on the scenario information and a large language model, allocates the video channel to each scenario corresponding to the application scenario type, and recommends algorithm combinations for each scenario to generate a scenario configuration scheme.

[0045] In traditional video surveillance deployments, scene allocation typically relies on engineers' experience and is done manually. Different engineers may use vastly different allocation schemes, making it difficult to fully leverage the combined advantages of various algorithms. In this step, the large language model automatically identifies the scene type based on the scene description in the scene information, allocates video channels according to the scene type, ensuring the sum of the video channels for each scene equals the total number of video channels, and recommends no more than a preset number of algorithm combinations for each scene.

[0046] In scene recognition, the large language model leverages its ability to learn from large-scale corpora to understand the semantic information in scene descriptions. Simultaneously, it utilizes the algorithm knowledge base embedded in system prompts to acquire information on the applicable scene types and detection capabilities of each algorithm, thus achieving collaborative scene recognition and algorithm recommendation. For example, when a user describes a "construction site," the large language model can identify that this scene typically includes sub-scenes such as access control, construction area monitoring, and material storage area management, and allocates appropriate video channels based on the security monitoring needs of each sub-scene. In the algorithm recommendation stage, the large language model traverses the applicable scene type field of each algorithm in the algorithm knowledge base, filters out algorithms matching each sub-scene type as candidate algorithms, and selects the most suitable algorithm combination for the scene's requirements from the candidate algorithms based on their functional descriptions and detection capabilities.

[0047] In one specific embodiment, for the requirement of installing 100 cameras in a construction site scenario, the large language model automatically allocates the total video channels into multiple sub-scenes based on the typical monitoring layout of a construction site scenario. For example, 20 cameras are used for entrance / exit capture, 50 for construction area monitoring, and 30 for material storage area monitoring. Simultaneously, algorithm combinations are recommended for each scene: for the entrance / exit scene, SnapFace (face capture), SingleVehicleSnap (license plate recognition), and HumanVehicleCrossAlarm (human-vehicle cross-alarm detection) are recommended; for the construction area, WithoutHelmetOnSite (safety helmet detection), PersonWalkingIllegally (violation of walking rules), and OffDutyAlarm (absence from duty detection) are recommended; and for the material storage area, HeapOfMaterial (material storage detection) and AreaAlarm (area alarm) are recommended. In one specific embodiment, the preset number is 5, meaning that the number of algorithms recommended for each scene does not exceed 5. It should be understood that in other embodiments, the preset number can be adjusted according to GPU capacity and actual business needs.

[0048] After scene allocation, the system assembles information such as the number of video channels and the list of bound algorithms for each scene into a scene configuration scheme for use in subsequent computing power calculation steps. The scene configuration scheme is stored in a structured data format, where each scene entry includes the scene name, the number of allocated video channels, and a list of algorithm identifiers bound to that scene. This structured scene configuration scheme provides a clear data foundation for the subsequent scene-by-scene and algorithm-by-algorithm resource requirement traversal in computing power calculations.

[0049] Step S3: Perform multi-dimensional computing power calculations based on the scenario configuration scheme.

[0050] After obtaining the scenario configuration scheme, the system calculates the required number of GPUs from multiple resource dimensions according to the scheme, and takes the maximum value of the calculation results of each resource dimension as the final required number of GPUs.

[0051] In actual GPU hardware operation, many factors affect GPU capacity, including at least memory capacity, inference throughput, and video decoding capacity. If evaluated from only a single dimension, resources may be abundant in one dimension while insufficient in another, leading to system malfunctions. This solution calculates the GPU requirement independently from multiple resource dimensions and takes the maximum value from each dimension, ensuring that no resource bottlenecks occur in any dimension.

[0052] The multiple resource dimensions include: a memory dimension for evaluating the GPU memory resources required for the algorithm model to run; a QPS dimension for evaluating the algorithm's inference throughput; and a decoding dimension for evaluating the video channel's decoding capacity. The calculation process for each dimension is explained below.

[0053] Step S3.1: Calculate the number of GPUs in terms of video memory.

[0054] The calculation of the number of GPUs required for the memory dimension includes the following sub-steps.

[0055] The system queries the model list corresponding to each algorithm in the scenario configuration scheme and accumulates the memory requirements of each model to obtain the ModelMemory. In the field of video analysis, an algorithm may correspond to one or more deep learning inference models, and different models have different memory usage. The system queries a pre-set model parameter database to obtain the memory requirements of each model and accumulates them. During the accumulation process, if different algorithms share the same underlying model, the memory of that model is only counted once to avoid duplicate statistics. For example, face capture algorithms and personnel intrusion detection algorithms may share the same human detection base model. When calculating the model memory, the memory requirement of that base model is only accumulated once. The system implements this mechanism by maintaining a model deduplication list: it iterates through all algorithms in all scenarios, queries the model identifier corresponding to each algorithm, filters out duplicate model identifiers, and only accumulates the memory for the unique deduplicated model list.

[0056] Decoding memory is calculated based on the number of video channels and the memory usage per channel. Before performing video analysis, the GPU needs to perform hardware decoding of the video stream, and the decoding process for each video channel consumes a certain amount of GPU memory. The formula for calculating decoding memory is:

[0057] ;

[0058] in, This represents the total number of video channels. This refers to the single-channel video memory usage. In one specific embodiment, the single-channel video memory usage... The value is 51MB, which is based on the measured average of 1080p resolution video streams on mainstream GPU decoders. It should be understood that in other embodiments, this value may be adjusted according to different video resolutions and encoding formats; for example, the single-channel video memory usage of a 4K video stream may be around 200MB.

[0059] The sum of the model's video memory and the decoding's video memory is taken as the total video memory requirement (TotalMemory).

[0060] ;

[0061] Calculate the required number of GPUs (MemoryNum) for the memory dimension based on the total memory requirement and the memory capacity of a single GPU, and set a safety threshold to ensure that memory utilization does not exceed a preset ratio:

[0062] ;

[0063] in, This refers to the video memory capacity of a single GPU. The safety threshold ratio is defined by `ceil`, which is the rounding function. The purpose of setting this safety threshold is that, in actual operation, the GPU needs to reserve a certain amount of video memory space for the operating system, drivers, and temporary computing buffers, in addition to algorithm inference and video decoding. If the video memory utilization is too high, it may cause a video memory overflow error, leading to the interruption of the inference task. In a specific embodiment, the preset ratio is 80%, i.e. = 0.8, a value that balances resource utilization and operational stability in engineering practice. It should be understood that in other embodiments, the preset ratio can be adjusted within the range of 70% to 90% based on GPU model characteristics and actual operating conditions.

[0064] This also includes handling composite algorithms during GPU memory computation. The system identifies composite algorithms in the scene configuration scheme, automatically expands them into multiple sub-models, and calculates the resource requirements for each sub-model. In the field of video analysis, some algorithms are composite algorithms, requiring the sequential invocation of multiple independent deep learning models. For example, the license plate recognition algorithm (LicenseVehiclePlateRecognize) requires the sequential invocation of three sub-models: the license plate detection model (LicensePlateDetect), the vehicle attribute recognition model (VehicleAttribute), and the license plate character recognition model (LicensePlateRecognize). Treating composite algorithms as single models would lead to an underestimation of GPU memory requirements. This solution ensures the accuracy of GPU memory computation by automatically identifying composite algorithms and expanding them into a list of sub-models.

[0065] Step S3.2: Calculate the number of GPUs in the QPS dimension.

[0066] The calculation of the number of GPUs required for the QPS dimension includes the following sub-steps. QPS (queries per second) reflects the inference throughput capacity of the GPU. When the inference requests that the deployed algorithm needs to process exceed the QPS limit that the GPU can handle, inference queue backlog and processing latency will occur.

[0067] Algorithms are categorized into high-frequency and low-frequency algorithms based on their detection intervals. In video analytics scenarios, the detection frequencies of different algorithms vary significantly: real-time behavior detection algorithms need to process every frame or every few frames, while periodic inspection algorithms may only need to be executed every few seconds or even longer. The detection interval (in seconds) represents the time interval between two consecutive executions of the algorithm. Algorithms with detection intervals less than a preset threshold are considered high-frequency algorithms, while those with detection intervals not less than the preset threshold are considered low-frequency algorithms. In one specific embodiment, the preset threshold is 2 seconds, meaning algorithms with interval < 2 seconds are high-frequency algorithms, and algorithms with interval ≥ 2 seconds are low-frequency algorithms. It should be understood that in other embodiments, this threshold can be adjusted according to the real-time requirements of the business.

[0068] Calculate the QPS cost for each algorithm. The QPS cost of a single algorithm is determined by the number of video channels in the scene and the detection interval.

[0069] ;

[0070] in, The number of video channels in the scene where the algorithm is used. This represents the detection interval of the algorithm. For example, if a scene has 50 video channels and a high-frequency algorithm with a detection interval of 1 second is running, then the QPS consumption of this algorithm is 50 × (1 / 1) = 50.

[0071] For multiple algorithms belonging to the same model, the maximum QPS consumption among these algorithms is taken as the QPS consumption of that model. This is because in a GPU inference engine, multiple algorithms sharing the same underlying model actually reuse the same model instance for inference, and their QPS consumption is not simply additive. When the same model is called by multiple algorithms, the GPU inference engine can handle the requests of multiple algorithms simultaneously in a single model forward propagation; therefore, the actual QPS consumption of the model depends on the algorithm with the highest QPS consumption. That is:

[0072] ,in To use the same type of algorithm.

[0073] To further illustrate this deduplication principle, let's take the helmet detection algorithm and the intrusion detection algorithm as examples: Assuming these two algorithms share the same basic human detection model, and the QPS cost for helmet detection is 50 while the QPS cost for intrusion detection is 30, then the QPS cost of the human detection model will be the maximum of the two, 50, rather than their sum of 80. This is because the GPU inference engine can generate feature outputs for both algorithms simultaneously by performing a single forward inference on the model, eliminating the need for two separate inference operations. Using a simple summation approach would overestimate the number of GPUs required, resulting in wasted resources.

[0074] The total QPS requirement is obtained by summing the QPS consumption of all models. Based on the total QPS requirement and the maximum QPS per GPU, the number of GPUs required for the QPS dimension, QpsNum, is calculated, and a safety threshold is set to ensure that the QPS utilization does not exceed a preset ratio.

[0075] ;

[0076] in, This refers to the maximum QPS processing capability of a single GPU (which can be found in the GPU model parameter database). The preset safety threshold ratio is the same as that used for the video memory dimension. In one specific embodiment, the preset ratio is 80%. It should be understood that in other embodiments, the safety threshold for the QPS dimension can be different from that for the video memory dimension to accommodate different operational stability requirements.

[0077] Step S3.3: Calculate the number of GPUs for the decoding dimension.

[0078] The calculation of the number of GPUs required for the decoding dimension includes the following sub-steps. The GPU's video hardware decoder has a fixed upper limit on its capacity. When the number of video streams to be decoded in parallel exceeds this upper limit, some video streams will not be able to be decoded, resulting in the algorithm being unable to obtain input data.

[0079] The number of video channels corresponding to the high-frequency algorithm (HighChannels) and the number of video channels corresponding to the low-frequency algorithm (LowChannels) are counted. During the count, if a video channel runs both the high-frequency and low-frequency algorithms simultaneously, it is categorized according to its highest running frequency, i.e., it is classified as a high-frequency algorithm channel. Because of its high detection frequency, the high-frequency algorithm requires the GPU's video decoder to decode image frames from the video stream at a higher frame rate for algorithm processing, resulting in a relatively smaller number of concurrent channels that a single GPU can stably handle. Conversely, the low-frequency algorithm has a lower detection frequency, allowing the decoder to employ a frame-skipping strategy, decoding only when detection is needed, thus enabling a significantly larger number of concurrent channels that a single GPU can handle.

[0080] Based on the upper limit of the number of channels per GPU in high-frequency algorithms The maximum number of channels that a single GPU can handle for low-frequency algorithms. Calculate the required number of GPUs for each algorithm separately, and sum the number of GPUs required for the high-frequency algorithm and the number of GPUs required for the low-frequency algorithm as the number of GPUs required for the decoding dimension, DecodeNum:

[0081] ;

[0082] In one specific embodiment, 25-way / GPU The figure is 128 streams / GPU. This value is based on actual performance tests of video decoders on mainstream GPUs (such as NVIDIA T4, RTX 3060, etc.): In high-frequency algorithm scenarios, each frame requires decoding, and the measured stable capacity of a single GPU is approximately 25 1080P video streams; in low-frequency algorithm scenarios, the decoder can skip frames, and the measured stable capacity of a single GPU can be increased to approximately 128 streams. It should be understood that in other embodiments, the above capacity limit can be adjusted according to different GPU models, video resolutions, and encoding formats.

[0083] Step S3.4: Take the maximum value of each dimension to determine the final number of GPUs.

[0084] After completing the independent calculations for the three dimensions, the maximum value of the results for each dimension is taken as the final number of GPUs required.

[0085] ;

[0086] The rationale for adopting the maximum value strategy is that a GPU is a holistic hardware unit, and its memory, inference throughput, and decoding capabilities cannot be independently expanded. If the demand in any dimension exceeds the capacity of the current number of GPUs, the number of GPUs needs to be increased. Therefore, the final number of GPUs must satisfy the dimension with the highest demand. For example, in a certain configuration scenario, if memory requires 1 GPU, QPS requires 2 GPUs, and decoding requires 1 GPU, then the final number of GPUs is the maximum of 2. While only 1 GPU is needed for memory, configuring only 1 GPU would lead to a backlog of inference requests in the QPS dimension, significantly increasing latency in video analysis and potentially causing task loss. By maximizing the value, sufficient resource margins are ensured for each dimension, guaranteeing stable system operation across all dimensions.

[0087] Step S4: Generate computing power configuration scheme.

[0088] Based on the final required number of GPUs and the scene configuration scheme, a computing power configuration scheme is generated, containing the number of GPUs and the algorithm deployment mapping relationship for each scene. The computing power configuration scheme includes: the final required number of GPUs, GPU models, scene names and corresponding video channel numbers, a list of algorithms bound to each scene, GPU requirements and resource utilization for each dimension, etc. The algorithm deployment mapping relationship explicitly records the correspondence between each scene and its bound algorithm, as well as the underlying model information upon which each algorithm depends, enabling the subsequent deployment configuration system to automatically complete algorithm loading and video channel binding based on this mapping relationship. The resource utilization rate for each dimension is calculated as follows: memory utilization rate equals the total memory requirement divided by the actual configured total GPU memory capacity; QPS utilization rate equals the total QPS requirement divided by the actual configured total GPU QPS capacity. Resource utilization information helps operations and maintenance personnel assess the resource redundancy of the configuration scheme and determine whether there is over-configuration or under-configuration. This computing power configuration scheme is structured data and can be directly used for the deployment configuration of the video surveillance platform.

[0089] In some preferred embodiments, the method further includes converting the computing power configuration scheme into a configuration scheme description in natural language form using a large language model. The large language model receives structured computing power configuration scheme data and converts the technical parameters and values ​​therein into easily understandable natural language descriptions, enabling non-technical personnel to quickly understand the configuration results and resource utilization, facilitating procurement decisions and scheme reviews.

[0090] Example 2

[0091] like Figure 2 As shown, the present invention also provides an intelligent computing power configuration system based on a large language model, which includes an intent understanding module, a scene allocation module, a computing power calculation module, and a scheme generation module.

[0092] The intent understanding module receives natural language input from the user, parses the user's intent from the natural language input using a large language model, extracts key information, and generates structured configuration parameters. The intent understanding module integrates a large language model inference engine and a system prompt word management component. The system prompt word management component maintains prompt word templates optimized for different computing power configurations. These templates embed an algorithm knowledge base and a GPU parameter database. The algorithm knowledge base contains standard identifiers, functional descriptions, applicable scenario types, and resource consumption parameters for each algorithm. The GPU parameter database contains hardware specifications for various GPU models, such as memory capacity, maximum QPS, and decoding capacity limits. This allows the large language model to perform algorithm recognition, parameter completion, and GPU model matching based on the aforementioned knowledge base data. After receiving user input, the intent understanding module concatenates the user input with the system prompt words and sends it to the large language model inference engine to obtain structured parsing results.

[0093] The scene allocation module is used to automatically identify application scene types based on the scene information in the structured configuration parameters using a large language model, allocate video channels to the corresponding scenes for each application scene type, and recommend algorithm combinations for each scene to generate a scene configuration scheme. The scene allocation module receives the structured configuration parameters output by the intent understanding module, extracts the scene description information, performs scene semantic analysis using a large language model, and determines the scene type and the video channel allocation ratio for each scene. This module also maintains an algorithm knowledge base containing information such as the applicable scenarios, detection capabilities, and resource consumption of various algorithms to assist the large language model in making more accurate algorithm recommendations.

[0094] The computing power calculation module is used to calculate the required number of GPUs from multiple resource dimensions according to the scenario configuration scheme, and take the maximum value of the calculation results of each resource dimension as the final required number of GPUs. The computing power calculation module is the module that performs core technology calculations in this system. It includes a video memory calculation submodule, a QPS calculation submodule, a decoding calculation submodule, and a GPU number determination submodule. The calculation process of the four submodules corresponds one-to-one with the steps described in step S3 of embodiment one. The video memory calculation submodule is responsible for querying the model parameter database to obtain the video memory requirements of each model, calculating the decoding video memory, and summarizing to obtain the total video memory requirements; the QPS calculation submodule is responsible for distinguishing between high-frequency and low-frequency algorithms, calculating the QPS consumption of each algorithm, performing QPS deduplication for the same model, and summarizing to obtain the total QPS requirements; the decoding calculation submodule is responsible for counting the number of video channels corresponding to various algorithms and calculating the number of GPUs required for decoding according to the upper limit of the capacity; the GPU number determination submodule takes the maximum value of the calculation results of the above three submodules as the final number of GPUs.

[0095] The scheme generation module is used to generate a computing power configuration scheme that includes the number of GPUs and the algorithm deployment mapping relationship for each scene, based on the final required number of GPUs and the scene configuration scheme. The scheme generation module receives the GPU requirement number, final number of GPUs, and resource utilization rate for each dimension from the computing power calculation module, and simultaneously receives the scene configuration scheme from the scene allocation module, integrating the two into a complete computing power configuration scheme. This scheme is output in a structured data format, including information such as GPU model and quantity, video channel allocation for each scene, algorithm binding relationships, and resource utilization rate for each dimension. The scheme generation module is also responsible for verifying the rationality of the configuration results, such as checking whether the final number of GPUs exceeds the preset hardware scale limit and whether the resource utilization rate for each dimension is within the safe threshold range. It marks unreasonable configuration results with warning messages to assist users in making adjustment decisions.

[0096] The data flow between the modules is as follows: User natural language input → Intent understanding module → Structured configuration parameters → Scene allocation module → Scene configuration scheme → Computing power calculation module → GPU quantity calculation result → Scheme generation module → Computing power configuration scheme. The modules communicate through standardized data interfaces, and both the input and output of each module are in structured data format. This allows each module to be upgraded and its performance optimized independently without affecting the overall system operation.

[0097] In some preferred embodiments, the system further includes a context management module that uses a caching strategy to maintain a preset number of dialogue history rounds for each user. When a user initiates a subsequent dialogue, the intent understanding module retrieves the user's dialogue history from the context management module, combines the historical context with the current input to perform intent parsing, thereby supporting iterative optimization of configuration schemes in multi-turn dialogue scenarios. The context management module internally maintains a cache space with the user identifier as the key. After each dialogue, the input and output of this round are appended to the corresponding user's history record, and expired records are cleaned up according to a preset eviction policy.

[0098] Experimental Example

[0099] To verify the effectiveness of the method of the present invention, a computing power configuration experiment was conducted using a construction site scenario as an example.

[0100] Experimental conditions: The user inputs "Install 100 cameras at a construction site, using an RTX 3060 graphics card, requiring personnel safety detection and vehicle management algorithms"; the GPU model is NVIDIA GeForce RTX 3060 (12GB VRAM).

[0101] Step S1 execution result: After parsing the user input, the large language model generates structured configuration parameters, extracts 100 video channels, uses an NVIDIA GeForce RTX 3060 GPU, and includes algorithms such as helmet detection, personnel intrusion detection, and license plate recognition.

[0102] Step S2 execution result: The large language model allocates 100 video channels into three scenarios: 20 channels for entrance and exit capture, 50 channels for construction area monitoring, and 30 channels for material stacking area, and recommends corresponding algorithm combinations for each scenario.

[0103] Step S3 execution result: The system calculates GPU requirements from three dimensions. In terms of memory, ModelMemory is 2800MB, DecodingMemory is 100 × 51 = 5100MB, and the total memory requirement is 7900MB. Based on an RTX 3060 single card with 12GB of memory and an 80% safety threshold, MemoryNum = ceil(7900 / (12000 × 0.8)) = ceil(7900 / 9600) = 1. In terms of QPS, the total QPS requirement (TotalQps) after accumulating the QPS consumption of all models is 45.6. QpsNum is calculated based on the maximum QPS per card and the 80% safety threshold. In terms of decoding, DecodeNum is calculated based on the upper limit of 25 channels / GPU for high-frequency algorithms and 128 channels / GPU for low-frequency algorithms. Finally, the number of GPUs, GpuNum, is the maximum value of the three dimensions, determined to be 2 GPUs.

[0104] Step S4 execution result: The system generates a computing power configuration scheme, which includes a GPU configuration of 2 NVIDIA RTX3060, video channel allocation and algorithm binding relationship for each scene, a video memory utilization rate of 65.8%, a QPS utilization rate of 72.3%, and resource utilization rates in all dimensions are within the safe threshold range, indicating that the configuration scheme is reasonable.

[0105] The experimental results above demonstrate that the method of this invention can automatically complete the entire process of scene allocation, multi-dimensional computing power calculation, and configuration scheme generation, starting from natural language input. The resource utilization rates of each dimension in the output computing power configuration scheme are all within safe threshold ranges, and the configuration scheme is reasonable and can be directly used to guide the GPU hardware deployment of video surveillance systems. In this experimental example, the four core components of the method—natural language intent understanding, scene semantic allocation, multi-dimensional computing power calculation, and configuration scheme generation—all executed according to the expected process and produced correct outputs, verifying the technical feasibility and end-to-end automation capabilities of the method of this invention.

[0106] It's worth noting that in the above experimental example, the final number of GPUs was determined to be two by taking the maximum value from the three dimensions. The memory utilization rate was 65.8%, and the QPS utilization rate was 72.3%, both within the 80% safety threshold. This demonstrates that the maximum-value fusion strategy ensures sufficient resources without excessive waste. If only the memory dimension is considered, only one GPU might be needed to meet the memory requirements, but in reality, the capacity for QPS and decoding would be insufficient, leading to system instability.

[0107] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various improvements and substitutions can be made to the technical solutions in the above embodiments without departing from the technical principles of the present invention, and these improvements and substitutions should also be considered within the protection scope of the present invention.

Claims

1. A method for configuring intelligent computing power based on a large language model, characterized in that, include: Receive natural language input from users, parse the user intent in the natural language input through a large language model, extract key information and generate structured configuration parameters; Based on the scene information in the structured configuration parameters, the application scene type is automatically identified through a large language model, the video channel is allocated to each scene corresponding to the application scene type, and an algorithm combination is recommended for each scene to generate a scene configuration scheme. Based on the scenario configuration scheme, the required number of GPUs is calculated from multiple resource dimensions, and the maximum value among the calculation results of each resource dimension is taken as the final required number of GPUs. Based on the final required number of GPUs and the scene configuration scheme, a computing power configuration scheme is generated that includes the number of GPUs and the algorithm deployment mapping relationship for each scene; The multiple resource dimensions include: a memory dimension for evaluating the GPU memory resources required for the algorithm model to run, a QPS dimension for evaluating the algorithm's inference throughput, and a decoding dimension for evaluating the video channel's decoding capacity. The calculation methods for the number of GPUs required for the video memory dimension include: Query the list of models corresponding to each algorithm in the scenario configuration scheme, and sum up the memory requirements of each model to obtain the model memory. Calculate the decoding memory based on the number of video channels and the memory usage per channel; The sum of the model video memory and the decoding video memory is taken as the total video memory requirement; The number of GPUs required for the memory dimension is calculated based on the total memory requirement and the memory capacity of a single GPU, and a safety threshold is set to ensure that the memory utilization does not exceed a preset ratio. The calculation methods for the number of GPUs required for the QPS dimension include: Based on the detection interval of each algorithm, the algorithms are divided into high-frequency algorithms and low-frequency algorithms. Among them, the algorithms with a detection interval less than a preset interval threshold are high-frequency algorithms, and the algorithms with a detection interval not less than the preset interval threshold are low-frequency algorithms. Calculate the QPS cost of each algorithm separately. For multiple algorithms belonging to the same model, take the maximum QPS cost among the multiple algorithms as the QPS cost of the model. The total QPS requirement is obtained by summing up the QPS consumption of all models; Calculate the number of GPUs required for the QPS dimension based on the total QPS requirement and the maximum QPS of a single GPU, and set a safety threshold to ensure that the QPS utilization does not exceed a preset ratio; The calculation methods for the number of GPUs required for the decoding dimension include: Count the number of video channels corresponding to the high-frequency algorithm and the number of video channels corresponding to the low-frequency algorithm; Calculate the required number of GPUs for each algorithm based on the upper limit of the number of channels per GPU for high-frequency algorithms and the upper limit of the number of channels per GPU for low-frequency algorithms. The sum of the number of GPUs required for the high-frequency algorithm and the number of GPUs required for the low-frequency algorithm is taken as the number of GPUs required for the decoding dimension.

2. The intelligent computing power configuration method based on a large language model according to claim 1, characterized in that, It also includes identifying composite algorithms in the scenario configuration scheme, automatically expanding the composite algorithm into multiple sub-models, and calculating resource requirements for each of the multiple sub-models.

3. The intelligent computing power configuration method based on a large language model according to claim 1, characterized in that, The key information includes the number of video channels, algorithm list, GPU model, and scene configuration information. The structured configuration parameters organize the key information in a preset data format.

4. The intelligent computing power configuration method based on a large language model according to claim 1, characterized in that, It also includes using a caching strategy to maintain a preset number of dialogue history for each user, and when receiving subsequent natural language input, combining the dialogue history to perform intent parsing, so as to support the optimization of configuration schemes through multi-turn dialogue iterations.

5. The intelligent computing power configuration method based on a large language model according to claim 1, characterized in that, The large language model automatically identifies the scene type based on the scene description in the scene information, allocates video channels according to the scene type, and the sum of the number of video channels in each scene is equal to the total number of video channels. It also recommends no more than a preset number of algorithm combinations for each scene.

6. An intelligent computing power configuration system based on a large language model, characterized in that, include: The intent understanding module is used to receive the user's natural language input, parse the user's intent in the natural language input through a large language model, extract key information, and generate structured configuration parameters; The scene allocation module is used to automatically identify the application scene type based on the scene information in the structured configuration parameters through a large language model, allocate video channels to each scene corresponding to the application scene type, and recommend algorithm combinations for each scene to generate a scene configuration scheme. The computing power calculation module is used to calculate the required number of GPUs from multiple resource dimensions according to the scenario configuration scheme, and take the maximum value of the calculation results of each resource dimension as the final required number of GPUs. The scheme generation module is used to generate a computing power configuration scheme that includes the number of GPUs and the algorithm deployment mapping relationship for each scenario, based on the final required number of GPUs and the scenario configuration scheme. The multiple resource dimensions include: a memory dimension for evaluating the GPU memory resources required for the algorithm model to run, a QPS dimension for evaluating the algorithm's inference throughput, and a decoding dimension for evaluating the video channel's decoding capacity. The calculation methods for the number of GPUs required for the video memory dimension include: Query the list of models corresponding to each algorithm in the scenario configuration scheme, and sum up the memory requirements of each model to obtain the model memory. Calculate the decoding memory based on the number of video channels and the memory usage per channel; The sum of the model video memory and the decoding video memory is taken as the total video memory requirement; The number of GPUs required for the memory dimension is calculated based on the total memory requirement and the memory capacity of a single GPU, and a safety threshold is set to ensure that the memory utilization does not exceed a preset ratio. The calculation methods for the number of GPUs required for the QPS dimension include: Based on the detection interval of each algorithm, the algorithms are divided into high-frequency algorithms and low-frequency algorithms. Among them, the algorithms with a detection interval less than a preset interval threshold are high-frequency algorithms, and the algorithms with a detection interval not less than the preset interval threshold are low-frequency algorithms. Calculate the QPS cost of each algorithm separately. For multiple algorithms belonging to the same model, take the maximum QPS cost among the multiple algorithms as the QPS cost of the model. The total QPS requirement is obtained by summing up the QPS consumption of all models; Calculate the number of GPUs required for the QPS dimension based on the total QPS requirement and the maximum QPS of a single GPU, and set a safety threshold to ensure that the QPS utilization does not exceed a preset ratio; The calculation methods for the number of GPUs required for the decoding dimension include: Count the number of video channels corresponding to the high-frequency algorithm and the number of video channels corresponding to the low-frequency algorithm; The required number of GPUs is calculated based on the upper limit of the number of channels per GPU for the high-frequency algorithm and the upper limit of the number of channels per GPU for the low-frequency algorithm respectively; the sum of the number of GPUs required for the high-frequency algorithm and the number of GPUs required for the low-frequency algorithm is taken as the number of GPUs required for the decoding dimension.

Images