Video enhancement inference method and device based on fuzzy control, equipment and medium

Abstract

The present application relates to a kind of video enhancement inference method, device, equipment and medium based on fuzzy control, method includes: initialization multiple different scale pre-training video inference model;Define the fuzzy semantic set of input variable and output variable and membership function;Based on prior knowledge, construct fuzzy rule table;Real-time acquisition input data and complete fuzzy inference, and accurate score is obtained by solving fuzzy output by barycenter method;According to the score value range and anti-shake strategy, the optimal video inference model of next frame is dynamically selected.This method makes full use of the continuity characteristics of target between video frames, realizes the dynamic balance of hardware resources and inference accuracy, the utilization rate of GPU is obviously improved, the target detection rate is also obviously improved, and the temperature is stable within the safety threshold, suitable for real-time video scene such as traffic monitoring, unmanned aerial vehicle inspection.

Description

Technical Field

[0001] This invention relates to the field of video enhancement inference technology, and in particular to a video enhancement inference method, apparatus, device and medium based on fuzzy control. Background Technology

[0002] Current video inference technologies (such as object detection models YOLO and DETR) mainly rely on a single fixed model to process continuous video streams, which has significant drawbacks:

[0003] Resource imbalance: While large-scale models offer high accuracy, they also demand significant computing power, easily leading to GPU overload and overheating; small-scale models, while having low resource consumption, struggle to handle complex scenarios (such as dense targets), resulting in high false negative rates. In practical applications, it is difficult to optimize hardware resource utilization and inference performance in a coordinated manner.

[0004] Ignoring inter-frame correlation: The state (quantity, location) of targets between adjacent frames in a video usually has spatiotemporal continuity, with an extremely low probability of abrupt changes. However, existing methods infer independently for each frame, neither utilizing this characteristic to reduce computational redundancy nor dynamically adjusting the model according to scene changes, resulting in wasted resources or poor inference efficiency.

[0005] For example, in traffic monitoring scenarios where traffic flow changes gradually, using a fixed large model consistently consumes over 90% of GPU utilization, while using a fixed small model results in a 15% increase in false negatives during peak hours. Although some studies have attempted multi-model switching, they lack a joint perception mechanism for hardware status (such as GPU temperature and utilization) and scene complexity (such as the number of targets). Furthermore, the switching strategy relies on hard thresholds, making it prone to frequent invalid switching due to data jitter, which actually reduces system stability. Therefore, there is an urgent need for an intelligent video inference solution that can balance resource efficiency and inference accuracy in real time and fully utilize frame continuity. Summary of the Invention

[0006] The technical problem to be solved by this invention is the mismatch between resource utilization and performance of video inference models.

[0007] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a video enhancement inference method based on fuzzy control, comprising the following steps:

[0008] S10. Load multiple pre-trained video inference models with different computational complexities;

[0009] S20. Define input parameters that reflect the hardware resource status and scenario complexity, as well as output score parameters that represent the model selection tendency, and define fuzzy sets for each parameter.

[0010] S30. Configure membership functions for each input parameter to map the real-time collected input parameter values ​​to the corresponding fuzzy sets.

[0011] S40. Based on the expert experience knowledge base of the video reasoning scenario, establish the mapping rule between the fuzzy set of input parameters and the fuzzy set of output scores;

[0012] S50. Collect the input parameter values ​​of video frames, map them to fuzzy sets through membership functions, and generate fuzzy subsets of output scores by combining the mapping rules.

[0013] S60. Perform centroid method calculation on the fuzzy subset of the output score to obtain the accurate model selection score;

[0014] S70. Based on the model selection score of the current frame and the preset anti-shaking strategy, determine the optimal video inference model for the next frame.

[0015] Furthermore, in step S10, the multiple pre-trained video inference models with different computational complexities include small-scale models, medium-scale models, and large-scale models, with the medium-scale model being used as the initial frame inference model by default.

[0016] Furthermore, in step S20, the input parameters include:

[0017] GPU utilization is defined as a fuzzy set of {low utilization, medium utilization, high utilization}.

[0018] GPU temperature, whose fuzzy set is defined as {low temperature, medium temperature, high temperature};

[0019] The number of targets in the current frame is defined by a fuzzy set as {few, medium, many}.

[0020] The output scoring parameter is the model recommendation score, and its fuzzy semantic label is {low score, medium score, high score}.

[0021] Furthermore, in step S30, the membership function is a piecewise linear function used to map the value of the input variable to the membership degree in the interval [0,1].

[0022] Furthermore, in step S40: the mapping rules are stored in the form of a fuzzy rule table, which is established based on the video reasoning experience knowledge base and measurement data.

[0023] Furthermore, in step S70, the mapping relationship between the model selection score and the inference model is as follows:

[0024] When the score is in the range [0,30], choose a large-scale model;

[0025] Choose a medium-sized model when the score is in the (30, 50) range;

[0026] Choose a small-scale model when the score is in the range of (50, 100).

[0027] Furthermore, in step S70, the anti-switching jitter strategy is: the model switching operation is only performed when the model selection results of n consecutive frames are consistent and different from the current model.

[0028] The present invention also provides a video enhancement inference device based on fuzzy control, comprising:

[0029] The model loading module is used to load multiple pre-trained video inference models with different computational complexities.

[0030] The fuzzy variable definition module is used to define input parameters that reflect the hardware resource status and scene complexity, as well as output score parameters that represent the model selection tendency, and to define fuzzy sets for each parameter;

[0031] The membership configuration module is used to configure membership functions for each input parameter, mapping the real-time collected input parameter values ​​to the corresponding fuzzy sets;

[0032] The rule base module is used to establish a knowledge base of expert experience based on video reasoning scenarios, and to establish mapping rules between the fuzzy set of input parameters and the fuzzy set of output scores.

[0033] The fuzzy control module is used to collect the input parameter values ​​of video frames, map them to fuzzy sets through membership functions, and generate a fuzzy subset of the output score by combining the mapping rules.

[0034] The scoring conversion module is used to perform centroid calculation on a fuzzy subset of the output scores to obtain an accurate model selection score;

[0035] The model decision module is used to determine the optimal inference model for the next frame based on the model selection score of the current frame and the preset anti-handover jitter strategy.

[0036] The present invention also provides a computer device, the computer device including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the video enhancement inference method based on fuzzy control as described above.

[0037] The present invention also provides a storage medium storing a computer program that, when executed by a processor, can implement the video enhancement inference method based on fuzzy control as described above.

[0038] The beneficial effects of this invention are as follows: By initializing multi-scale models, defining fuzzy variables and mapping rules, constructing fuzzy control rules, performing real-time dynamic inference, and integrating anti-shake strategies, this invention significantly solves the core problems of resource utilization and effect imbalance and lack of inter-frame correlation in video inference. On the one hand, based on fuzzy perception of hardware status (GPU utilization, temperature) and scene complexity (number of targets), it dynamically switches the optimal scale model, automatically downgrading to a smaller model to prevent overload under high load and upgrading to a larger model to improve accuracy under low load, thus achieving synergistic optimization of resource utilization and inference effect. On the other hand, by utilizing the characteristic of continuous target change between video frames, it suppresses frequent switching through anti-shake strategies, avoids excessively frequent model switching, ensures the stability of the inference process, and improves the target detection rate compared to a single model in scenarios such as UAV inspection and traffic monitoring. Attached Figure Description

[0039] The specific structure of the present invention will now be described in detail with reference to the accompanying drawings.

[0040] Figure 1 This is a flowchart of the video enhancement inference method based on fuzzy control according to an embodiment of the present invention;

[0041] Figure 2 This is a fuzzy control architecture diagram according to an embodiment of the present invention;

[0042] Figure 3 This is a membership function diagram of the input and output quantities in an embodiment of the present invention.

[0043] Figure 4 This is a schematic diagram of the center of gravity method according to an embodiment of the present invention;

[0044] Figure 5 This is a comparison diagram of the reasoning effects of embodiments of the present invention;

[0045] Figure 6 This is a comparison chart of inference chip utilization rates in embodiments of the present invention.

[0046] Figure 7 This is a temperature comparison chart of the inference device according to an embodiment of the present invention;

[0047] Figure 8 This is a block diagram of a video enhancement inference device based on fuzzy control according to an embodiment of the present invention;

[0048] Figure 9 This is a schematic block diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0051] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0052] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0053] The first embodiment of the present invention is: a video enhancement inference method based on fuzzy control, comprising the following steps:

[0054] S10. Load multiple pre-trained video inference models with different computational complexities.

[0055] In one specific embodiment, in step S10, the multiple pre-trained video inference models with different computational complexities include small-scale models, medium-scale models, and large-scale models, and the medium-scale model is used as the initial frame inference model by default.

[0056] In this embodiment, several pre-trained video inference models with different parameter sizes are selected. Taking three as examples, they are denoted as small-scale model (model_n), medium-scale model (model_m), and large-scale model (model_l). The medium-scale model (model_m) is selected as the initial frame inference model and loaded into memory.

[0057] S20. Define input parameters that reflect the hardware resource status and scenario complexity, as well as output score parameters that represent the model selection tendency, and define fuzzy sets for each parameter.

[0058] In one specific embodiment, in step S20, the input parameters include:

[0059] GPU utilization is defined as a fuzzy set of {low utilization, medium utilization, high utilization}.

[0060] GPU temperature, whose fuzzy set is defined as {low temperature, medium temperature, high temperature};

[0061] The number of targets in the current frame is defined by a fuzzy set of {few, medium, many}.

[0062] The output scoring parameter is the model recommendation score, and its fuzzy semantic label is {low score, medium score, high score}.

[0063] In this embodiment, the input parameters refer to:

[0064] GPU utilization (GPU_use), GPU temperature (GPU_temp), and the number of targets inferred from the current video frame (Obj_num).

[0065] Their ranges are GPU_use∈[0,100], GPU_temp∈[t1,t2], and Obj_num∈[0,n1], respectively.

[0066] Where t1 and t2 are the minimum and maximum values ​​of GPU_temp, respectively, which are typically the minimum and maximum temperatures at which the GPU operates, and n1 is the maximum number of targets that the current video stream may contain.

[0067] The fuzzy distribution is:

[0068] GPU_use: {low utilization, medium utilization, high utilization} = {UL, UM, UH};

[0069] GPU_temp:{low temperature, medium temperature, high temperature} = {TL, TM, TH};

[0070] Obj_num: {small quantity, medium quantity, large quantity} = {NL, NM, NH};

[0071] Output refers to:

[0072] The score of the current frame is Score, which ranges from Score∈[0,100].

[0073] The fuzzy distribution is:

[0074] Score: {low score, medium score, high score} = {SL, SM, SH}.

[0075] Figure 2The diagram shows the core architecture of the fuzzy controller. In this embodiment, the inputs to the fuzzy controller are GPU temperature, GPU utilization, and the number of inference targets in the current frame; the output is the inference score for the current frame. The knowledge base refers to the expert knowledge or experience used during video inference, as detailed in the fuzzy rule table in Table 1. The number of combinations follows the following pattern:

[0076]

[0077] Where, m i Let be the number of fuzzy sets for the i-th input quantity.

[0078] S30. Configure membership functions for each input parameter to map the real-time collected input parameter values ​​to the corresponding fuzzy sets.

[0079] In one specific embodiment, in step S30, the membership function is a piecewise linear function used to map the value of the input variable to the membership degree in the interval [0,1].

[0080] In this embodiment, a membership function is defined based on the input parameters and output quantities, and the input parameters are fuzzy mapped to specific values ​​into fuzzy subsets. The membership function refers to the function that maps specific input quantities to fuzzy sets. The membership functions of the input quantities GPU_use, GPU_temp, and Obj_num are denoted as μ1, μ2, and μ3, respectively, and their expressions are as follows:

[0081]

[0082] Where x is the input quantity, max(x) is the maximum value of the input quantity, i.e., the right limit of the input quantity, min(x) represents the minimum value of x, i.e., the left limit of the input quantity, and a, b, c, d are all constants, satisfying a <b<c<d。

[0083] Figure 3 This is a membership function for three input quantities, responsible for mapping definite input quantities to fuzzy sets. The horizontal axis represents the specific numerical values ​​of the input or output quantities, and the vertical axis represents the relative size of the corresponding fuzzy subset. Taking GPU_use as an example, when its utilization rate is 35%, its model set is {UL, UM, UH} = {0.3, 0.3, 0}, indicating that its degree of belonging to UL is 0.3, UM is 0.3, and UH is 0.

[0084] S40. Based on the expert experience knowledge base of the video reasoning scenario, establish the mapping rules between the fuzzy set of input parameters and the fuzzy set of output scores.

[0085] In one specific embodiment, in step S40: the mapping rules are stored in the form of a fuzzy rule table, which is established based on the video reasoning experience knowledge base and measurement data.

[0086] In this embodiment, video inference fuzzy control rules are established based on a video inference experience knowledge base and measurement data. This refers to human control based on experience, summarizing the rules for the corresponding tasks. For example, it is generally believed that if the GPU utilization is low, it indicates that there is a large redundancy in computing power, and a larger-scale model should be switched to make full use of computing power; otherwise, a smaller-scale model should be prioritized. If the GPU temperature is low, it indicates that there is still performance redundancy, and a larger-scale model should be switched to fully utilize performance; otherwise, a smaller-scale model should be considered. If the number of targets to be checked in the current frame is large, it is believed that the detection pressure is high, and a large-scale model should be switched to improve the detection effect and detect as many targets as possible; otherwise, a smaller-scale model should be prioritized.

[0087] In practice, this is specifically described using rule tables, as shown in Rule Table 1:

[0088] Table 1: Fuzzy Rule Table

[0089]

[0090]

[0091] S50. Collect the input parameter values ​​of video frames, map them to fuzzy sets through membership functions, and generate fuzzy subsets of output scores by combining the mapping rules.

[0092] In this embodiment, the step of defining a membership function based on the input and output quantities and performing fuzzy mapping on the input quantities to map specific values ​​to a fuzzy subset refers to the operation of mapping the input quantity x to a fuzzy set according to the membership function.

[0093] S60. Perform centroid method calculation on the fuzzy subset of the output score to obtain the accurate model selection score.

[0094] In this embodiment, fuzzy inference is performed, and the fuzzy controller output score is obtained using the centroid method based on the membership function of the output quantity. The membership function of the output quantity refers to the function that clarifies the fuzzy quantity (fuzzy subset) of the output into a specific numerical value, and its expression is as follows:

[0095]

[0096] Where x is the specific value of the output quantity, max(x) is the maximum value of the output quantity, i.e., the right limit of the output quantity, min(x) represents taking the minimum value of x, i.e., the left limit of the output quantity, and a, b, c, d are all constants, satisfying a <b<c<d。

[0097] Based on the membership function of the output, the fuzzy controller output score is obtained using the centroid method, which refers to obtaining the fuzzy subset {s1,s2,s3} of the Score, where s...

[0098]

[0099] Among them, X * This represents the precise value of the output score after decryption, in μ. S (x) is the membership function of the output fuzzy set, x is the value of the output quantity, and the value range of the output quantity Score is [0-100].

[0100] To more clearly illustrate the reasoning process of the fuzzy controller, an example will be used to explain the entire calculation process of its fuzzy inference:

[0101] Assuming the current input values ​​for the measurement are GPU_use = 30, GPU_temp = 36, and Obj_num = 15, their fuzzy subsets can be calculated based on their respective membership functions.

[0102] {UL, UM, UH} = {0.4, 0, 0}

[0103] {TL, TM, TH} = {0.1, 0.4, 0}

[0104] {NL, NM, NH} = {0, 1, 0}

[0105] Based on the fuzzy subset values ​​and the fuzzy rule table, it is known that rules 2 (second row of the rule table) and 5 (fifth row of the rule table) are activated, namely UL, TL, NM, SH and UL, TM, NM, SM. Their activation intensities are calculated as follows: SH = min{0.4, 0.1, 1} = 0.1, SM = min{0.4, 0.4, 1} = 0.4. Based on the membership function of the output and the centroid method calculation formula, the score is approximately 54. A schematic diagram of the centroid method calculation is shown below. Figure 4 As shown.

[0106] S70. Based on the model selection score of the current frame and the preset anti-shaking strategy, determine the optimal video inference model for the next frame.

[0107] In one specific embodiment, in step S70, the mapping relationship between the model selection score and the inference model is as follows:

[0108] Choose a large-scale model when the score is in the range [0,30].

[0109] Choose a medium-sized model when the score is in the (30, 50) range;

[0110] Choose a small-scale model when the score is in the range of (50, 100).

[0111] In this embodiment, the optimal inference model for the next frame is determined based on the output score of the current frame and whether the switching criteria are met. The optimal model is defined as follows: a score between 0 and 30 indicates a large-scale model, 31-50 indicates a medium-scale model, and 51-100 indicates a small-scale model. That is:

[0112] if 0 <Score≤30,then model_n

[0113] if 30 <Score≤50,then model_m

[0114] if 51 <Score≤100,then model_l

[0115] In one specific embodiment, in step S70, the anti-switching jitter strategy is: the model switching operation is only performed when the model selection results of n consecutive frames are consistent and different from the current model.

[0116] In this embodiment, the optimal inference model for the next frame is determined based on the output score of the current frame and whether the switching criteria are met. The switching criteria refer to the anti-shake strategy. Specifically, it avoids model switching errors and excessive and frequent invalid switching caused by model calculation score jitter. Model switching is only performed when the optimal models calculated for n consecutive frames are the same and different from the previous models. Figure 5 This comparison examines the number of targets inferred from the same frame within the same video segment using the method of this invention, in order to ensure fairness. In this example, to accommodate the YOLOv8n model used in general solutions, the method of this invention employs three models of different sizes: YOLOv8n, YOLOv8s, and YOLOv8l. These belong to the YOLOv8 family but differ in size. The dataset used is the benchmark dataset Visdrone2019, and the video stream is from Visdrone2019. The implementation platform is a PC equipped with an AMD 9950x CPU and an NVIDIA 4070Tis GPU, and inference is performed using GPU inference. As can be seen, the method used in this invention significantly outperforms general solutions in terms of the number of targets detected in the same frame, fully demonstrating the superiority of the method of this invention.

[0117] Figure 6 This is a dynamic curve of GPU utilization for each frame of inference during the video inference process, with experimental conditions and... Figure 5 Consistency, from Figure 6 It can be seen that the method proposed in this invention has a much higher GPU utilization rate than general methods, which fully demonstrates that the method proposed in this invention can effectively improve hardware utilization efficiency.

[0118] Figure 7 This is a dynamic GPU temperature curve for each frame of the video inference process, with experimental conditions and... Figure 5 Consistent. From Figure 7 As can be seen, although the method of this invention will slightly increase the GPU temperature, the temperature is quickly and effectively controlled (never exceeding 42 degrees Celsius) due to its real-time dynamic switching characteristics. This strategy is especially effective on some platforms with low computing power.

[0119] In summary, the enhanced video inference method based on fuzzy control proposed in this invention can dynamically switch models for inference based on hardware resources and inference conditions, effectively improving video inference performance and hardware resource utilization.

[0120] like Figure 8 As shown, this embodiment of the invention also provides a video enhancement inference device based on fuzzy control, comprising:

[0121] Model loading module 10 is used to load multiple pre-trained video inference models with different computational complexities;

[0122] The fuzzy variable is defined modulo 20 to define the input parameters that reflect the hardware resource status and scenario complexity, as well as the output score parameters that represent the model selection tendency, and to define fuzzy sets for each parameter.

[0123] The membership configuration module 30 is used to configure the membership function for each input parameter, mapping the real-time collected input parameter values ​​to the corresponding fuzzy set;

[0124] Rule base module 40 is used to establish an expert experience knowledge base for video reasoning scenarios and to establish mapping rules between the fuzzy set of input parameters and the fuzzy set of output scores.

[0125] The fuzzy control module 50 is used to collect the input parameter values ​​of video frames, map them to fuzzy sets through membership functions, and generate a fuzzy subset of the output score by combining the mapping rules.

[0126] The scoring conversion module 60 is used to perform centroid calculation on the fuzzy subset of the output scores to obtain accurate model selection scores;

[0127] The model decision module 70 is used to determine the optimal inference model for the next frame based on the model selection score of the current frame and the preset anti-handover jitter strategy.

[0128] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned fuzzy control-based video enhancement inference device can be referred to the corresponding description in the foregoing method embodiments. For the sake of convenience and brevity, it will not be repeated here.

[0129] The aforementioned fuzzy control-based video enhancement inference device can be implemented as a computer program, which can, for example... Figure 9 It runs on the computer device shown.

[0130] Please see Figure 9 , Figure 9 This is a schematic block diagram of a computer device provided in an embodiment of this application. The computer device 500 can be a terminal or a server. The terminal can be an electronic device with communication functions, such as a smartphone, tablet, laptop, desktop computer, personal digital assistant, or wearable device. The server can be a standalone server or a server cluster composed of multiple servers.

[0131] See Figure 9 The computer device 500 includes a processor 502, a memory, and a network interface 505 connected via a system bus 501. The memory may include a non-volatile storage medium 503 and internal memory 504.

[0132] The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions that, when executed, cause the processor 502 to perform a video enhancement inference method based on fuzzy control.

[0133] The processor 502 provides computing and control capabilities to support the operation of the entire computer device 500.

[0134] The internal memory 504 provides an environment for the execution of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a video enhancement inference method based on fuzzy control.

[0135] This network interface 505 is used for network communication with other devices. Those skilled in the art will understand that... Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device 500 to which the present application is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0136] The processor 502 is used to run the computer program 5032 stored in the memory to implement the video enhancement inference method based on fuzzy control as described above.

[0137] It should be understood that, in the embodiments of this application, the processor 502 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0138] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program includes program instructions and can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0139] Therefore, the present invention also provides a storage medium. This storage medium can be a computer-readable storage medium. The storage medium stores a computer program, wherein the computer program includes program instructions. When executed by a processor, the program instructions cause the processor to perform the fuzzy control-based video enhancement inference method as described above.

[0140] The storage medium can be any computer-readable storage medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.

[0141] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0142] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of each unit is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0143] The steps in the method of this invention can be adjusted, merged, or reduced in order according to actual needs. The units in the device of this invention can be merged, divided, or reduced according to actual needs. Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0144] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0145] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A video enhancement inference method based on fuzzy control, characterized in that, Includes the following steps: S10. Load multiple pre-trained video inference models with different computational complexities; the multiple pre-trained video inference models with different computational complexities include small-scale models, medium-scale models and large-scale models, and the medium-scale model is used as the initial frame inference model by default. S20. Define input parameters reflecting hardware resource status and scene complexity, and output scoring parameters representing model selection bias, and define fuzzy sets for each parameter; the input parameters include: GPU utilization, whose fuzzy set is defined as {low utilization, medium utilization, high utilization}; GPU temperature, whose fuzzy set is defined as {low temperature, medium temperature, high temperature}; current frame target quantity, whose fuzzy set is defined as {few quantity, medium quantity, many quantity}; the output scoring parameter is the model recommendation score, whose fuzzy semantic label is {low score, medium score, high score}; S30. Configure membership functions for each input parameter to map the real-time collected input parameter values ​​to the corresponding fuzzy sets. S40. Based on the expert experience knowledge base of the video reasoning scenario, establish the mapping rule between the fuzzy set of input parameters and the fuzzy set of output scores; S50. Collect the input parameter values ​​of video frames, map them to fuzzy sets through membership functions, and generate fuzzy subsets of output scores by combining the mapping rules. S60. Perform centroid method calculation on the fuzzy subset of the output score to obtain the accurate model selection score; S70. Based on the model selection score of the current frame and the preset anti-switching jitter strategy, determine the optimal inference model for the next frame; the anti-switching jitter strategy is: only when the model selection results of n consecutive frames are consistent and different from the current model, will a model switching operation be performed.

2. The video enhancement inference method based on fuzzy control according to claim 1, characterized in that, In step S30, the membership function is a piecewise linear function used to map the value of the input variable to the membership degree in the interval [0,1].

3. The video enhancement inference method based on fuzzy control according to claim 1, characterized in that, In step S40, the mapping rules are stored in the form of a fuzzy rule table, which is established based on the video reasoning experience knowledge base and measurement data.

4. The video enhancement inference method based on fuzzy control according to claim 1, characterized in that, In step S70, the mapping relationship between the model selection score and the inference model is as follows: When the score is in the range [0, 30], choose a large-scale model; Choose a medium-sized model when the score is in the range of (30, 50); Choose a small-scale model when the score is in the range of (50, 100).

5. A video enhancement inference device based on fuzzy control, characterized in that, include: The model loading module is used to load multiple pre-trained video inference models with different computational complexities; the multiple pre-trained video inference models with different computational complexities include small-scale models, medium-scale models and large-scale models, and the medium-scale model is used as the initial frame inference model by default. The fuzzy variable definition module is used to define input parameters reflecting hardware resource status and scene complexity, as well as output scoring parameters representing model selection bias, and to define fuzzy sets for each parameter. The input parameters include: GPU utilization, whose fuzzy set is defined as {low utilization, medium utilization, high utilization}; GPU temperature, whose fuzzy set is defined as {low temperature, medium temperature, high temperature}; and the number of targets in the current frame, whose fuzzy set is defined as {few, medium, many}. The output scoring parameter is the model recommendation score, whose fuzzy semantic label is {low score, medium score, high score}. The membership configuration module is used to configure membership functions for each input parameter, mapping the real-time collected input parameter values ​​to the corresponding fuzzy sets; The rule base module is used to establish a knowledge base of expert experience based on video reasoning scenarios, and to establish mapping rules between the fuzzy set of input parameters and the fuzzy set of output scores. The fuzzy control module is used to collect the input parameter values ​​of video frames, map them to fuzzy sets through membership functions, and generate a fuzzy subset of the output score by combining the mapping rules. The scoring conversion module is used to perform centroid calculation on a fuzzy subset of the output scores to obtain an accurate model selection score; The model decision module is used to determine the optimal inference model for the next frame based on the model selection score of the current frame and a preset anti-handover jitter strategy. The anti-handover jitter strategy is: a model switching operation is only performed when the model selection results of n consecutive frames are consistent and different from the current model.

6. A computer device, characterized in that: The computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the video enhancement inference method based on fuzzy control as described in any one of claims 1 to 4.

7. A storage medium, characterized in that: The storage medium stores a computer program that, when executed by a processor, can implement the video enhancement inference method based on fuzzy control as described in any one of claims 1 to 4.

Images