69 results about "Model inference" patented technology

A configuration model dividing and configuration sub-model inference processing system and procedure addresses the issue of configuration model and query complexity by breaking a configuration problem down into a set of smaller problems, solving them individually and recombining the results into a single result that is equivalent to a conventional inference procedure. In one embodiment, a configuration model is divided into configuration sub-models that can respectively be processed using existing data processing resources. A sub-model inference procedure provides a way to scale queries to larger and more complicated configuration models. Thus, the configuration model dividing and configuration sub-model processing system and inference procedure allows processing by a data processing system of configuration models and queries whose collective complexity exceeds the complexity of otherwise unprocessable conventional, consolidated configuration models and queries.

A method of compressing a pre-trained deep neural network model includes inputting the pre-trained deep neural network model as a candidate model. The candidate model is compressed by increasing sparsity of the candidate, removing at least one batch normalization layer present in the candidate model, and quantizing all remaining weights into fixed-point representation to form a compressed model. Accuracy of the compressed model is then determined utilizing an end-user training and validation data set. Compression of the candidate model is repeated when the accuracy improves. Via the method, the size of the DNN is reduced, implementation requirements, such as memory, hardware, or processing necessities, are also reduced. Inference can be achieved with much increased speed and much decreasedcomputational requirements. Lastly, due to the disclosed method of compression, these benefits occur while guaranteeing accuracy during inference.

The invention discloses a new energy accommodation method based on the power generation frequency limit value adjustment, and the method gives consideration to the improvement of the operation frequency limit value of a power grid in a reasonable range when a system is in normal operation and in a time period of low load, stabilizes the characteristics of randomness, fluctuation and reverse peak regulation of new energy power generation, and improves the accommodation capacity of the system for new energy power generation. The method comprises the steps: constructing a new energy accommodation model inference model based on the power generation frequency limit value adjustment, wherein the system can accommodate more new energy output through primary frequency modulation during normal operation; accommodating the new energy in a low load time period through reducing the technical output of the conventional energy resource, wherein the new energy, which is not completely accommodated, can be used for solving the change transient process of the frequency and a new steady-state value of the system after new energy grid connection through an inference formula after the conventional energy resource has already reached the minimum technical output. According to the embodiment of the invention, the method improves the new energy accommodation capacity of the power grid during normal operation and the low-load time period through the real-time dynamic adjustment of the frequency of the power grid.

The invention discloses a multi-model inference service deployment method and device based on k8s cluster. The method includes: deploying a scheduling service in the smallest scheduling unit of the k8s cluster, and configuring memory, computing resources and scheduling policies for the scheduling service; deploying multiple model inference services according to the memory of the scheduling service, and inferring each model A service is configured to use computing resources of the scheduling service and is configured to be associated with the scheduling service; the scheduling service invokes the plurality of model inference services to process inference tasks according to the scheduling policy. The solution of the present invention realizes the ability of multiple model reasoning services to share the minimum scheduling unit, and the multi-model reasoning service can be elastically scaled with the service load, and the deployment operation is relatively simple.

The present application relates to a method, data processing method, apparatus, computer equipment and storage medium for constructing a model inference network. The method includes: acquiring a deep learning network, where the deep learning network includes a plurality of network layers; acquiring test data; performing compilation and detection on each network layer according to the test data, obtaining compilation results and detection results, and according to the compilation of each network layer The results and / or detection results determine the resource allocation strategy of each network layer, build a model inference network according to the resource allocation strategy of each network layer, and obtain the data to be processed; input the data to be processed into the model inference network, and pass the resources in the model inference network through the model inference network. The network layer whose configuration strategy is to optimize the configuration strategy processes the data to be processed, and the network layer whose resource allocation strategy is the original configuration strategy through the model inference network processes the data to be processed, and obtains the processing result of the data to be processed, which improves the performance of the entire network. Data processing efficiency.