High-concurrency python algorithm calling method based on environment isolation and medium

An environment isolation and algorithm technology, applied in software deployment, software design, etc., can solve the problems of different dependent environments, insufficient support, only a few minutes, etc., to save network resources and improve performance.

Active Publication Date: 2021-11-12
中汽数据(天津)有限公司
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] 1) The execution time of general complex deep learning and machine learning algorithms is generally half an hour to a few hours, but the conventional web service call mode is HTTP request, and the HTTP connection can wait for a short time, generally only a few minutes, which is not enough to support
[0005] 2) The TPS of the conventional web service call mode can reach up to 20,000, but it cannot support the current large data volume of hundreds of thousands or millions
[0006] 3) The dependent environments of libraries that may be required by different python algorithms are different and may even conflict. When thousands of python algorithms need to be deployed, conventional server deployment cannot support
[0007] 4) The python code provided by most algorithm engineers will not use web frameworks, only algorithm scripts, which cannot be directly deployed as services

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • High-concurrency python algorithm calling method based on environment isolation and medium
  • High-concurrency python algorithm calling method based on environment isolation and medium
  • High-concurrency python algorithm calling method based on environment isolation and medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0033] figure 1 It is a flow chart of a high-concurrency python algorithm calling method based on environment isolation provided by this embodiment. The method can be executed by an environment-isolated high-concurrency python algorithm calling device, which can be composed of software and / or hardware, and is generally integrated in electronic equipment.

[0034] see figure 1 , the method includes the following steps:

[0035] S110. Deploy the python algorithm script into the docker.

[0036] Preferably, the deployment of the python algorithm script into the docker includes:

[0037] According to the python algorithm script, write the container image file dockerfile, and create a mirror image containing python basic environment in different language versions;

[0038] Saving the deployment work order information to the mysql database, the deployment work order information including the script file and the language environment required by the script;

[0039] According to ...

Embodiment 2

[0077] see figure 2 , this embodiment provides a high-concurrency python algorithm calling device based on environment isolation, including:

[0078] Algorithm script deployment module 101, for deploying python algorithm script in docker;

[0079] The message queue builds and receives module 102, is used for setting up Kafka message queue and receiving message;

[0080] Algorithm calling module 103, used to call the python algorithm in the python algorithm script;

[0081] The execution result return module 104 is used to return the execution result of the python algorithm through the Kafka message queue.

[0082] The device is used for executing the above-mentioned high-concurrency python algorithm calling method based on environment isolation, and thus at least has functional modules corresponding to the method and beneficial effects.

Embodiment 3

[0084] like image 3 As shown, this embodiment provides an electronic device, including:

[0085] at least one processor; and

[0086] a memory communicatively coupled to at least one of said processors; wherein,

[0087] The memory stores instructions executable by at least one of the processors, and the instructions are executed by at least one of the processors, so that at least one of the processors can execute the above method. At least one processor in the electronic device is capable of executing the above method, and thus has at least the same advantages as the above method.

[0088] Optionally, the electronic device further includes interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and can be mounted on a common motherboard or otherwise as desired. The processor can process instructions executed in the electronic device, including storing in or on ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to the field of software development, in particular to a high-concurrency python algorithm calling method based on environment isolation and a medium. The method comprises the following steps of: deploying a python algorithm script into a docker; building and receiving a Kafka message queue; calling a python algorithm in the python algorithm script; and returning an execution result of the python algorithm through the Kafka message queue. According to the method, thousands of python algorithms are deployed on a physical machine by introducing the docker, the environment is isolated, the concurrency of hundreds of thousands of millions of large data volume is realized through the message queue, and the problem that a returned result cannot be received due to long algorithm execution time is solved by re-establishing an independent message return result mode, and the problem that calling cannot be achieved due to the characteristics of large data size, high concurrency, long-time calculation and the like an actual project is solved.

Description

technical field [0001] The invention relates to the field of software development, in particular to a method and medium for invoking a highly concurrent python algorithm based on environment isolation. Background technique [0002] Python is an object-oriented interpreted computer programming language with rich and powerful standard libraries and third-party extension libraries. Because of its simplicity and ease of learning, Python has become one of the most popular programming languages. Due to the simplicity, readability, scalability and complete free of the Python language, there are more and more institutions and personnel using Python for algorithm research at home and abroad. [0003] In practical applications, especially when combined with actual project applications, the execution of algorithms often requires large amounts of data, high concurrency, complex and long-term calculations, and the conventional algorithm call mode can no longer meet this demand. The mai...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F8/20G06F8/60
CPCG06F8/24G06F8/60
Inventor 张聪聪徐洋李鹏旭郝晓东辛祥彬石伟单泽萌申玲彩
Owner 中汽数据(天津)有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap