Distributed system for large-scale deep learning inference

Abstract

According to the distributed system for large-scale deep learning inference, tasks are decoupled step by step by adopting queues, so that the stability of a back-end detection thread is improved, and the problem of high concurrency of the system is solved. The system comprises a service layer, a workflow layer and an infrastructure layer, the service layer comprises an inference logic business block, a model training business block and a public business block, the inference logic business block bears business logic for inference and judgment of a system, and the model training business block bears business logic for system training. The public service block undertakes public logic of the inference logic service block and the model training service block; the workflow layer comprises a plurality of access points and a plurality of detection pipelines, the access points provide access ports of access protocols required by the equipment, and the detection pipelines execute detection and return results; and the infrastructure layer is used for providing basic services required by the service layer. The method is applied to the technical field of computing platform systems.

Description

technical field [0001] The invention is applied to the technical field of computing platform systems, and particularly relates to a distributed system for large-scale deep learning inference. Background technique [0002] In industrial production and inspection, it is necessary to judge the position of the product. The commonly used detection methods include photoelectric sensors to perform in-place detection and machine vision system detection. The machine vision system uses machine vision products to ingest the target according to pixel distribution and brightness, color and other information. It is converted into a digital signal and sent to a dedicated image processing system. The image processing system performs various operations on these signals to extract the characteristics of the target, and then controls the on-site equipment actions according to the results of the discrimination. [0003] However, the calculation model and processing accuracy of the in-place dete...

AI Technical Summary

Problems solved by technology

[0005] 1. Generally, a single application architecture is adopted, the system has poor fault tolerance and cannot dynamically and quickly change the computing power support

[0006] 2. Deep learning inference is a long and time-consuming process, so the system is in a high concurrency state in most cases, which can easily cause system stability to decline or even become unavailable

[0007] 3. Deep learning has a wide range of application scenarios. It not only needs to be connected with a variety of different devices, but also serves a variety of different processes in the entire project process, such as positioning, image optimization and various detections. Most of the existing technologies provide software tools The method of docking is carried out through the package, which has a high degree of coupling and a high degree of intrusion into the business of the upper computer, which will expand the range of explosions due to various errors

Images