A target detection algorithm service third-party standardized access system and its use method
By deploying the object detection model and FastAPI interface on the GPU server, using the ONNX model and TensorRT's Python API for model loading and inference engine creation, and building a standardized HTTP API interface, solving the problem of third-party access of the object detection model service in the existing technology, which is not standardized, low efficiency and limited scalability, and implements an efficient, convenient and standardized object detection algorithm service.
Patent Information
- Application Number
- CN202411816688.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-11
- Publication Date
- 2025-04-04
AI Technical Summary
In the prior art, third-party access to the target detection model service is not standardized enough, is not efficient and has limited scalability, which leads to users need to have deep learning knowledge and technical capabilities, and need to write specific code for each service to interact, which increases the difficulty and threshold for development and integration.
Design a third-party standardized access system for object detection algorithms. By deploying object detection models and FastAPI interfaces on GPU servers, using ONNX model and TensorRT's Python API for model loading and inference engine creation, and building a standardized HTTP API interface, so that third parties can access object detection models through HTTP APIs.
It realizes efficient loading, inference and result return of the object detection model, improves the availability and convenience of algorithm services, reduces system resource consumption, simplifies operation and use processes, adapts to different service needs, and improves user experience and system scalability.
Smart Images
Figure CN119762945A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the fields of computer vision, service technology and Web backend technology, and relates to a target detection algorithm service third-party standardized access system and a use method thereof. Background Art
[0002] In the existing technology, the deployment of target detection algorithms requires users to have certain deep learning knowledge and technical capabilities, and to have an in-depth understanding of model conversion, environment configuration, and dependency library compatibility. Existing target detection models usually exist as software packages or independent applications, lacking a unified and standard interface for accessing different model services. Users need to write specific code for each service to interact.
[0003] When users need to integrate target detection algorithms into their own services or products, additional development work is required to ensure that the model is compatible with existing systems. How to deal with model compatibility issues and the model's reasoning performance issues must also be considered. These issues set a high threshold for general users to use target detection algorithms.
[0004] Therefore, a target detection algorithm service third-party standardized access system and its use method are designed to overcome the above problems. Summary of the invention
[0005] The purpose of the present invention is to overcome the technical problems in the prior art that third-party access to target detection model services is not standardized enough, inefficient and has limited scalability, and to provide a target detection algorithm service third-party standardized access system and its use method that improves the efficiency and convenience of target detection while reducing system resource consumption and enhancing the availability and adaptability of the system, so that third parties can access the target detection model in an efficient and standardized manner through the HTTP API, thereby improving access efficiency, facilitating integration and expansion, and adapting to different service requirements.
[0006] The present invention is implemented by the following technical solution: a target detection algorithm service third-party standardized access system, the system includes target detection service hardware and target service detection software, the target detection service hardware includes: GPU server, used to deploy the target detection model and FastAPI interface. The server needs to have sufficient computing power to run the target detection algorithm. Network equipment, used to ensure that the server can provide external services through the network; Storage devices, SSDs are used to increase data reading speed and ensure fast response of the inference engine; The target detection service software includes: Object detection ONNX model: used to perform object detection tasks; TensorRT Python API: used to load ONNX models on the server and create an inference engine; FastAPI: used to create HTTP API interfaces that receive external data input and return inference results.
[0007] As a preferred embodiment: the HTTP API interface is created, and the third party can send request data to the target detection system general interface through the HTTP general protocol, and use the inference engine created above to infer the result, that is, the target detection result. When the inference engine processes the input data to obtain the detection result, the system will package the target detection result obtained by the TensorRT Python API into structured information and return it to the API caller, that is, the third-party user.
[0008] A method for using a third-party standardized access system for target detection algorithm services, the step 1) deploying a trained target detection ONNX model to a server; Step 2) Use TensorRT’s Python API to load the model on the server and create an inference engine. Step 3) Use the FastAPI framework to build a standardized HTTP API interface; Step 4) Provide interface services to the outside world. Third-party systems can send pictures or video data to the server through HTTP API. Step 5) After receiving the data, the server processes it through the inference engine to obtain the target detection result; Step 6) Return the processing result to the API caller through the FastAPI interface.
[0009] As a preferred embodiment: the specific method in the specific steps 4-5) is: a. The API caller sends a request containing image or video data to the API interface via HTTP request; b. After receiving the request, the server first caches the data in memory; c. The TensorRT inference engine takes over the data and performs the target detection task; d. The detection results are organized according to a predetermined format, and the detected target location, category, confidence level and other information are encoded into JSON format; The e.fastAPI interface returns the processed detection results to the API caller in the form of an HTTP response body.
[0010] The beneficial effects of the present invention are as follows: The target detection algorithm service third-party standardized access system and its use method designed by the present invention can well solve the technical problems in the prior art that the target detection model service third-party access is not standardized enough, inefficient and has limited scalability. That is, the present invention can enable third parties to access the target detection model in an efficient and standardized manner through the HTTP API, thereby improving access efficiency, facilitating integration and expansion, and adapting to different service requirements. The present invention realizes the efficient loading, reasoning and result return of the target detection model through a series of technical means, which can greatly improve the availability and convenience of the algorithm service.
[0011] The system in the present invention uses the ONNX model, an open neural network exchange format, as a bridge for target detection, so that the system is not restricted by a specific deep learning framework, is compatible with different computing environments, and can support more types of target detection models. When new algorithms or models appear in the future, the system can be easily expanded and updated.
[0012] Using TensorRT's Python API for model loading and inference engine creation can make more efficient use of computing resources, significantly reducing the required computing time and energy consumption. For users who need high-frequency and large-scale target detection, this invention can process more data in a shorter time and improve the overall detection efficiency. For complex target detection tasks, resource savings will directly translate into cost reduction.
[0013] The present invention also provides a simple and easy-to-use interface, so that third-party users can easily access the service through a standard HTTP API without having to worry about complex implementation details. Complex target detection tasks can be completed using simple HTTP requests, which significantly improves the user experience and reduces the technical threshold for users in the integration and use process. Due to the versatility of the HTTP API, the service of the present invention can be widely used in various computing environments, and has good adaptability to different operating environments, from small mobile devices to large servers, and is adaptable to a variety of hardware configurations, and can send requests through the network to perform target detection. This environmental adaptability allows the present invention to cover a wider range of application scenarios.
[0014] In summary, the technical features of the present invention, through reasonable structural design and process flow, realize the efficiency, popularization and energy saving of target detection services, improve the efficiency, quality and accuracy of target detection, simplify the operation and use process, reduce costs and resource consumption, and have practical application value and market potential. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION
[0016] In order to enable those skilled in the art to more clearly understand the objectives, technical solutions and advantages of the present invention, the present invention is further described below in conjunction with the accompanying drawings and embodiments.
[0017] In the description of the present invention, it is necessary to understand that the orientation or position relationship indicated by the terms such as "up", "down", "left", "right", "inside", "outside", "horizontal" and "vertical" is based on the orientation or position relationship shown in the accompanying drawings, and is only for the convenience of describing the present invention, rather than indicating or implying that the device or element referred to must have a specific orientation, and therefore cannot be understood as a limitation on the present invention.
[0018] The present invention will be described in detail below with reference to the accompanying drawings: Figure 1 As shown, a target detection algorithm service third-party standardized access system includes target detection service hardware and target service detection software, and the target detection service hardware includes: GPU server, used to deploy the target detection model and FastAPI interface. The server needs to have sufficient computing power to run the target detection algorithm. Network equipment, used to ensure that the server can provide external services through the network; Storage devices, SSDs are used to increase data reading speed and ensure fast response of the inference engine; The target detection service software includes: Object detection ONNX model: used to perform object detection tasks; TensorRT Python API: used to load ONNX models on the server and create an inference engine; FastAPI: used to create HTTP API interfaces that receive external data input and return inference results.
[0019] By creating the HTTP API interface, a third party can send request data to the target detection system general interface through the HTTP general protocol, and use the inference engine created above to infer the result, that is, the target detection result. After the inference engine processes the input data and obtains the detection result, the system will package the target detection result obtained by the TensorRT Python API into structured information and return it to the API caller, that is, the third-party user.
[0020] A method for using a third-party standardized access system for target detection algorithm services, the step 1) deploying a trained target detection ONNX model to a server; Step 2) Use TensorRT’s Python API to load the model on the server and create an inference engine. Step 3) Use the FastAPI framework to build a standardized HTTP API interface; Step 4) Provide interface services to the outside world. Third-party systems can send pictures or video data to the server through HTTP API. Step 5) After receiving the data, the server processes it through the inference engine to obtain the target detection result; Step 6) Return the processing result to the API caller through the FastAPI interface.
[0021] The specific method in the specific steps 4-5) is: a. The API caller sends a request containing image or video data to the API interface via HTTP request; b. After receiving the request, the server first caches the data in memory; c. The TensorRT inference engine takes over the data and performs the target detection task; d. The detection results are organized according to a predetermined format, and the detected target location, category, confidence level and other information are encoded into JSON format; The e.fastAPI interface returns the processed detection results to the API caller in the form of an HTTP response body. Example
[0022] The system of the present invention uses ONNX (Open Neural Network Exchange Format) to define the model of target detection. The ONNX model is used as the input of the target detection model. The ONNX model is an open model format standard that allows model sharing between different deep learning frameworks. Using this format can ensure the compatibility and versatility of the model, so that more third-party developers and users can take advantage of this service.
[0023] The system in the present invention uses TensorRT to load the ONNX model and uses TensorRT's Python API to access the loaded ONNX model. TensorRT is a high-performance neural network inference engine for deep learning reasoning, which can optimize the model to improve reasoning speed and efficiency. By using TensorRT, the ONNX model can be optimized, including layer fusion, precision calibration, kernel automatic tuning, etc., to adapt to specific hardware platforms and improve reasoning performance, ensuring the efficient operation of the target detection model in practical applications.
[0024] Then you need to use TensorRT to create an inference engine, which is responsible for processing input data, executing the forward propagation of the model, performing the target detection calculation process, and obtaining the result through the optimized model. Through the inference engine, you can achieve a rapid response to target detection tasks and provide users with real-time services.
[0025] In order to enable third parties to access target detection services in a standardized manner, the present invention uses FastAPI to create an external standardized access interface, where FastAPI is a modern high-performance Web framework used to build a standardized universal HTTPAPI interface to provide unified services for third-party callers.
[0026] Through the created interface, the third party can send request data to the target detection system general interface through the HTTP general protocol, and use the inference engine created above to infer the result, that is, the target detection result. After the inference engine processes the input data and obtains the detection result, the system will package the target detection result obtained by the TensorRT Python API into structured information and return it to the API caller, that is, the third-party user.
[0027] In summary, the technical solution of the present invention realizes an efficient, convenient and standardized target detection algorithm service by using a series of technical means such as ONNX model, TensorRT optimization, inference engine creation, FastAPI interface construction, etc., so that third-party users can easily access and use the target detection service through HTTP API.
[0028] The target detection algorithm service third-party standardized access system and its use method designed by the present invention can well solve the technical problems in the prior art that the target detection model service third-party access is not standardized enough, inefficient and has limited scalability. That is, the present invention can enable third parties to access the target detection model in an efficient and standardized manner through the HTTP API, thereby improving access efficiency, facilitating integration and expansion, and adapting to different service requirements. The present invention realizes the efficient loading, reasoning and result return of the target detection model through a series of technical means, which can greatly improve the availability and convenience of the algorithm service.
[0029] The system in the present invention uses the ONNX model, an open neural network exchange format, as a bridge for target detection, so that the system is not restricted by a specific deep learning framework, is compatible with different computing environments, and can support more types of target detection models. When new algorithms or models appear in the future, the system can be easily expanded and updated.
[0030] Using TensorRT's Python API for model loading and inference engine creation can make more efficient use of computing resources, significantly reducing the required computing time and energy consumption. For users who need high-frequency and large-scale target detection, this invention can process more data in a shorter time and improve the overall detection efficiency. For complex target detection tasks, resource savings will directly translate into cost reduction.
[0031] The present invention also provides a simple and easy-to-use interface, so that third-party users can easily access the service through a standard HTTP API without having to worry about complex implementation details. Complex target detection tasks can be completed using simple HTTP requests, which significantly improves the user experience and reduces the technical threshold for users in the integration and use process. Due to the versatility of the HTTP API, the service of the present invention can be widely used in various computing environments, and has good adaptability to different operating environments, from small mobile devices to large servers, and is adaptable to a variety of hardware configurations, and can send requests through the network to perform target detection. This environmental adaptability allows the present invention to cover a wider range of application scenarios.
[0032] In summary, the technical features of the present invention, through reasonable structural design and process flow, realize the efficiency, popularization and energy saving of target detection services, improve the efficiency, quality and accuracy of target detection, simplify the operation and use process, reduce costs and resource consumption, and have practical application value and market potential.
[0033] The specific embodiments described herein are merely illustrative of the principles and effects of the present invention and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed by the present invention shall still be covered by the claims of the present invention.
Claims
1. A target detection algorithm service third-party standardized access system, the system includes target detection service hardware and target service detection software, characterized in that: The target detection service hardware includes: GPU server, used to deploy the target detection model and FastAPI interface. The server needs to have sufficient computing power to run the target detection algorithm. Network equipment, used to ensure that the server can provide external services through the network; Storage devices, SSDs are used to increase data reading speed and ensure fast response of the inference engine; The target detection service software includes: Object detection ONNX model: used to perform object detection tasks; TensorRT Python API: used to load ONNX models on the server and create an inference engine; FastAPI: used to create HTTP API interfaces that receive external data input and return inference results.
2. The target detection algorithm service third-party standardized access system according to claim 1, characterized in that: By creating the HTTP API interface, a third party can send request data to the target detection system general interface through the HTTP general protocol, and use the inference engine created above to infer the result, that is, the target detection result. After the inference engine processes the input data and obtains the detection result, the system will package the target detection result obtained by the TensorRT Python API into structured information and return it to the API caller, that is, the third-party user.
3. A method for using the target detection algorithm service third-party standardized access system according to claim 1 or 2, characterized in that: The step 1) deploys the trained object detection ONNX model to the server; Step 2) Use TensorRT’s Python API to load the model on the server and create an inference engine. Step 3) Use the FastAPI framework to build a standardized HTTP API interface; Step 4) Provide interface services to the outside world. Third-party systems can send pictures or video data to the server through HTTP API. Step 5) After receiving the data, the server processes it through the inference engine to obtain the target detection result; Step 6) Return the processing result to the API caller through the FastAPI interface.
4. The method according to claim 3, characterized in that The specific method in the specific steps 4-5) is: a. The API caller sends a request containing image or video data to the API interface via HTTP request; b. After receiving the request, the server first caches the data in memory; c. The TensorRT inference engine takes over the data and performs the target detection task; d. The detection results are organized in a predetermined format, and the detected target location, category, and confidence information are encoded in JSON format; The e.fastAPI interface returns the processed detection results to the API caller in the form of an HTTP response body.