Construction method of accelerated deep learning model and factory AI visual system
Through the accelerated deep learning model construction method based on TensorRT, the problem of high time cost and poor detection effect caused by relying on manual factory visual detection is solved, and efficient, low-cost, integrated visual detection is achieved, improving detection speed and accuracy , and real-time results push through intelligent networking.
Patent Information
- Application Number
- CN202410980563.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-22
- Publication Date
- 2024-11-12
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Modern manufacturing factories rely on manual inspection in visual projects, resulting in problems such as high time cost, poor detection effect, low degree of intelligent networking, high cost, low software integration and low operability.
Using the accelerated deep learning model construction method based on TensorRT, including determining tasks and data sets, designing deep learning models, converting them into TensorRT format, configuring optimizers, building inference engines, and integrating factory AI vision systems to provide functional selection, data acquisition, Accelerate model loading, database and result display modules to realize automated detection of visual tasks.
It realizes high-efficiency, low-cost and highly integrated visual inspection, improves detection speed and accuracy, reduces labor costs, and realizes real-time results push through intelligent networking.
Smart Images

Figure CN118941891A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of computer vision, and in particular to a method for constructing an accelerated deep learning model based on TensorRT and a factory AI vision system. Background Art
[0002] Modern manufacturing factories have a demand for automation of some processes that can be replaced by machines, such as visual projects, identification of whether staff members are wearing helmets, masks, and work clothes in accordance with regulations in daily management. Identification of the opening and closing of one-way valves in some work scenarios, flame and smoke detection and warning, and warning identification of areas where unauthorized personnel are prohibited from entering dangerous areas. Faced with the above needs, manufacturers mostly use manual methods for identification and screening, which brings the following problems: First, manual inspection is time-consuming: manual inspection is very time-consuming. For example, there are many one-way valves inside a factory. If manual identification is required, special personnel will be required. The large number of valves will increase the time cost.
[0003] Second, the detection effect is poor: manual detection is completely dependent on human energy and knowledge reserves. In some key positions such as fire forecasting, continuous monitoring is required, which leads to long-term repetitive work by personnel. Over time, their energy will decline, resulting in failure to provide timely warnings.
[0004] Third, the degree of intelligent networking is not high: in modern society, mobile phones, tablets, computers and other devices frequently appear in our daily life. If various test results are pushed to relevant personnel in a timely manner through intelligent networking, work efficiency will inevitably be improved. However, using manual methods cannot achieve real-time results.
[0005] Fourth, the cost is relatively high: using machines as a substitute can save time and labor costs. If the same computing power is to be achieved, the cost of the host will be relatively high.
[0006] Fifth, the degree of software integration is low: model loading, function implementation, module communication, etc. are independent of each other in traditional methods.
[0007] Sixth, the software operability is low: many operating software are too complicated, and operators find it difficult to handle overly complex interfaces. Summary of the invention
[0008] In response to the above-mentioned technical problems in the related art, the present invention provides a method for accelerating the construction of a deep learning model and a factory AI vision system, which can solve the above-mentioned problems.
[0009] To achieve the above technical objectives, the technical solution of the present invention is implemented as follows: A method for accelerating the construction of a deep learning model includes the following steps: S100, build deep learning models based on different tasks and datasets; S200, builds TensorRT's accelerated deep learning model based on different deep learning models; S210, Model Optimization: Convert deep learning models to a format supported by TensorRT; S220, Network definition: Use TensorRT API to define the corresponding network in C++ or Python for the deep learning model converted to the format supported by TensorRT; S230, optimizer configuration: configure the TensorRT optimizer to optimize the network according to the hardware and performance requirements of the project; S240, Build Engine: Build an optimized inference engine using the TensorRT API.
[0010] Furthermore, step S100 also includes: S110, determine the tasks and data sets: determine the tasks of the project and collect data sets related to the tasks; S120, data preprocessing: preprocessing the data to improve the training effect of the model; S130, Select model architecture: Select an appropriate model architecture based on the nature of the task and the characteristics of the dataset; S140, Model design: Based on the selected model architecture, design the specific model structure; S150, model training: Use the training data set to train the designed model.
[0011] Furthermore, the tasks of the project determined in step S110 include image classification, object detection, and semantic segmentation.
[0012] Furthermore, step S150 also includes: S151, Model evaluation: Use the test dataset to evaluate the performance of the trained model, including accuracy, recall, and F1 score; S152. Model optimization: Optimize the model based on the evaluation results.
[0013] Furthermore, in step S210, the deep learning model is exported from the machine learning framework to the TensorRT format, or an ONNX model is used for intermediate conversion.
[0014] Furthermore, the network defined in the TensorRT format in step S220 includes defining the network structure, the input and output of the layer, and the computational graph of the network.
[0015] Furthermore, the construction of the engine in step S240 is to build an efficient reasoning engine based on the network definition and optimizer configuration parameters.
[0016] Factory AI vision system, including: Function selection module, select corresponding function options according to usage requirements; The acceleration model loading module loads the corresponding acceleration model according to the selected function options. Each function option corresponds to an acceleration model, which is an accelerated deep learning model based on TensorRT. The data acquisition module collects data from the detection end according to the selected function options, and transmits the collected data to the inference engine in the acceleration model. The inference engine performs inference analysis on the collected data and outputs the detection results; Database module, which keeps the test results in a historical database; The result display module displays the test results on the display interface.
[0017] Furthermore, the acceleration model loading module integrates an acceleration model for detecting the opening and closing of a one-way valve, an acceleration model for detecting whether a safety helmet is worn, an acceleration model for detecting whether a mask is worn, an acceleration model for detecting liquid accumulation in a separation barrel, an acceleration model for detecting flames and smoke, an acceleration model for identifying whether pedestrians have entered a dangerous area, and an acceleration model for detecting whether work clothes are worn.
[0018] Beneficial effects of the invention: The present invention has the characteristics of high scalability, high efficiency, low cost and high integration. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0020] The present invention is further described in detail below with reference to the accompanying drawings.
[0021] Figure 1 It is a schematic diagram of the construction and use of the accelerated deep learning model based on TensorRT described in an embodiment of the present invention; Figure 2 This is a schematic diagram of the use principle of the factory AI vision system described in an embodiment of the present invention. DETAILED DESCRIPTION
[0022] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.
[0023] According to the present invention, a method for constructing an accelerated deep learning model is disclosed. First, a deep learning model is constructed, and then an accelerated deep learning model of TensorRT is constructed based on the deep learning model.
[0024] The deep learning model is constructed in the following steps: Determine the task and dataset: First, determine the task of the project, such as image classification, object detection, semantic segmentation, etc., and collect task-related datasets.
[0025] Data preprocessing: Preprocess the data, including but not limited to resizing, standardization, data augmentation, etc., to improve the training effect of the model.
[0026] Choose the model architecture: According to the nature of the task and the characteristics of the dataset, choose an appropriate model architecture, such as convolutional neural network (CNN), recurrent neural network (RNN), Transformer, etc.
[0027] Model design: Based on the selected model architecture, design the specific model structure, including the number of network layers, the number of neurons in each layer, activation functions, etc.
[0028] Model training: Train the designed model using the training dataset. This involves choosing a loss function, an optimizer, and iteratively passing data to the model to adjust the model parameters to minimize the loss function.
[0029] Model evaluation: Use the test dataset to evaluate the performance of the trained model, including indicators such as accuracy, recall, and F1 score.
[0030] Model optimization: Based on the evaluation results, the model is optimized, which may include adjusting hyperparameters, adjusting model structure, adding regularization, etc.
[0031] Deployment and application: Deploy the trained model to the actual application system, which can be an embedded device, server or mobile terminal, and integrate it into a customized application to solve practical problems.
[0032] The accelerated deep learning model of TensorRT is carried out in the following steps: Model optimization: First, convert the deep learning model to a format supported by TensorRT. This usually involves exporting the deep learning model from a machine learning framework (such as TensorFlow, PyTorch, etc.) to the TensorRT format, or using the ONNX model for intermediate conversion.
[0033] Network definition: Then use the TensorRT API to define the corresponding network in C++ or Python using the TensorFlow, PyTorch, or other models to be applied. These networks defined in TensorRT format include the definition of the network structure, layer inputs and outputs, and the network's computational graph.
[0034] Optimizer configuration: Next, you need to configure TensorRT's optimizer, which can optimize the network according to the hardware and performance requirements of the project. These optimizations involve selecting appropriate optimizer configuration parameters, such as the maximum batch size, maximum workspace size, etc.
[0035] Build the engine: Finally, you can use the TensorRT API to build an optimized inference engine. The engine construction will build an efficient inference engine based on the network definition and optimizer configuration parameters in the previous steps.
[0036] Inference: After the TensorRT inference engine is built, you can perform inference. You can provide input data to the engine and get output results.
[0037] A factory AI vision system is also provided, including: Function selection module, select corresponding function options according to usage requirements; The acceleration model loading module loads the corresponding acceleration model according to the selected function options. Each function option corresponds to an acceleration model, which is an accelerated deep learning model based on TensorRT. The data acquisition module collects data from the detection end according to the selected function options, and transmits the collected data to the inference engine in the acceleration model. The inference engine performs inference analysis on the collected data and outputs the detection results; Database module, which keeps the test results in a historical database; The result display module displays the test results on the display interface.
[0038] Embodiment 1: like Figure 1The figure shows the inference acceleration process of the visual model using TensorRT. The model trained by machine learning frameworks such as PyTorch or TensorFlow is exported to ONNX format. After the network is built using TensorRT, the ONNX model is parsed to obtain the final acceleration model that can be executed for inference. After the acquired image is resized, channel converted, and normalized, inference can be performed to obtain the detection result.
[0039] Embodiment 2: like Figure 2 The figure shows the use of the AI vision system. This system provides a variety of acceleration models to choose from. Each acceleration model corresponds to a function. Therefore, the user needs to select the function first when using it. After the selection, the system will call the corresponding acceleration model to perform reasoning. After the detection results are obtained, they will be displayed in the user interaction interface and will also be retained in the historical database.
[0040] Embodiment three: The system of this application integrates a variety of visual acceleration models, specifically detecting the opening and closing of a one-way valve, detecting whether a helmet is worn, detecting whether a mask is worn, segmenting the accumulated liquid in the barrel, detecting flames and smoke, identifying whether pedestrians have entered a dangerous area, and detecting whether work clothes are worn. The above models can not only assist enterprises in the daily management of employees, but also assist enterprises in automating and upgrading specialized production links through customized production, and can also complete manual replacement in dangerous situations. The present invention not only meets the daily needs of the factory, but also realizes the professional and personalized needs of the factory, and integrates them into a system.
[0041] For the opening and closing detection of the one-way valve, first predefine the opening and closing shapes of the one-way valve, collect a certain number of original images of the one-way valve including the opening and closing modes, use image acquisition and annotation software to annotate the sampled images, obtain the pixel coordinates and category labels of the one-way valve in the sampled images and save them to txt files, build a complete one-way valve training set, and then use the pytorch framework to train the data set. The specific training steps are carried out according to the above-mentioned deep learning model building method, import the trained model into Tensorrt, build the relevant acceleration model according to the above-mentioned Tensorrt acceleration steps, and finally integrate the model into the application system. After the user turns on the one-way valve opening and closing detection function, the application interface will display the real-time picture and the captured one-way valve and detection information, and all the detection results will be saved in the historical database for subsequent use.
[0042] For the detection of helmets, masks, and work clothes, first determine the corresponding styles of helmets, masks, and work clothes. If there are multiple styles of clothing, number them for different styles. This number corresponds to the style category and name of the helmets, masks, and work clothes. Then sample the images for the relevant styles. Other functions are the same as the one-way valve.
[0043] For flame and smoke detection, first determine whether the use scenario requires fill light, then determine the alarm threshold of flame and smoke, that is, the effective pixel area of flame and smoke in the image, and then match the corresponding alarm level according to the different pixel areas, effectively establishing a multi-level alarm system. Other functions are the same as the one-way valve.
[0044] For the segmentation of the effusion in the barrel, we first build a training data set for the effusion. According to the shape of the effusion, we use the annotation software to perform polygon annotation along the edge of the effusion in the image, and finally obtain the original image of the effusion and the corresponding polygon coordinate set. After that, we conduct deep learning training and build an acceleration model. Other functions are the same as those of the one-way valve.
[0045] For pedestrian intrusion recognition in dangerous areas, the scope of the dangerous area is first determined, and then the warning information after the pedestrian intrusion is determined, that is, the warning light flashes, the audio broadcast prompts, etc. Other functions are the same as the one-way valve.
[0046] In summary, the present invention has the characteristics of high scalability, high efficiency, low cost and high integration. The present invention can bring the following advantages: Acceleration effect on visual models: By optimizing the visual model with TensorRT, it is possible to reduce the complexity of the model structure, improve the model operation efficiency, and accelerate the model detection speed.
[0047] High scalability of visual models: The visual functions required by enterprises vary, which requires the ability to provide personalized professional customized models. The system designed by the present invention provides a standardized interface that can achieve access to different visual models and has high scalability.
[0048] Lower usage cost: The present invention can be implemented on an edge computing box, which has lower cost than a host. It can be connected to the factory's production system through network devices such as network switches, without the need for more equipment, thus bringing the advantage of low cost.
[0049] High integration: The present invention is a highly integrated visual system. The visual needs of different scenarios can be connected to the system. Through intelligent networking technology, all detection results can be pushed to relevant personnel in real time according to the manufacturer's requirements, which has a high integration advantage.
[0050] Strong interactivity: The present invention is a highly operable system that fully considers the logical thinking of operators, can adopt one-key function processing, has a simple user operation interface and high operability.
[0051] Through the processing of this system, users can improve their production efficiency.
[0052] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for accelerating the construction of a deep learning model, characterized in that: The steps include: S100, build deep learning models based on different tasks and datasets; S200, builds TensorRT's accelerated deep learning model based on different deep learning models; S210, Model Optimization: Convert deep learning models to a format supported by TensorRT; S220, Network definition: Use TensorRT API to define the corresponding network in C++ or Python for the deep learning model converted to the format supported by TensorRT; S230, optimizer configuration: configure the TensorRT optimizer to optimize the network according to the hardware and performance requirements of the project; S240, Build Engine: Build an optimized inference engine using the TensorRT API.
2. The method for constructing an accelerated deep learning model according to claim 1, characterized in that: Step S100 also includes: S110, determine the tasks and data sets: determine the tasks of the project and collect data sets related to the tasks; S120, data preprocessing: preprocessing the data to improve the training effect of the model; S130, Select model architecture: Select an appropriate model architecture based on the nature of the task and the characteristics of the dataset; S140, Model design: Based on the selected model architecture, design the specific model structure; S150, model training: Use the training data set to train the designed model.
3. The method for constructing an accelerated deep learning model according to claim 2, characterized in that: The tasks of the project determined in step S110 include image classification, object detection, and semantic segmentation.
4. The method for constructing an accelerated deep learning model according to claim 2, characterized in that: Step S150 also includes: S151, Model evaluation: Use the test data set to evaluate the performance of the trained model, including accuracy, recall, and F1 score; S152. Model optimization: Optimize the model based on the evaluation results.
5. The method for constructing an accelerated deep learning model according to claim 1, characterized in that: In step S210, the deep learning model is exported from the machine learning framework to the TensorRT format, or an ONNX model is used for intermediate conversion.
6. The method for constructing an accelerated deep learning model according to claim 1, characterized in that: The network defined in the TensorRT format in step S220 includes defining the network structure, the input and output of the layer, and the computational graph of the network.
7. The method for constructing an accelerated deep learning model according to claim 1, characterized in that: The construction of the engine in step S240 is to build an efficient reasoning engine based on the network definition and optimizer configuration parameters.
8. Factory AI vision system, characterized by: include: Function selection module, select corresponding function options according to usage requirements; An acceleration model loading module, which loads a corresponding acceleration model according to a selected function option, each function option corresponds to an acceleration model, and the acceleration model is an accelerated deep learning model as described in any one of claims 1 to 8; The data acquisition module collects data from the detection end according to the selected function options, and transmits the collected data to the inference engine in the acceleration model. The inference engine performs inference analysis on the collected data and outputs the detection results; Database module, which keeps the test results in a historical database; The result display module displays the test results on the display interface.
9. The factory AI vision system according to claim 8, characterized in that: The acceleration model loading module integrates an acceleration model for detecting the opening and closing of a one-way valve, an acceleration model for detecting whether a safety helmet is worn, an acceleration model for detecting whether a mask is worn, an acceleration model for detecting liquid accumulation in a dividing barrel, an acceleration model for detecting flames and smoke, an acceleration model for identifying whether a pedestrian has entered a dangerous area, and an acceleration model for detecting whether work clothes are worn.
Citation Information
Patent Citations
Method and device for accelerating deep learning image recognition based on TensorRT, and medium
CN112364744A
Sheep individual identity recognition method and system based on deep metric learning
CN116798066A
Vehicle flow detection method based on Engine
CN117475383A
ONNX-based target detection model library system construction method
CN117975246A
Cited By
AI accelerator construction method and related device
CN120994247A
