Industrial bubble detection method, system and equipment based on lightweight detection model and medium

Abstract

The invention provides an industrial bubble detection method, system and device based on a lightweight detection model, and a medium, and the method comprises the steps: obtaining gas-liquid two-phase flow working condition video data, and carrying out the frame extraction and bubble target labeling, and obtaining a bubble recognition data set; based on YOLOv5, YOLOv8 and YOLOv11s, constructing a bubble detection model evolved generation by generation, and training the model by using a bubble identification data set; and performing bubble detection on real-time gas-liquid two-phase flow working condition video data acquired in an industrial field based on the trained lightweight bubble detection model, executing a frame discarding strategy and a frame rate scheduling mechanism, and outputting a detection reasoning result. The problems that in the gas-liquid two-phase flow technological process, bubble recognition precision is low, calculation efficiency is insufficient, deployment is complex, and system real-time performance is poor are solved, and particularly under the actual working conditions that the background is complex, targets are adhered, small bubbles are dense, and video images are high in interference, an existing detection method cannot give consideration to the detection effect and engineering practicability at the same time.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of artificial intelligence and automatic control technology, and particularly relates to an industrial bubble detection method, system, device and medium based on a light-weight detection model. BACKGROUND

[0002] Gas-liquid two-phase flow is a common and complex physical phenomenon in engineering fields, widely existing in multiple key industries such as chemical engineering, petroleum engineering, nuclear energy, and environmental engineering. Its core feature is the interaction and dynamic change of gas and liquid phases in space and time dimensions. Research and control of this phenomenon are of great significance for optimizing industrial processes and improving system efficiency.

[0003] In experimental research of multiphase flow, high-precision measurement techniques can be used to capture the spatiotemporal changes of interface shapes between different phases. These measurement techniques include high-speed photography, laser-induced fluorescence, X-ray imaging, and particle image velocimetry, etc. These techniques can provide detailed interface information, helping researchers analyze and understand the complex transport phenomena in multiphase flow.

[0004] Especially in dealing with gas-liquid two-phase (bubble) flow, accurate measurement of the spatiotemporal changes of interface shapes is crucial for analyzing the transfer phenomena between phases. The shape and motion of bubbles during flow affect the flow characteristics of fluids, heat and mass transfer efficiency, etc. Therefore, precise interface measurement not only helps to reveal the basic physical mechanisms, but also provides experimental data for the establishment and verification of multiphase flow models.

[0005] In addition, the spatiotemporal changes of interface shapes have practical significance for engineering applications. For example, in chemical processes, bubble behavior affects the efficiency of reactors and the quality of products; in the field of nuclear energy, gas-liquid two-phase flow affects the safety and reliability of cooling systems; in environmental engineering, bubble flow affects the performance of water treatment equipment. Through the combination of experiments and numerical simulations, researchers can better understand and control these key processes, thereby optimizing the design and operation of engineering systems.

[0006] Although existing research has made significant progress in the basic theory and experimental methods of gas-liquid two-phase flow, there are still many challenges, especially in the measurement and analysis of interface dynamic changes. Traditional measurement methods often require a lot of manual intervention, and data acquisition and analysis are time-consuming, and it is difficult to achieve real-time detection and statistics under complex flow conditions. This limitation hinders the promotion and in-depth development of gas-liquid two-phase flow research in more extensive application scenarios. SUMMARY

[0007] In view of the problems in the prior art, the present application provides an industrial bubble detection method, system, device and medium based on a lightweight detection model, so as to solve the key problems of low bubble recognition accuracy, insufficient calculation efficiency, complex deployment and poor system real-time performance in the gas-liquid two-phase flow process, especially under actual working conditions such as complex background, target adhesion, small bubble density and high video image interference, and the existing detection method cannot balance the detection effect and engineering practicability.

[0008] The present application is realized by the following technical solutions: An industrial bubble detection method based on a lightweight detection model, comprising: Obtaining gas-liquid two-phase flow working condition video data containing bubbles, after frame extraction processing of the video data, obtaining original image frames; After bubble target labeling of the original image frames, obtaining a bubble recognition data set; Based on YOLOv5, YOLOv8 and YOLOv11s, a bubble detection model evolving from generation to generation is constructed, the model is trained using the bubble recognition data set, and a trained lightweight bubble detection model is obtained; Based on the trained lightweight bubble detection model, real-time gas-liquid two-phase flow working condition video data collected in an industrial field is detected for bubbles, and a frame dropping strategy and a frame rate scheduling mechanism are executed, and the detection inference result is output.

[0009] Preferably, after frame extraction processing of the video data, the original image frames are obtained, specifically: The gas-liquid two-phase flow working condition video data is extracted frame by frame at a fixed frame rate using a frame extraction algorithm, and low-quality frames with serious blur and occlusion are removed by combining an artificial screening mechanism; The original frame image after frame extraction processing is subjected to size cropping and local magnification operation, and the image input size is unified to meet the data format requirements of model training, and standardized original image frames are obtained; Preferably, after bubble target labeling of the original image frames, the bubble recognition data set is obtained, specifically: The bubbles appearing in the original image frames are finely labeled frame by frame, and the labeling content includes the boundary box position and size information of each bubble target; After labeling is completed, the image is subjected to various forms of data enhancement operation to obtain the bubble recognition data set; The various forms of data enhancement operation include one or more of horizontal flipping, random cropping, brightness disturbance, contrast adjustment, rotation transformation and image scaling.

[0010] Preferably, the bubble detection model evolving from generation to generation is constructed based on YOLOv5, YOLOv8 and YOLOv11s, specifically: Based on YOLOv5, a first generation bubble detection model is established, and a bubble recognition dataset is used for training; In the trained first generation bubble detection model, YOLOv8 is introduced as the model backbone structure to build the second generation bubble detection model, so as to optimize the loss function and network parameters, and the bubble recognition dataset is used for training; In the trained second generation bubble detection model, YOLOv11s is introduced as the model basic structure to build the third generation bubble detection model, and pruning and quantization compression are implemented on the third generation bubble detection model to form a lightweight model, and the bubble recognition dataset is used for training to obtain a trained lightweight bubble detection model; The trained lightweight bubble detection model is exported as an ONNX format and further converted into a TensorRT engine, while enabling an FP16 precision acceleration mode to be deployed on a GPU platform; Among them, the third generation bubble detection model is pruned and quantized to form a lightweight model, specifically: The third generation bubble detection model adopts a lightweight deep convolutional backbone network, and a multi-scale feature fusion strategy is introduced in the Neck structure. The pruning and quantization strategies are applied in the structure of the third generation bubble detection model to reduce redundant calculation and network redundancy channels while keeping the model inference speed and memory occupation controllable.

[0011] Preferably, the frame discarding strategy and frame rate scheduling mechanism are as follows: The trained lightweight bubble detection model is used to monitor the processing frame rate of real-time gas-liquid two-phase flow working condition video data in an industrial field in real time; The processing frame rate is compared with a preset performance baseline; When the processing frame rate is lower than the performance baseline, a frame discarding strategy is started to balance the system processing load by periodically skipping part of the input frames to maintain a stable output frame rate; The frame discarding strategy discards M frames for every N frames processed, where N and M are positive integers and N>M.

[0012] Preferably, the inference result is structured and packaged as a JSON format output, including the number of bubbles, bounding box information and discrimination attributes of each frame, and can be transmitted to an upper computer or PLC through a ModBus / TCP protocol to realize data connection with an industrial control system.

[0013] An industrial bubble recognition and processing system based on a lightweight detection model, comprising: An image acquisition module for acquiring gas-liquid two-phase flow working condition video data containing bubbles; The data preprocessing and labeling module is configured to extract frames from the video data to obtain original image frames, and label the original image frames to obtain a bubble recognition dataset; The model training module is configured to construct a bubble detection model evolving from YOLOv5, YOLOv8 and YOLOv11s, train and optimize the model based on the bubble recognition dataset, and obtain a trained lightweight bubble detection model. The model detection and output module is configured to deploy the trained lightweight bubble detection model to an industrial site, detect bubbles in real-time gas-liquid two-phase flow working condition video data, perform a frame dropping strategy and a frame rate scheduling mechanism, and output a detection inference result.

[0014] Preferably, the model detection and output module comprises an inference acceleration module, a frame scheduling module and a backend integration module. The inference acceleration module is configured to export an ONNX format and further convert it into a TensorRT engine, and enable an FP16 precision acceleration mode to be deployed on a GPU platform. The frame scheduling module is configured to perform a frame dropping strategy and a frame rate scheduling mechanism to ensure the frame processing rate during continuous system operation. The backend integration module comprises a multi-threaded inference processing framework for managing video acquisition, model inference and result rendering processes.

[0015] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the industrial bubble detection method based on the lightweight detection model when executing the computer program.

[0016] A computer readable storage medium stores a computer program, and the computer program implements the steps of the industrial bubble detection method based on the lightweight detection model when executed by a processor.

[0017] Compared with the prior art, the present application has the following beneficial technical effects: The application provides an industrial bubble detection method based on a lightweight detection model, proposes an efficient bubble recognition and statistics method for complex industrial scenes, realizes full-process closed-loop design from data construction, model optimization to system deployment, and significantly improves recognition accuracy, running speed and deployment stability, and has good engineering adaptability and expansion capability. First, a complete data preparation and labeling system is constructed, including extraction, format conversion and cleaning of original video images, design of standardized manual labeling process, and introduction of model pre-labeling and manual correction combination mechanism to improve large batch image labeling efficiency. On this basis, combined with image symmetry transformation, contrast disturbance, cutting and reorganization and other enhancement strategies, the training set sample coverage range is expanded, and the robustness and generalization ability of the model are significantly enhanced. In terms of model design, the application adopts a generation-by-generation evolution strategy and constructs three generations of detection models: the first generation is based on YOLOv5, and a preliminary available bubble detection system is established; the second generation introduces a YOLOv8 backbone structure, optimizes the loss function and network parameters, and the detection accuracy is improved to 80%; the third generation adopts YOLOv11s as the basic structure, and prunes and quantizes the model to form a lightweight model, which effectively reduces the model size and computational complexity while improving the detection accuracy to more than 94%, meeting the deployment level application requirements. Thus, the key problems such as low bubble recognition accuracy, insufficient computing efficiency, complex deployment and poor system real-time performance in the gas-liquid two-phase flow process are solved, especially in the actual working conditions such as complex background, target adhesion, small bubble density and high interference of video images, the existing detection methods cannot balance the detection effect and engineering practicability Further, the application is suitable for various gas-liquid two-phase flow monitoring scenes, especially in the chemical industry, energy industry, material synthesis and environmental engineering industry, and real-time monitoring and statistical analysis of bubble generation behavior can be realized. The system can not only be used for process visualization and safety control, but also has technical potential for other visual task scene expansion such as small target detection and continuous state recognition, and has wide industrial application value and engineering transformation prospect. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0019] Figure 1 A flowchart of an industrial bubble detection method based on a lightweight detection model of the application; Figure 2 A schematic diagram of bubble data labeled by a single person in an embodiment; Figure 3 A diagram of manually annotated bubble data for the embodiment; Figure 4 A training result diagram of the first generation YOLOv5 model for the embodiment, with an accuracy of 66.7%, a recall rate of 73.4%, an mAP50 of 0.656, and an mAP50-95 of 0.238; Figure 5 A training result diagram of the second generation YOLOv8 model for the embodiment, with an accuracy of 68.2%, a recall rate of 70.6%, an mAP50 of 0.650, and an mAP50-95 of 0.239; Figure 6 A processing process diagram of the new data set for the embodiment, which first samples the original video frames, then crops and scales the sampling results, pre-labels using the YOLOv8 model, and finally manually corrects dirty data and packs it into a YOLO data set; Figure 7 A training result diagram of the third generation YOLOv11s model for the embodiment, with an accuracy of 93.7%, a recall rate of 93.1%, an mAP50 of 0.980, and an mAP50-95 of 0.851; Figure 8 A single image test result diagram of the third generation YOLOv11s model for the embodiment; Figure 9 A comparison diagram of the original framework and the inference delay after using the TensorRT FP16 framework for the embodiment, with an acceleration ratio of 1.83 times; Figure 10 A frame skipping strategy execution diagram for the embodiment; Figure 11 A system real-time running FPS change curve for the embodiment; Figure 12 An FPS change curve after starting the frame scheduling mechanism for the embodiment; Figure 13 A result diagram of the original data processed using several data enhancement techniques for the embodiment. DETAILED DESCRIPTION

[0020] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0021] The present application is aimed at developing an automatic detection and statistical system for bubble quantity and density by means of computer vision technology, to solve the bottleneck problem of traditional experimental techniques in efficiency and accuracy. Through this system, dynamic data of bubble behavior can be obtained in real time, providing strong support for in-depth research on the interface phenomenon in gas-liquid two-phase flow. The core task of the present application is to build an automatic index system to serve the automatic execution of industrial production processes, thereby improving production efficiency, ensuring system safety, and promoting engineering optimization and technological innovation in related fields.

[0022] An industrial bubble detection method based on a lightweight detection model, as shown in Figure 1 , includes: Obtain gas-liquid two-phase flow condition video data containing bubbles, and obtain original image frames after frame extraction processing of the video data; specifically: Frame extraction algorithm is used to extract frames of gas-liquid two-phase flow condition video data at a fixed frame rate, and low-quality frames with severe blur and occlusion are removed by combining artificial screening mechanism; Perform size cropping and local magnification operations on the original frame images after frame extraction processing, and unify the image input size to meet the data format requirements of model training, to obtain standardized original image frames; After labeling the original image frames, a bubble recognition dataset is obtained; specifically: Each bubble target is finely labeled frame by frame, and the labeling content includes the boundary box position and size information of each bubble target; After labeling is completed, perform multiple forms of data enhancement operations on the image to obtain the bubble recognition dataset; The multiple forms of data enhancement operations include one or more of horizontal flipping, random cropping, brightness disturbance, contrast adjustment, rotation transformation and image scaling.

[0023] Based on YOLOv5, YOLOv8 and YOLOv11s, a bubble detection model evolving from generation to generation is constructed, and a trained lightweight bubble detection model is obtained by training the model using a bubble recognition dataset; Specifically: Based on YOLOv5, a first generation bubble detection model is established, and the bubble recognition dataset is used for training; YOLOv8 is introduced as the model backbone structure in the trained first generation bubble detection model to construct a second generation bubble detection model, optimize the loss function and network parameters, and use the bubble recognition dataset for training; YOLOv11s is introduced as the model base structure in the trained second generation bubble detection model to construct a third generation bubble detection model, and pruning and quantization compression are performed on the third generation bubble detection model to form a lightweight model, and the bubble recognition dataset is used for training to obtain a trained lightweight bubble detection model; The trained lightweight bubble detection model is exported as an ONNX format and further converted into a TensorRT engine, while enabling an FP16 precision acceleration mode to be deployed on a GPU platform; Among them, the third generation bubble detection model is pruned and quantized and compressed to form a lightweight model, specifically: The third generation bubble detection model adopts a lightweight deep convolutional backbone network, and a multi-scale feature fusion strategy is introduced in the Neck structure. The pruning and quantization strategies are applied in the structure of the third generation bubble detection model to reduce redundant calculation and network redundancy channels while keeping the model inference speed and memory occupation controllable.

[0024] Based on the trained lightweight bubble detection model, real-time gas-liquid two-phase flow condition video data collected in an industrial site is detected for bubbles, and a frame dropping strategy and a frame rate scheduling mechanism are executed, and the detection inference result is output.

[0025] The frame dropping strategy and the frame rate scheduling mechanism are as follows: The trained lightweight bubble detection model is monitored in real time for processing frame rate of real-time gas-liquid two-phase flow condition video data in an industrial site; The processing frame rate is compared with a preset performance baseline; When the processing frame rate is lower than the performance baseline, a frame dropping strategy is started to balance the processing load by periodically skipping some input frames to maintain a stable output frame rate; The frame dropping strategy is to drop M frames for every N frames processed, where N and M are positive integers and N>M.

[0026] The inference result is output in JSON format through structured packaging, including the number of bubbles, bounding box information and discrimination attributes of each frame, and can be transmitted to the host computer or PLC through the ModBus / TCP protocol to realize data docking with the industrial control system.

[0027] The output result is in JSON format, including the number of bubbles, position, size, category and recognition confidence in each frame of image, and can be sent to PLC or other control devices at a stable frequency of 1Hz, and the total response delay of the system is not more than 60ms An industrial bubble recognition and processing system based on a lightweight detection model, for implementing the detection method, comprising: An image acquisition module for acquiring gas-liquid two-phase flow working condition video data containing bubbles; A data preprocessing and labeling module for extracting frames from the video data to obtain original image frames, and obtaining a bubble recognition dataset after labeling the original image frames as bubble targets; A model training module for constructing bubble detection models evolving from generation to generation based on YOLOv5, YOLOv8 and YOLOv11s, training and optimizing the models using the bubble recognition dataset, and obtaining trained lightweight bubble detection models; A model detection and output module for deploying the trained lightweight bubble detection models to an industrial site, performing bubble detection on real-time gas-liquid two-phase flow working condition video data, executing frame dropping strategy and frame rate scheduling mechanism, and outputting detection inference results.

[0028] The model detection and output module includes an inference acceleration module, a frame scheduling module and a backend integration module; The inference acceleration module is used to export in ONNX format and further convert to TensorRT engine, while enabling FP16 precision acceleration mode to be deployed on GPU platform; The inference acceleration module has an average inference delay of less than 13ms under the TensorRT deployment environment, which is at least 1.8 times faster than the original PyTorch deployment.

[0029] The frame scheduling module is used to execute frame dropping strategy and frame rate scheduling mechanism to ensure the frame processing rate during continuous system operation; based on frame number, periodic frame skipping is performed, or whether to execute frame dropping strategy is dynamically judged according to processing load.

[0030] The backend integration module includes a multi-threaded inference processing framework and an industrial communication interface, which supports sending recognition results to the control system through the ModBus / TCP protocol to complete the closed-loop operation from visual recognition to control response.

[0031] The multi-thread reasoning framework is divided into an image acquisition thread, a reasoning thread and a result rendering thread, and the three are asynchronously run to improve system throughput.

[0032] The technical solutions of the present application will be described below clearly and completely. Obviously, the described embodiments are part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor belong to the scope of protection of the present application.

[0033] Embodiment 1 In the specific implementation process of the present application, the collection and preprocessing of the original data are first completed to construct the first generation of the basic data set for the bubble target detection model training.

[0034] Specifically, a high-resolution video data with a duration of about 3 minutes is collected from an industrial production site, which covers various bubble behavior states such as micro boiling and vigorous boiling in the typical process operation. In order to facilitate subsequent image recognition model processing, the frame extraction algorithm is used to extract the video frame by frame at a fixed frame rate, and the low-quality frames with serious blur and occlusion are removed by using the artificial screening mechanism. Finally, 342 clear and effective original image frames are obtained. Subsequently, the original frame image is subjected to size cropping and local magnification operation, and the image input size is unified to meet the data format requirements of the model training and improve the prominence of small bubble targets in the image.

[0035] After obtaining the standardized images, the bubbles appearing in the 342 images are finely labeled frame by frame in an artificial manner. The labeling content includes the position and size information of the boundary box of each bubble target, such as Figure 2 In order to improve the scale and diversity of the data set, after the artificial labeling is completed, further data enhancement operations are performed on the images in various forms, including but not limited to horizontal flip, random cropping, brightness disturbance, contrast adjustment, rotation transformation and image scaling, etc. so as to effectively expand the data space and improve the generalization ability of the model under complex working conditions. After the enhancement processing, a total of 682 training images are finally generated, such as Figure 3 , which constitutes the basic data set of the first generation of bubble recognition model training of the present application.

[0036] The data set constructed in this stage has typical industrial scene characteristics, covering different bubble density, size, distribution and background lighting conditions, which lays a solid data foundation for the effective training and performance evaluation of the subsequent model.

[0037] After the first generation of bubble recognition data set is constructed, the present application selects YOLOv5 as the initial detection model to perform target detection training experiment on the labeled image data.

[0038] First, the 682 training images are divided into training and validation sets in an 8:2 ratio and standardized. The model input size is set to 640x640, the default mosaic data augmentation and anchor automatic clustering mechanism are used, the SGD optimizer and cosine learning rate scheduling strategy are used, the batch size is set to 16, and the total number of training rounds is 100 rounds.

[0039] During training, the cross-entropy loss function and the CIoU target box regression loss are used for joint optimization, and the precision, recall, average precision, and other key performance indicators during each training round are recorded.

[0040] After training, the final test results on the validation set are as follows: mAP50: 0.656 mAP50-95: 0.238 Precision: 0.667 Recall: 0.734 The index change trend during training is shown in Figure 4 After the 10th round, the model converges and stabilizes, with high precision and recall, but the mAP50-95 index on the fine scale is significantly lower, reflecting the model's detection ability in small size bubbles, multiple target dense areas, and edge blurred targets.

[0041] The comprehensive evaluation shows that the first generation YOLOv5 model can complete the basic bubble detection task, but the detection accuracy, boundary box regression accuracy, and robustness in occlusion and blurred scenes still do not meet the requirements of engineering applications. Therefore, subsequent improvements need to be made through more powerful feature expression ability of the model structure and more complex data enhancement strategies.

[0042] Based on the training and test results of the first generation YOLOv5 model, to further improve the system's detection ability for complex bubble scenes, the present invention introduces a new generation of detection framework YOLOv8 and conducts a second stage of model training experiments on the same data set.

[0043] YOLOv8, as an important update version of the YOLO series, introduces more flexible network modules in feature extraction and decoding structure, with stronger representation ability. The model training still uses the aforementioned 682 data sets, the data division, preprocessing, and enhancement strategy remain the same, and the training parameter settings use the default recommended configuration, with an input image size of 640x640, an optimizer of AdamW, an initial learning rate of 0.01, and a training round of 100 rounds.

[0044] On the validation set, the test performance of the second generation model YOLOv8 is as follows: mAP50: 0.650 mAP50-95: 0.239 Precision: 0.682 Recall: 0.706 As Figure 5 shown, the model converges rapidly in the early stage of training, and the precision and recall are relatively stable, but compared with the first generation YOLOv5 model, only the accuracy and recall are slightly improved, and the mAP50-95 is almost unchanged, and the overall improvement is limited. This shows that in the current small-scale data set, although the YOLOv8 model has a more advanced structure, it does not fully exert its performance potential in the small target dense detection scene.

[0045] Combined with the experimental results, the present application judges that the current model performance improvement is restricted by the training data size and quality, especially in the detection effect of multiple bubble overlapping, boundary blur and complex shape area is still not ideal. Therefore, in the subsequent stage, a larger scale of data set is introduced, the pre-labeling combined with manual correction method is used to expand the sample quantity, and the improved structure YOLOv11 series model is introduced, and the pruning and quantization and other structure optimization means are used, so as to realize the significant breakthrough of detection performance.

[0046] On the basis of the limited performance improvement of the second generation model (YOLOv8), the present application further constructs the third generation detection model YOLOv11s, and systematically improves from the aspects of data expansion, model optimization and engineering deployment, significantly enhances the recognition ability and inference efficiency of complex bubble images.

[0047] In order to improve the scale and diversity of training data, the present application introduces a pre-labeling combined with manual correction mechanism on the basis of the original data construction process. Specifically, first, frame sampling and image cropping are performed on the newly collected video segments, and then the existing YOLOv8 model is called to automatically predict and label, as Figure 6 , a preliminary label file is formed; then, the pre-labeling results are manually fine-tuned frame by frame to ensure that the boundary box position is consistent with the bubble target. After the correction is completed, the images and label files are organized in YOLO format, and packaged into a standard training set structure through the PyCharm platform, and finally an enhanced data set with larger scale and more accurate labels is formed, providing data support for subsequent high-precision model training.

[0048] In terms of model structure, the third generation model is based on YOLOv11s, adopts a lightweight deep convolutional backbone network, and introduces a multi-scale feature fusion module in the Neck structure to improve small target detection performance. To further improve the efficiency of engineering deployment, pruning and quantization strategies are applied in the model structure, effectively reducing redundant calculations and network redundancy channels while keeping the model inference speed and memory usage controllable, thereby improving model compression ratio.

[0049] During training, the model input size is set to 640x640, the training round is 100 rounds, the optimizer uses AdamW, and improved Label Smoothing and Mosaic data augmentation methods are used. The performance indicators on the validation set are as follows: mAP50: 0.980 mAP50-95: 0.851 Precision: 0.937 Recall: 0.931 As Figure 7 shown, the model significantly outperforms the previous two generations in all performance indicators, with mAP50 improving to 98.0% and mAP50-95 reaching 85.1%, indicating excellent positioning accuracy performance at different IoU thresholds and significantly improved detection stability. The results of single-image detection are shown in Figure 8 .

[0050] In summary, the third generation YOLOv11s model improves the system's performance in bubble image detection tasks through structural optimization, data augmentation, and training strategy improvement, and has advantages such as high precision, low latency, and strong deployability, providing a key foundation for subsequent engineering integration and industrial application.

[0051] To ensure that the proposed bubble detection system has real-time and stability in actual industrial scenarios, the invention not only realizes the integration and deployment of the model and video stream, but also designs a complete set of frame discarding strategies and frame rate scheduling mechanisms to optimize the recognition system comprehensively.

[0052] (1) Model deployment and video stream integration This system uses Hiwin robot industrial cameras for image acquisition, receives video stream frame data in real time through SDK interfaces, and inputs them into the YOLOv11s model for online inference. The backend processing module is based on a multi-threaded inference processing framework, which manages video acquisition, model inference, and result rendering processes respectively, supports stable operation under high-resolution images (1080p), and has good image throughput capacity.

[0053] To reduce deployment delay and improve frame processing speed, the YOLOv11s model is exported through ONNX and converted to TensorRT format, enabled with FP16 precision acceleration mode, and deployed on a GPU platform. The inference time test results show that, under the same hardware conditions, the average inference delay of the TensorRT version is 12.3ms, which is 1.83 times faster than the original PyTorch version (22.5ms), effectively improving the number of frames that can be processed per unit time, as shown in Figure 9 ; Under the condition of high frame rate continuous video input, the model has short-term processing pressure fluctuations. To prevent delay accumulation caused by inference accumulation, the present application designs a frame discarding strategy, which periodically skips some frames based on frame number, a typical mode is shown in Figure 10 , discarding one frame for every two processed frames to stabilize the load and achieve dynamic balance of inference load.

[0054] In addition, the present application also introduces a frame rate scheduling mechanism: real-time monitoring of system processing frame rate (FPS) changes and comparison with the set performance baseline, when the inference frame rate is lower than the baseline, the frame discarding module is dynamically enabled and the resource scheduling is adapted. Experimental results show that, without starting the control mechanism, the system FPS decreases rapidly with running time, dropping to below 8fps, as shown in Figure 11 ; while starting the scheduling, the system frame rate is always maintained in the interval of 16fps-17.5fps, as shown in Figure 12 , effectively eliminating system performance fluctuations and ensuring real-time identification.

[0055] (3) Data augmentation and robustness improvement To improve the adaptability of the system to complex working conditions, the present application constructs a multi-type enhanced sample library, including grayscale, cropped, adjusted, grid, rotation, scaling and other styles, as shown in Figure 13 , which is used to assist the training model in generalizing performance under different interference conditions. Experiments show that these sample enhancement strategies significantly improve the model's ability to recognize fuzzy boundaries, adhering bubbles, low-contrast backgrounds, etc.

[0056] (4) Interaction adaptation with control system The inference results are output in JSON format through structured packaging, including the number of bubbles, boundary box information and discrimination attributes of each frame, and support transmission to the host computer or PLC through the ModBus / TCP protocol, realizing data docking with the industrial control system. The system supports one information update per second, with a response delay of no more than 60ms, ensuring the closed-loop operation capability of visual recognition → control response.

[0057] In summary, the present application constructs a stable, efficient and deployable bubble recognition backend processing system through video stream access, multi-thread asynchronous processing, model inference acceleration, frame rate dynamic regulation, data enhancement and control protocol adaptation, and provides comprehensive and reliable technical support for actual industrial deployment.

[0058] The present application has the following remarkable beneficial effects: (1) Through systematic data preparation and model optimization, the bubble recognition accuracy is significantly improved from 70% to more than 94%; (2) Through pruning, quantization and inference acceleration optimization, the model inference efficiency is effectively improved, and the edge deployment and GPU efficient computing scenarios are adapted; (3) Combined with multi-thread processing, frame discard scheduling and physical property constraints, real-time and efficient processing under industrial video stream is realized; (4) The robustness of the system to different working conditions, light, liquid color change and other interference factors is enhanced, and the detection stability is guaranteed; (5) The system structure is general and has strong portability, and can be seamlessly integrated into existing industrial control systems.

[0059] In another embodiment of the present application, a computer device is provided, which includes a processor and a memory, the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, and are suitable for implementing one or more instructions, and are specifically suitable for loading and executing one or more instructions to realize corresponding method processes or corresponding functions; the processor of the embodiment of the present application can be used for the operation of the industrial bubble detection method based on the lightweight detection model.

[0060] In still another embodiment of the present application, the present application also provides a storage medium, specifically a computer readable storage medium (Memory), which is a memory device in a computer device, used for storing programs and data. It can be understood that the computer readable storage medium herein can include an internal storage medium in the computer device, and of course can also include an extended storage medium supported by the computer device. The computer readable storage medium provides a storage space, which stores an operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and the instructions can be one or more computer programs (including program codes). It should be noted that the computer readable storage medium herein can be a high-speed RAM memory, or a non-volatile memory such as at least one disk memory. One or more instructions stored in the computer readable storage medium can be loaded and executed by the processor to implement the corresponding steps of the industrial bubble detection method based on the lightweight detection model in the above embodiment.

[0061] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. In addition, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code.

[0062] The present application is described with reference to flowcharts and / or block diagrams according to the methods, devices (systems), and computer program products of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 2 The functions specified in one or more flows and / or blocks Figure 2 The means for performing the functions specified in one or more flows and / or blocks.

[0063] These computer program instructions can also be stored in a computer readable memory capable of directing the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including instruction means, which implements the functions specified in the flowcharts and / or block diagrams. Figure 2 The functions specified in one or more flows and / or blocksFigure 2 The function specified in one or more boxes.

[0064] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 2 One or more processes and / or boxes Figure 2 The steps of the function specified in one or more boxes.

[0065] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

[0066] Furthermore, the terms include and have, and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or apparatus.

[0067] It should be understood that, when used in this specification and the appended claims, terms include and encompass the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0068] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The terminology used herein includes, and / or encompasses, any and all combinations of one or more of the associated listed items.

[0069] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Those skilled in the art can readily implement the present invention based on the accompanying drawings and the above description. However, any modifications, alterations, or variations made by those skilled in the art without departing from the scope of the present invention, utilizing the disclosed technical content, are equivalent embodiments of the present invention. Furthermore, any modifications, alterations, or variations made to the above embodiments based on the essential technology of the present invention are still within the protection scope of the present invention.

Claims

1. An industrial bubble detection method based on a lightweight detection model, characterized in that, include: Acquire video data of gas-liquid two-phase flow containing bubbles, and obtain the original image frames after frame extraction processing of the video data; After labeling the original image frames with bubble targets, a bubble recognition dataset is obtained; A progressively evolving bubble detection model is built based on YOLOv5, YOLOv8, and YOLOv11s. The model is trained using a bubble recognition dataset to obtain a trained lightweight bubble detection model. Based on a trained lightweight bubble detection model, bubble detection is performed on real-time gas-liquid two-phase flow video data collected in industrial sites. A frame discarding strategy and frame rate scheduling mechanism are executed, and the detection inference results are output.

2. The industrial bubble detection method based on a lightweight detection model according to claim 1, characterized in that, After performing frame extraction processing on the video data, the original image frames are obtained, specifically: A frame extraction algorithm is used to extract the video data of gas-liquid two-phase flow at a fixed frame rate, and a manual screening mechanism is used to remove low-quality frames that are blurry or severely occluded. The original frame images after frame extraction are cropped and partially enlarged to unify the image input size and meet the data format requirements for model training, thus obtaining standardized original image frames.

3. The industrial bubble detection method based on a lightweight detection model according to claim 1, characterized in that, After labeling the original image frames with bubbles, a bubble recognition dataset is obtained, specifically: The bubbles appearing in the original image frames are finely annotated frame by frame, and the annotation content includes the bounding box position and size information of each bubble target; After annotation, various forms of data augmentation operations are performed on the images to obtain a bubble recognition dataset; The various forms of data augmentation operations include one or more of the following: horizontal flipping, random cropping, brightness perturbation, contrast adjustment, rotation transformation, and image scaling.

4. The industrial bubble detection method based on a lightweight detection model according to claim 1, characterized in that, A progressively evolving bubble detection model is constructed based on YOLOv5, YOLOv8, and YOLOv11s, specifically as follows: A first-generation bubble detection model was built based on YOLOv5 and trained using a bubble recognition dataset. YOLOv8 was introduced as the backbone structure in the first-generation bubble detection model after training to build a second-generation bubble detection model to optimize the loss function and network parameters, and the model was trained using a bubble recognition dataset. YOLOv11s was introduced as the basic structure of the trained second-generation bubble detection model to build a third-generation bubble detection model. The third-generation bubble detection model was then pruned and quantized to form a lightweight model. The model was trained using a bubble recognition dataset to obtain a trained lightweight bubble detection model. The trained lightweight bubble detection model is exported to ONNX format and further converted to the TensorRT engine, while FP16 precision acceleration mode is enabled for deployment on the GPU platform. Specifically, the third-generation bubble detection model was pruned and quantized to create a lightweight model, as follows: A lightweight deep convolutional backbone network is adopted for the third-generation bubble detection model, and a multi-scale feature fusion strategy is introduced into the Neck structure. Pruning and quantization strategies are applied to the structure of the third-generation bubble detection model to reduce redundant computation and network redundant channels while keeping the model inference speed and memory usage controllable.

5. The industrial bubble detection method based on a lightweight detection model according to claim 1, characterized in that, The frame dropping strategy and frame rate scheduling mechanism are as follows: Real-time monitoring of the frame rate of the trained lightweight bubble detection model in processing real-time gas-liquid two-phase flow video data in industrial settings. The processing frame rate is compared with a preset performance baseline; When the processing frame rate is lower than the performance baseline, a frame dropping strategy is initiated to balance the system processing load by periodically skipping some input frames in order to maintain a stable output frame rate. The frame discarding strategy is to discard M frames for every N frames processed, where N and M are positive integers and N > M.

6. The industrial bubble detection method based on a lightweight detection model according to claim 1, characterized in that, The inference results are structured and encapsulated into JSON format, which includes the number of bubbles, bounding box information and discrimination attributes for each frame. It can be transmitted to a host computer or PLC via ModBus / TCP protocol to achieve data interface with industrial control systems.

7. An industrial bubble identification and processing system based on a lightweight detection model, used to implement the detection method as described in any one of claims 1-6, characterized in that, include: The image acquisition module is used to acquire video data of gas-liquid two-phase flow conditions containing bubbles; The data preprocessing and annotation module is used to extract frames from the video data to obtain original image frames, and to annotate the original image frames with bubble targets to obtain a bubble recognition dataset. The model training module is used to build progressively evolving bubble detection models based on YOLOv5, YOLOv8 and YOLOv11s, and to train and optimize the models using bubble recognition datasets to obtain a trained lightweight bubble detection model. The model detection and output module is used to deploy the trained lightweight bubble detection model to the industrial site, perform bubble detection on real-time gas-liquid two-phase flow video data, execute frame discarding strategy and frame rate scheduling mechanism, and output the detection inference results.

8. The industrial bubble recognition and processing system based on a lightweight detection model according to claim 7, characterized in that, The model detection and output module includes an inference acceleration module, a frame scheduling module, and a backend integration module. The inference acceleration module is used to execute the export to ONNX format and further convert it to the TensorRT engine, while enabling FP16 precision acceleration mode for deployment on the GPU platform. The frame scheduling module is used to execute the frame discarding strategy and frame rate scheduling mechanism to ensure the frame processing rate during continuous system operation. The backend integration module includes a multi-threaded inference processing framework, which manages the video acquisition, model inference, and result rendering processes respectively.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the industrial bubble detection method based on a lightweight detection model as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the industrial bubble detection method based on the lightweight detection model as described in any one of claims 1 to 7.

Images