Bubble size monitoring method and device
By using the SCR-DET model to detect the monitoring video, the problem of high workload and insufficient accuracy of the manual monitoring method of bubble size in the prior art is solved, and automatic bubble monitoring and identification is realized, reducing costs and improving monitoring accuracy.
Patent Information
- Application Number
- CN202110260386.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-03-10
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2041-03-10
AI Technical Summary
The existing manual monitoring methods for bubble size are too high in workload and cost, and the monitoring accuracy is insufficient.
The SCR-DET model including feature extraction network, feature fusion network and multi-dimensional attention network is used to detect the target object of the monitoring video, obtain the position and size information of the bubbles, and realize automatic monitoring and identification of bubbles.
It reduces the monitoring workload and cost, improves bubble monitoring accuracy, and realizes automatic bubble monitoring and identification.
Smart Images

Figure CN112967335B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of intelligent monitoring technology, and in particular to a bubble size monitoring method and device. Background Art
[0002] With the development and progress of society, the requirements and expectations of safety work in all walks of life have reached unprecedented heights. In the beneficiation process of platinum mines, it is necessary to monitor the sorting situation in real time, specifically to identify the size of bubbles and to warn of emergencies. The common method at present is for staff to observe the monitoring video and judge frame by frame, or for staff to directly judge at the production site, and take effective emergency measures when abnormal bubbles occur.
[0003] However, the existing manual monitoring methods for bubble size are too labor-intensive and costly, and the monitoring accuracy is insufficient. Summary of the invention
[0004] The invention solves the problems of excessively high workload and cost and insufficient monitoring accuracy in the existing manual bubble size monitoring method.
[0005] To solve the above problems, the present invention provides a bubble size monitoring method, which includes: obtaining a monitoring video of the equipment to be inspected; performing target object detection on the image frame of the monitoring video according to a pre-trained SCR-DET model to obtain a bubble detection result of the image frame; the SCR-DET model includes a feature extraction network, a feature fusion network and a multi-dimensional attention network connected in sequence; and performing bubble size monitoring according to the bubble detection result.
[0006] Optionally, the target object detection is performed on the image frame of the surveillance video according to the pre-trained SCR-DET model to obtain a bubble detection result of the image frame, including: the feature extraction network extracts a first feature map of the image frame of the surveillance video; the feature fusion network performs feature fusion and fine sampling on the first feature map to obtain a second feature map; the channel attention network and the pixel attention network of the multidimensional attention network respectively extract the second feature map to obtain a third feature map, and multiply the third feature map with the second feature map to obtain a new feature map; and predicts the position information and size information of the bubbles in the image frame according to the new feature map.
[0007] Optionally, the feature fusion network performs feature fusion and fine sampling on the first feature map to obtain a second feature map, including: the feature fusion network upsamples the high-level feature map extracted by the feature extraction network to obtain a fourth feature map with a preset step size; the feature fusion network upsamples the low-level feature map extracted by the feature extraction network to obtain a fifth feature map with the preset step size, and deeply analyzes the fifth feature map to obtain a sixth feature map; the feature fusion network adds the fourth feature map and the sixth feature map element by element to obtain the second feature map.
[0008] Optionally, the method further includes: using multiple step sizes to upsample the sample set and identify the target, and obtaining the detection accuracy and network training time corresponding to each step size; and determining the preset step size based on the detection accuracy and network training time corresponding to each step size.
[0009] Optionally, the feature extraction network extracts a first feature map of the image frame of the surveillance video, including: the feature extraction network extracts a first feature map corresponding to a preset area in the image frame of the surveillance video, and the preset area is determined by a pre-set target recognition area parameter.
[0010] Optionally, the monitoring of bubble size according to the bubble detection result includes: determining the length and width of a detection frame corresponding to the bubble according to the bubble detection result; and issuing a bubble size abnormality alarm if the length and / or the width is greater than a bubble size threshold.
[0011] Optionally, the method further includes: marking a detection frame corresponding to the bubble in the image frame according to the bubble detection result; and outputting the image frame marked with the detection frame to a display device.
[0012] The present invention provides a bubble size monitoring device, which comprises: an acquisition module, used for acquiring a monitoring video of a device to be inspected; a bubble detection module, used for performing target object detection on an image frame of the monitoring video according to a pre-trained SCR-DET model to obtain a bubble detection result of the image frame; the SCR-DET model comprises a feature extraction network, a feature fusion network and a multi-dimensional attention network connected in sequence; and a size monitoring module, used for monitoring the bubble size according to the bubble detection result.
[0013] Optionally, the bubble detection module is specifically used to: the feature extraction network extracts a first feature map of the image frame of the surveillance video; the feature fusion network performs feature fusion and fine sampling on the first feature map to obtain a second feature map; the channel attention network and the pixel attention network of the multidimensional attention network respectively extract the second feature map to obtain a third feature map, and multiply the third feature map with the second feature map to obtain a new feature map; and predict the position information and size information of the bubbles in the image frame according to the new feature map.
[0014] Optionally, the bubble detection module is specifically used to: the feature fusion network upsamples the high-level feature map extracted by the feature extraction network to obtain a fourth feature map of a preset step size; the feature fusion network upsamples the low-level feature map extracted by the feature extraction network to obtain a fifth feature map of the preset step size, and deeply analyzes the fifth feature map to obtain a sixth feature map; the feature fusion network adds the fourth feature map and the sixth feature map element by element to obtain a second feature map.
[0015] The bubble size monitoring method and device provided by the present invention realize bubble detection based on a feature extraction network, a feature fusion network and a multi-dimensional attention network SCR-DET model connected in sequence. Compared with the existing target detection algorithm applied to bubble detection, the bubble size and position can be determined more accurately, and automatic bubble monitoring and identification can be realized, thereby reducing the monitoring workload and cost and improving the bubble monitoring accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.
[0017] Figure 1 is a schematic flow chart of a bubble size monitoring method in an embodiment of the present invention;
[0018] Figure 2 This is a schematic diagram of the structure of the SCR-DET model in an embodiment of the present invention;
[0019] Figure 3 is a schematic flow chart of a method for monitoring bubble size of gold mining equipment in an embodiment of the present invention;
[0020] Figure 4 This is the normal output result of the visualized bubble size in the embodiment of the present invention;
[0021] Figure 5 The output result of the abnormal bubble size visualized in the embodiment of the present invention;
[0022] Figure 6 Schematic diagram of the structure of a bubble size monitoring device in an embodiment of the present invention.
[0023] Description of reference numerals:
[0024] 601-acquisition module; 602-bubble detection module; 603-size monitoring module. DETAILED DESCRIPTION
[0025] In order to make the above-mentioned objects, features and advantages of the present invention more clearly understood, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0026] The existing manual monitoring method of bubble size is labor-intensive and costly; the remote monitoring video method is not timely enough, and it is difficult to arrange personnel, and the detection information cannot be communicated in a timely manner. The bubble size monitoring method provided in the embodiment of the present invention replaces manual bubble size monitoring. Taking the flotation equipment of platinum mine as an example, remote online identification is realized, which greatly improves the monitoring speed and accuracy while reducing labor, changes the traditional manual identification method, and realizes intelligent monitoring.
[0027] The bubble size monitoring method provided in an embodiment of the present invention is implemented based on the SCR-DET (Towards More Robust Detection for Small, Cluttered and Rotated Objects) neural network model. It uses the scene video collected by the camera in real time, and synchronously transmits the collected video stream to the analysis server. The analysis server identifies the bubble targets that need to be tracked on site to determine whether an early warning is needed.
[0028] The above method provided by the embodiment of the present invention is different from other methods for bubble size target detection in that the SCR-DET neural network model is used as the detection model for target detection. Since the size and characteristics of bubble objects are different from those of ordinary objects, and there are many of them in a single picture and the background is noisy, the SCR-DET neural network model was originally used for remote sensing detection and has a good solution for situations such as noisy background and a large number of objects. It is improved in the present invention and used in the bubble size monitoring scenario.
[0029] Figure 1 is a schematic flow chart of a bubble size monitoring method in one embodiment of the present invention, the method comprising:
[0030] S102, obtaining monitoring video of the equipment to be inspected.
[0031] In this embodiment, the flotation equipment of a gold mine is taken as an example. A high-definition camera is installed above the flotation equipment to clearly capture the entire inspection area of the flotation equipment and upload the video stream to the server. The detection model in the server performs the recognition function of the size and position of the bubbles.
[0032] S104, performing target object detection on the image frame of the surveillance video according to the pre-trained SCR-DET model to obtain a bubble detection result of the image frame.
[0033] The SCR-DET model includes a feature extraction network, a feature fusion network, and a multi-dimensional attention network connected in sequence. The feature extraction network can use a ResNet (Residual Network) backbone network to extract feature maps; the feature fusion network can use a SF-Net (Finer Sampling and Feature Fusion Network) to perform multi-layer feature fusion and effective anchor sampling to solve the problem of insufficient target information and insufficient anchor samples; the multi-dimensional attention network (Multi-Dimensional Attention Network, MDA-Net) uses a combination of spatial attention and channel attention to enhance target cues and weaken non-target information, and uses an attention mechanism to solve the problems of occlusion, noise, and blur.
[0034] This embodiment can use the above-mentioned pre-trained SCR-DET model to execute the target detection method and perform frame-by-frame detection on the surveillance video.
[0035] S106, monitoring the bubble size according to the bubble detection result.
[0036] After obtaining the bubble detection result of the above image frame, the length and width of the bubble corresponding detection frame can be determined according to the bubble detection result. If the length and / or width is greater than the bubble size threshold, an abnormal bubble size alarm is issued. The bubble size threshold can be determined according to the flotation requirements of the flotation equipment for mineral particles. When the bubble adhesion reaches a certain degree, resulting in the bubble size being greater than the above bubble size threshold, an abnormal alarm is issued and the staff is reminded to check.
[0037] The bubble size monitoring method of the embodiment of the present invention realizes bubble detection based on a feature extraction network, a feature fusion network and a multi-dimensional attention network SCR-DET model connected in sequence. Compared with the existing target detection algorithm used for bubble detection, it can more accurately determine the bubble size and position, realize automatic bubble monitoring and identification, reduce monitoring workload and cost, and improve bubble monitoring accuracy.
[0038] The target object detection process of the SCR-DET model is as follows:
[0039] First, the feature extraction network extracts the first feature map of the image frame of the surveillance video.
[0040] See also Figure 2 The structural diagram of the SCR-DET model shown in FIG. 1 shows a feature extraction network, which includes three convolutional layers C2, C3, and C4, which can respectively extract feature maps of different sizes, and gradually convolute the features in the image frame to expand the receptive field that can be fused with features.
[0041] Secondly, the feature fusion network performs feature fusion and fine sampling on the first feature map to obtain the second feature map.
[0042] like Figure 2 As shown, the SCR-DET model also includes a feature fusion network SF-NET, which is connected to the C3 and C4 convolutional layers of the feature extraction network respectively, and the feature maps output by C3 and C4 are feature fused in SF-NET to obtain a new feature map F3. Specifically, the following steps may be included: the feature fusion network upsamples the high-level feature map extracted by the feature extraction network to obtain a fourth feature map with a preset step size; the feature fusion network upsamples the low-level feature map extracted by the feature extraction network to obtain a fifth feature map with a preset step size, and the fifth feature map is deeply analyzed (Inception Module) to obtain a sixth feature map; the feature fusion network adds the fourth feature map and the sixth feature map element by element to obtain the second feature map.
[0043] There are usually two obstacles to detecting small objects: insufficient object feature information and insufficient anchors. Due to the use of pooling layers, small objects lose most of their feature information in deep networks. Feature fusion and effective sampling are the keys to better detect small objects. Specifically:
[0044] (1) Feature fusion. It is generally believed that low-level feature maps can retain the location information of small objects, while high-level feature maps can contain higher-level semantic features. The top-down information flow of Feature Pyramid Networks (FPN) is a common feature fusion method, which involves the combination of high-level and low-level feature maps of different features.
[0045] (2) Fine sampling. Insufficient and unbalanced training samples will affect detection performance. The anchor placement method directly affects the positive sample sampling rate. The classic anchor placement method is related to the resolution of the feature map, that is, the step size of the anchor placement. As the network deepens, the feature map resolution decreases, and the anchor step size increases, which often leads to the loss of sampling of small targets. A smaller step size can sample more high-quality samples, thereby capturing small objects well, which is helpful for both model training and reasoning.
[0046] In the anchor-based detection framework, the anchor step size is equal to the feature scaling factor relative to the original image. The anchor step size can only be an exponential multiple of 2. SF-Net solves this problem by changing the size of the feature map, making the setting of the anchor step size more flexible to allow more adaptive sampling. In order to reduce network parameters, SF-Net only uses C3 and C4 in feature extraction networks (such as Resnet) for fusion to balance semantic information and location information while ignoring other less relevant features.
[0047] The first pipeline of SF-Net upsamples C4 so that the scaling factor Sa of the feature map relative to the original image frame is equal to the desired anchor step size; the second pipeline also upsamples C3 to the same size to increase semantic information. The structure of the deep parsing module (Inception Module) contains convolution kernels of various sizes to capture the diversity of target shapes. Finally, the new feature map is obtained by adding the two channels element by element.
[0048] Optionally, by using multiple step sizes to upsample and identify the sample set, the detection accuracy and network training time corresponding to each step size can be obtained; then the preset step size can be determined based on the detection accuracy and network training time corresponding to each step size. The above preset step size is determined by comprehensively considering the high detection accuracy and the short network training time. Therefore, SF-Net can fuse C3 and C4 feature maps to enrich feature information; increase the feature map size (reduce Sa) to increase the number of anchors.
[0049] Then, the channel attention network and the pixel attention network of the multidimensional attention network respectively extract the second feature map to obtain the third feature map, and multiply the third feature map with the second feature map to obtain a new feature map.
[0050] like Figure 2As shown in the figure, the multi-dimensional attention network MDA-NET includes two attention mechanisms: Pixel Attention Network, and Channel Attention Network on the lower side. Due to the complexity of the captured image, the proposed region generated by the Region Proposal Network (RPN) may introduce a lot of noise information. Too much noise may confuse the object information, and the boundaries between objects will become blurred, resulting in missed detection and increased false alarms. In order to more effectively capture the bubble features under complex backgrounds, a supervised multi-dimensional attention network is used to reduce the impact of the background.
[0051] MDA-NET consists of two attention mechanisms: the upper part is the pixel attention network, and the lower part is the channel attention network. The second feature map mentioned above is used to highlight the main target and weaken the non-subject in both the channel attention network and the pixel attention network to obtain a new feature map.
[0052] Pixel attention network, the second feature map is passed through the deep analysis module using different convolution kernels to extract features of different scales, and then the convolution operation is performed to obtain a dual-channel saliency map (foreground and background), and the target area is binarized to obtain a binary map containing the target area (areas with high saliency are assigned a value of 1, and other unimportant areas are assigned a value of 0, turning the image into a pixel image composed of 0\1). The binary map is subjected to logistic regression (softmax) and the value is limited to [0, 1].
[0053] The channel attention network uses global average pooling (GAP) to output the numerical distribution of C feature channels, and then reduces the dimension of the feature to 1 / r of the input, passes through the activation function ReLu (Rectified Linear Units, linear rectification function) and then passes through a fully connected layer (FC, Fully Connected layer) to convert the dimension to the original dimension. Then the activation function sigmoid is used to obtain the normalized weight between [0, 1].
[0054] Multiply the above three parts to get a new feature map.
[0055] Finally, the location and size information of the bubbles are predicted based on the new feature map.
[0056] It should be noted that this embodiment has made modifications to the SCR-DET model, canceled the original rotating frame detection mechanism, and directly predicted based on the features output by MDA-Net, and finally output the detected bubble size and position information.
[0057] Figure 3 FIG. 1 is a schematic flow chart of a method for monitoring bubble size of gold mining equipment in one embodiment of the present invention. The process of monitoring bubble size of gold mining equipment is as follows:
[0058] S301, a high-definition camera shoots the inspection area and uploads the video stream to the analysis server.
[0059] Install a high-definition camera above the equipment to be inspected at the production site to clearly capture the entire inspection area and upload it to the analysis server.
[0060] S302, bubble identification is performed using the SCR-DET model.
[0061] A supervised learning target object labeling algorithm is used to perform real-time recognition and detection of all bubbles on the screen. Since there is a certain gap between the features of the objects and bubbles used by ordinary target detection algorithms, this embodiment uses a special neural network model to improve the detection accuracy, namely the SCR-DET network model, which outputs the detected bubble size and position.
[0062] Among them, the working principle of the target object labeling algorithm of supervised learning is as follows:
[0063] According to the video of the bubble flow on the conveyor belt shot by the high-definition camera, image samples of different sizes and positions are produced; the bubbles in the image samples are marked and produced as training samples; the target detection model is trained using the training samples so that the weight data meets expectations; the trained target detection model can be used to identify the bubbles contained in the video shot by the camera in real time in real time. Preferably, in this embodiment, the target detection model detects and alarms the bubbles in the monitoring video frame by frame.
[0064] S303, obtaining the bubble position and size.
[0065] S304, determine whether the size of the bubble exceeds a threshold. If yes, execute S305; if no, end.
[0066] S305, executing the bubble size abnormality alarm.
[0067] S306: Output an image frame marked with a visual bubble detection frame.
[0068] Specifically, a detection frame corresponding to the bubble may be marked in the image frame according to the bubble detection result, and then the image frame marked with the detection frame may be output to a display device for display.
[0069] Figure 4 Normal output results for visualizing bubble size, Figure 5The output result of the abnormal bubble size is visualized. When the bubbles in the image are adhered and the neural network recognition detection box is too large, it is determined that there is a bubble adhesion phenomenon, and the size abnormality is determined and an alarm is issued. To improve the recognition efficiency, the feature extraction network can only extract the first feature map corresponding to the preset area in the image frame of the monitoring video, and then monitor the bubble size based on the first feature map corresponding to the preset area. The preset area is determined by the preset target recognition area parameters.
[0070] Figure 4 and Figure 5 The two solid lines in can be realized by pre-setting parameters to identify bubbles in the area between the two lines.
[0071] Figure 6 : is a schematic structural diagram of a bubble size monitoring device in one embodiment of the present invention, the device comprising:
[0072] An acquisition module 601 is used to acquire monitoring video of the equipment to be inspected;
[0073] The bubble detection module 602 is used to perform target object detection on the image frame of the surveillance video according to the pre-trained SCR-DET model to obtain the bubble detection result of the image frame; the SCR-DET model includes a feature extraction network, a feature fusion network and a multi-dimensional attention network connected in sequence;
[0074] The size monitoring module 603 is used to monitor the bubble size according to the bubble detection result.
[0075] The bubble size monitoring device of the embodiment of the present invention realizes bubble detection based on a feature extraction network, a feature fusion network and a multi-dimensional attention network SCR-DET model connected in sequence. Compared with the existing target detection algorithm used for bubble detection, it can more accurately determine the bubble size and position, realize automatic monitoring and identification of bubbles, reduce monitoring workload and cost, and improve bubble monitoring accuracy.
[0076] Optionally, as an embodiment, the bubble detection module is specifically used for: the feature extraction network extracts a first feature map of the image frame of the surveillance video; the feature fusion network performs feature fusion and fine sampling on the first feature map to obtain a second feature map; the channel attention network and the pixel attention network of the multidimensional attention network respectively extract the second feature map to obtain a third feature map, and multiply the third feature map with the second feature map to obtain a new feature map; and predict the position information and size information of the bubbles in the image frame according to the new feature map.
[0077] Optionally, as an embodiment, the bubble detection module is specifically used to: the feature fusion network upsamples the high-level feature map extracted by the feature extraction network to obtain a fourth feature map of a preset step size; the feature fusion network upsamples the low-level feature map extracted by the feature extraction network to obtain a fifth feature map of the preset step size, and deeply analyzes the fifth feature map to obtain a sixth feature map; the feature fusion network adds the fourth feature map and the sixth feature map element by element to obtain a second feature map.
[0078] Optionally, as an embodiment, the device also includes a step size setting module, which is used to: use multiple step sizes to upsample the sample set and identify the target, and obtain the detection accuracy and network training time corresponding to each step size; determine the preset step size based on the detection accuracy and network training time corresponding to each step size.
[0079] Optionally, as an embodiment, the bubble detection module is specifically used for: the feature extraction network extracts a first feature map corresponding to a preset area in an image frame of the surveillance video, and the preset area is determined by a preset target recognition area parameter.
[0080] Optionally, as an embodiment, the size monitoring module is specifically used to: determine the length and width of the detection frame corresponding to the bubble according to the bubble detection result; if the length and / or the width is greater than the bubble size threshold, then issue a bubble size abnormality alarm.
[0081] Optionally, as an embodiment, the device further includes a detection frame output module, which is used to: mark a detection frame corresponding to the bubble in the image frame according to the bubble detection result; and output the image frame marked with the detection frame to a display device.
[0082] The bubble size monitoring device provided in the above embodiment can implement each process in the embodiment of the above bubble size monitoring method, and will not be described again here to avoid repetition.
[0083] The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above-mentioned bubble size monitoring method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0084] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the control device through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it may include the processes of the above-mentioned method embodiments, wherein the storage medium may be a memory, a disk, an optical disk, etc.
[0085] In this article, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0086] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.
[0087] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for monitoring bubble size, characterized in that: The method comprises: Obtain surveillance video of the equipment to be inspected; Performing target object detection on the image frame of the surveillance video according to a pre-trained SCR-DET model to obtain a bubble detection result of the image frame; the SCR-DET model includes a feature extraction network, a feature fusion network, and a multi-dimensional attention network connected in sequence; monitoring the bubble size according to the bubble detection result; The method performs target object detection on the image frame of the surveillance video according to the pre-trained SCR-DET model to obtain a bubble detection result of the image frame, including: extracting a first feature map of the image frame of the surveillance video by the feature extraction network; performing feature fusion and fine sampling on the first feature map by the feature fusion network to obtain a second feature map; extracting a third feature map from the second feature map by the channel attention network and the pixel attention network of the multidimensional attention network, respectively, and multiplying the third feature map by the second feature map to obtain a new feature map; and predicting the position information and size information of the bubbles in the image frame according to the new feature map. The feature fusion network performs feature fusion and fine sampling on the first feature map to obtain a second feature map, including: the feature fusion network upsampling the high-level feature map extracted by the feature extraction network to obtain a fourth feature map of a preset step size; the feature fusion network upsampling the low-level feature map extracted by the feature extraction network to obtain a fifth feature map of the preset step size, and performing depth analysis on the fifth feature map to obtain a sixth feature map; the feature fusion network adds the fourth feature map and the sixth feature map element by element to obtain a second feature map; The method further includes: using multiple step sizes to upsample the sample set and identify the target, obtaining the detection accuracy and network training time corresponding to each step size; and determining the preset step size based on the detection accuracy and network training time corresponding to each step size; The SCR-DET model eliminates the rotation box detection mechanism and directly makes predictions based on the features output by the multi-dimensional attention network.
2. The method according to claim 1, characterized in that The feature extraction network extracts a first feature map of the image frame of the surveillance video, including: The feature extraction network extracts a first feature map corresponding to a preset area in an image frame of the surveillance video, where the preset area is determined by a preset target recognition area parameter.
3. The method according to claim 1, characterized in that The bubble size monitoring according to the bubble detection result includes: Determine the length and width of the detection frame corresponding to the bubble according to the bubble detection result; If the length and / or the width is greater than the bubble size threshold, an abnormal bubble size alarm is issued.
4. The method according to claim 1, characterized in that The method further comprises: Marking a detection frame corresponding to the bubble in the image frame according to the bubble detection result; The image frame marked with the detection frame is output to a display device.
5. A bubble size monitoring device, characterized in that: The device comprises: The acquisition module is used to obtain the monitoring video of the equipment to be inspected; A bubble detection module is configured to perform target object detection on the image frames of the surveillance video according to a pre-trained SCR-DET model to obtain bubble detection results for the image frames; the SCR-DET model includes a feature extraction network, a feature fusion network, and a multi-dimensional attention network connected in sequence; A size monitoring module, configured to monitor the bubble size according to the bubble detection result; The bubble detection module is specifically configured to: extract a first feature map of the image frame of the surveillance video by the feature extraction network; perform feature fusion and fine sampling on the first feature map by the feature fusion network to obtain a second feature map; extract a third feature map from the second feature map by the channel attention network and the pixel attention network of the multidimensional attention network, respectively; and multiply the third feature map by the second feature map to obtain a new feature map; and predict the position and size information of the bubbles in the image frame based on the new feature map. The bubble detection module is specifically configured to: upsample the high-level feature map extracted by the feature extraction network by the feature fusion network to obtain a fourth feature map of a preset step size; upsample the low-level feature map extracted by the feature extraction network by the feature fusion network to obtain a fifth feature map of the preset step size, and perform depth analysis on the fifth feature map to obtain a sixth feature map; and add the fourth feature map and the sixth feature map element by element by the feature fusion network to obtain a second feature map; The device also includes a step size setting module for: using multiple step sizes to upsample the sample set and identify the target, obtaining the detection accuracy and network training time corresponding to each step size; and determining the preset step size based on the detection accuracy and network training time corresponding to each step size; The SCR-DET model eliminates the rotation box detection mechanism and directly makes predictions based on the features output by the multi-dimensional attention network.
