A video target detection method and system suitable for industrial fluid level tracking

Abstract

The application discloses a video target detection method and system suitable for industrial fluid level tracking. The method comprises the following steps: acquiring mixed data and a real data set; preprocessing the data set; constructing an initial neural network combined by a ResNet50 convolutional neural network and a Transformer attention mechanism; modifying the query part of the network, using a query based on object categories and a query based on previous frame position prediction; introducing a long-term feature memory module into the network; training using the preprocessed real data set and mixed data set; identifying video fluid targets using a video target detection model; and introducing a slow exit mechanism, which exits when linearly predicting that the fluid exits the picture or when a long time has passed without detection. The method is suitable for various industrial fluid target detection scenarios, has fast model convergence speed and high detection precision, and the slow exit mechanism can compensate for missed detection caused by single data.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of video target recognition in computer vision, and particularly relates to a video target detection method and system suitable for industrial fluid level tracking. BACKGROUND

[0002] In industrial production processes, liquid level monitoring is an important basic task, and accurate tracking and real-time monitoring of fluid levels are of great significance, especially in pipeline transportation and tank management. Traditional liquid level detection methods usually rely on sensors such as float level meters, capacitive level meters, and ultrasonic level meters. Although these methods can provide relatively accurate measurement data in static environments, in dynamic industrial scenarios such as rapid fluid flow, pipeline vibration, light interference, or other complex environments, the accuracy and stability of the sensors are easily limited.

[0003] In recent years, with the development of computer vision technology, video target detection technology has been gradually applied to the field of industrial fluid level tracking, achieving non-contact liquid level tracking through image processing and target detection algorithms. However, fluid monitoring in industrial scenarios faces many challenges, including:

[0004] Environmental complexity: Industrial sites may have complex light conditions, reflection interference, and equipment obstruction, affecting the accuracy of video target detection.

[0005] Data scarcity: It is difficult to obtain labeled data in industrial fluid level tracking scenarios, and insufficient data leads to poor generalization ability of the trained detection model.

[0006] Insufficient modeling of time series information: The dynamic changes and continuity of fluids require video target detection models to fully utilize time series information, while existing methods usually only focus on single-frame detection, lacking effective use of context information, leading to instability of detection results.

[0007] Missed detection problem: Due to the complexity of the industrial scene, the target detection model may miss detection, causing discontinuity or detection failure in liquid level tracking, thereby affecting the stability and reliability of the system. SUMMARY

[0008] To solve the above problems, a video target detection method and system suitable for industrial fluid level tracking are proposed.

[0009] The purpose of the present application is achieved by the following technical scheme: a video target detection method suitable for industrial fluid level tracking, comprising:

[0010] S1, acquiring a data set and performing labeling and preprocessing;

[0011] S2, construct an industrial fluid level target detection network, including a feature extraction network, a long-term feature memory module and a target detection network, and use a Kalman filter for next frame prediction after target detection, wherein the query in the target detection network is based on an object category query and a query based on a Kalman filter obtained front frame position prediction;

[0012] S3, use the preprocessed real data set and the mixed data set to train the network for multiple rounds to obtain a trained industrial fluid level target detection network;

[0013] S4, use the trained target detection model and Kalman filter to perform target detection and next frame prediction on the pipeline and liquid part in the video; wherein a slow exit mechanism is introduced for a target that does not appear, and the target is exited when linear prediction fluid exits the screen or is not detected for a long time.

[0014] Further, the data set includes real data acquisition and pseudo data acquisition, key frames of real data are extracted, pseudo data is randomly inserted into the real data set by means of frame interpolation, and a mixed data set is generated;

[0015] The real data is video data of part of the time period at part of the detection points in the industrial field, which is obtained by labeling the collected video, the pseudo data is generated by network search or computer simulation, and the computer simulation generates simulation pictures consistent with the real pipeline diameter and fluid flow characteristics.

[0016] Further, the data preprocessing includes: performing size normalization operation on all data, and using basic data enhancement operation for different image data in the same video sequence.

[0017] Further, the feature extraction network is specifically: using the residual module in ResNet50 network as a basic module, initializing using the pre-training weight on Imagenet, and using a skip connection to solve the gradient disappearance problem in deep neural network training.

[0018] Further, the long-term feature memory module is composed of feature maps of backbones of previous T frames of pictures, and is a fixed first-in-first-out queue structure, which saves feature maps of previous T frames of pictures. In long-term feature processing, all data and the current feature map are subjected to global pooling processing, dimension reduction and maximum pooling, and the similarity of several frames of feature maps with the current feature is selected by similarity measurement, and the corresponding features are aggregated to the current feature map.

[0019] Further, the feature aggregation comprises: maximum pooling of stacked features, fusion of features of the current frame and historical frames using a cross-attention mechanism, and then fusion of the current feature map, the maximum-pooled stacked feature map and the attention-set feature map to obtain final aggregated features.

[0020] Further, an encoder module using a Transformer in the target detection network globally models the fused feature map, and a query part is obtained by fusing an object query based on an object category and a query based on a previous frame position prediction; the object query based on the object category represents a detection target of a corresponding category, and in an initialization process, an embedding corresponding category label value is used as a query value for initialization.

[0021] The query based on the previous frame position prediction is specifically: a corresponding probability map is generated according to two corner points of a target box in a Kalman filter prediction result in a previous frame, and a prediction query is obtained through an encoder;

[0022] The fusion formula comprises: Q final =β·Q cls +(1-β)·Q pred

[0023] wherein Q final is a final fused query, Q cls is the object query based on the object category, Q pred is the query based on the previous frame position prediction, and β is a weight coefficient.

[0024] Further, a slow exit mechanism is introduced for a target that does not appear, and the target exits the picture when linear prediction fluid or is not detected for a long time; the specific steps are as follows:

[0025] For an undetected object, its corresponding coordinate information and motion information are found through a Kalman filter, and time series prediction is performed according to the coordinate information and the motion information; if the target area is close to 0 or the predicted target position is close to the edge of the picture, it can be confirmed that the object is not detected; otherwise, it is considered to be occluded or missed in a time sequence with a length of t, and the parameters in the filter are used as the motion standard; when t time elapses, it is considered to disappear if the object is still not detected.

[0026] On the other hand, the specification of the present application also provides a video target detection system suitable for industrial fluid level tracking, comprising:

[0027] A data processing module is configured to acquire real data and mixed data, and mark and preprocess the data;

[0028] The model construction module constructs a backbone through a partial Resnet50 convolutional neural network, constructs a target recognition core module using an encoder-decoder module of a Transformer self-attention mechanism structure, uses a query based on object categories and front frame position prediction instead of the original position-based query, and introduces a long-term feature memory module in the network to provide more abundant context information for target detection.

[0029] The model training module initializes and sets corresponding parameters of the model and trains the model.

[0030] The post-processing module performs post-processing on the case where the object is not detected through a slow exit mechanism.

[0031] On the other hand, the specification of the present application also provides a video target detection device suitable for industrial fluid level tracking, comprising a memory and one or more processors, wherein the memory stores executable code, and the processor executes the executable code to realize the video target detection method suitable for industrial fluid level tracking.

[0032] The present application has the following advantages: the present application is innovative in that it designs a query mechanism based on object categories and front frame position prediction in combination with a ResNet50 convolutional neural network and a Transformer attention mechanism for the needs of industrial scenarios, combines a long-term feature memory module to detect and track fluids, and significantly improves the accuracy and stability of detection. In addition, by introducing a slow exit mechanism, the interference caused by missed detection and false detection on liquid level tracking can be effectively alleviated, and the robustness of the system in complex scenarios is ensured. At the same time, through the targeted solution to the problem of data scarcity and the problem of missed detection in complex scenarios, the present application shows its wide application potential and practical value.

[0033] In the field of industrial fluid level tracking, the present application has significant technical advantages and economic benefits compared with traditional sensor-based monitoring technology, and provides a new solution for liquid level monitoring in complex dynamic scenarios, and has high application prospect and industrialization potential. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 The video target detection network model structure diagram provided for the embodiment of the present application;

[0035] Figure 2 The detailed structure diagram of the long-term feature memory module provided for the embodiment of the present application;

[0036] Figure 3 The slow exit mechanism introduced for the embodiment of the present application is shown in the figure;

[0037] Figure 4A video target detection device suitable for industrial fluid level tracking provided by an embodiment of the present application is shown in the schematic diagram. DETAILED DESCRIPTION

[0038] The specific embodiments of the present application are further described in detail below with reference to the accompanying drawings.

[0039] The specific embodiments of the present application are further described in detail below with reference to the accompanying drawings. Figures 1-3 The target detection method and system provided by the present application for the industrial fluid level tracking scene are described below.

[0040] EMBODIMENT

[0041] S1, data set construction and labeling, the data set includes real data acquisition and pseudo data. The real data is part of the video data of the detection point position at part of the time period obtained in the industrial field. The image data of the liquid flowing in the industrial pipeline is collected by deploying a camera, and the collected video is labeled to form a basic real data set. At the same time, in order to solve the problem of sample scarcity in the industrial data scene, a pseudo data set is generated.

[0042] The construction method of the pseudo data set includes the following two kinds: network search: using network search to obtain related pictures of pipes and fluids from publicly disclosed images in the field of industrial fluids, such as pipe fluid pictures obtained by web crawler and fluid pictures of the same pipe diameter obtained by computer simulation. Computer simulation generation: using computer simulation technology to generate simulation pictures consistent with the real pipe diameter and fluid flow characteristics to supplement data diversity. In order to enhance the robustness and accuracy of detection, the generated pseudo data is not directly used for training, but the key frames of the real data are extracted in the real data, and the pseudo data is randomly inserted into the real data set by frame interpolation to generate a mixed data set. Key frame extraction can optimize the redundant frames in the video, shorten the video length by using key frame extraction technology, and retain the key action frames to reduce the waste of computing resources, especially the invalid inference training caused by the lack of changes in short time video data in the industrial scene.

[0043] The data is preprocessed, and a size normalization operation is performed to adjust the resolution of all data to a uniform size of 1024x1024 to ensure the consistency of the input data during model training. Data augmentation is used on the data set to expand the generality of the data. For different image data within the same video sequence, basic data augmentation operations are adopted, such as color transformation (such as brightness and contrast adjustment), color noise injection, image horizontal flipping, color jittering, etc. Through the above data enhancement operations, small range noise is injected to improve the robustness of the model to different environments. For different video sequences, more powerful data enhancement methods can be adopted.

[0044] S2, construct an industrial fluid level target detection network, the method uses part of the ResNet50 network as a feature extraction and down-sampling network, initializes it using the pre-trained weights on Imagenet, identifies the target through the encoder and decoder of the transformer architecture, and uses a Kalman filter for next frame prediction after target detection. The main use of the residual module in the ResNet50 network as a basic module, using skip connection to solve the gradient disappearance problem in the training of deep neural network, the specific method is:

[0045] S21. Select part of ResNet50 as Backbone to extract features and perform down-sampling, use the weights of ResNet50 on Imagenet as pre-training parameters, input the picture to generate a feature map through backbone, and send the feature map processed by long-term feature to the Transformer structure encoder for encoding to further extract features;

[0046] S22. After obtaining the feature map through backbone, process it through the long-term feature memory module, which is composed of the feature maps of the previous T frames of pictures through backbone, and is a fixed first-in-first-out queue structure, which saves the feature maps of the previous T frames of pictures. In long-term feature processing, all data are globally pooled with the current feature map, and the similarity metric can select several frames of feature maps with the smallest similarity to the current feature map, and the corresponding features are aggregated to the current feature map through the feature aggregation module, to introduce more rich context information and timing information to the feature map;

[0047] S23. For the query part, a new query is composed of object class-based and previous frame position prediction-based queries. For the object class-based object query, it is defined as a set of learnable vectors related to the target class, and each object query represents a query for the corresponding class and position. The initial vector is obtained by random initialization + class label corresponding to the Embedding operation. For the query based on the prediction of the previous frame position, in the video sequence, it can be assumed that the movement of the target is linear between frames, and the Kalman filter and the Hungarian matching algorithm can provide the future possible position information for the object predicted in the previous frame. The Gaussian function can generate the corresponding heat map, and a simple encoder structure can generate the predicted query. The combination of the two queries generates the real query. In order to ensure the convergence of the training, the real label data of the previous frame with mixed Gaussian noise is added to the query based on the prediction of the previous frame position in the early training, that is, the result of the heat map of the prediction result and the mixed Gaussian noise heat map is used as the next frame prediction query.

[0048] S24. Generating a result map by the multi-layer decoder.

[0049] Preferably, the specific implementation process in the embodiment includes:

[0050] The long-term memory module is used for the feature map generated by the Resnet50 network to enhance the time sequence context information of target detection. The process is as follows:

[0051] For the long-term feature memory module, a time constant T is defined, representing the previous T frame pictures. First, the feature map extracted by the Backbone from the previous T frame pictures is stored in a fixed-length first-in-first-out (FIFO) queue:

[0052] M t =[F t-1 ,F t-2 ,…,F t-T ]

[0053] Where M t is the feature map set stored by the long-term memory module, and F t-i represents the feature map of the i-th frame.

[0054] As shown in Figure 2 , for the feature map in the queue, first, all the feature maps are adaptively pooled, then dimensionally reduced and maximally pooled. For the feature of the i-th frame,

[0055] g(F t-i )=F′ t-i =MaxPolling(Reduce dim(AdaptivePooling(F t-i ) i ))

[0056] After the above processing, F′ t-i changes from BXCXWXH to BXCX1X1, and through dimension reduction, it can become BXC.

[0057] The processed feature map is concatenated, the similarity between the current feature map and the historical feature map is calculated through similarity measurement, and the cosine function is used as the distance measurement function

[0058]

[0059] Where <·,·> represents the inner product of the feature vector, and ||·|| represents the L2 norm of the vector;

[0060] Through similarity measurement, the similarity between the current feature map and the historical feature map can be obtained, and the index of the 3 frames and the current feature map is found

[0061]

[0062] wherein is the processed feature vector of long-term feature memory

[0063] The three frames of feature maps with the largest gap from the current feature are obtained by similarity measurement, and the selected feature maps are stacked by concat

[0064] The historical features and current features are fused by feature aggregation to make up for the defects of the current feature map. Here, the feature aggregation is composed of parallel max-pooling structure, depth separable convolution and cross-attention mechanism and skip connection, etc., as shown in Figure 2

[0065] First, the stacked feature F stacked is max-pooled to extract significant features in different regions

[0066] F pool = MaxPooling(F stacked )

[0067] The depth separable convolution operation is performed on F stacked to make the number of channels correspond to the feature map. The cross-attention mechanism fuses the features of the current frame and the historical frame. Q, V are the features of the current frame, and K is the stacked feature F stacked of the historical frame. The output after the depth separable convolution, the attention aggregation formula is:

[0068]

[0069] The features output by the above modules are fused to obtain

[0070] F fused = F t +F atten +F pool

[0071] The encoder module of the Transformer is used to further globally model the fused feature map. Through the self-attention mechanism, the correlation information between distant pixels in the picture can be captured. Assuming that the feature map input is F fused , the self-attention calculation formula is as follows:

[0072]

[0073] where Q, K, V are query, key and value respectively, and d k is the scaling factor of the feature dimension.

[0074] ​In order to further improve the time sequence modeling capability of the target detection model in the industrial liquid level tracking, the query part is improved in the target detection network in the application, and through the object query based on the object category and the query based on the position prediction of the previous frame, the robustness and time sequence consistency of the model for target detection are significantly enhanced.

[0075] The object query Q based on the object category cls is a set of learnable vectors related to the target category, and each query represents the detected target of the corresponding category. In this way, the network can explicitly learn the specific representation of each category. Parameterizing the object query makes it converge to a suitable value during the training process, and the initialized query is composed of random initialization and embedding corresponding to the category label value. That is, the embedding corresponding to the category label value is used as the query value for initialization in the query part dimension.

[0076] In a video sequence, the motion of the target can be approximately assumed to be linear movement between frames. Therefore, the position of the target in the next frame can be predicted using the position of the target in the previous frame, and the corresponding query is generated. The Kalman filter is used in the application to estimate the motion state, including the following steps:

[0077] The working principle of the Kalman filter is as follows: including prediction and update,

[0078] Prediction stage

[0079]

[0080] wherein, is the predicted current state vector, F k is the transition matrix, which describes how the system evolves from time k-1 to time k. Through the above process, the predicted state at the current time can be obtained. P k|k-1 is the predicted error covariance matrix of the current state, which represents the uncertainty of the predicted value. P k|k-1 is the state error of the previous time, Q k is the process noise covariance, which represents the uncertainty in the system model. This formula updates the predicted error covariance, considering the uncertainty in state transition and the influence of process noise. In the application, the target position information of the next frame image can be obtained through the prediction of the previous frame image result, so as to feed back through the query.

[0081] Update:

[0082]

[0083] P k|k = (I - K k H k )P k|k-1

[0084] where K k is the Kalman gain, measuring the influence of the measurement on the state estimation. H k is the observation matrix, describing how the state is mapped to the measurement space. R k is the measurement noise covariance, representing the uncertainty of the measurement. z k is the measurement at the current time step, achieves the mapping of the predicted state to the measurement space. is the state vector after balancing the predicted value and the observed value. P k|k is the updated error covariance matrix. By updating the covariance, the uncertainty of the state is corrected by combining the measurement, which can make the covariance gradually converge.

[0085] In this invention, we define the state vector as follows:

[0086] For the detected object, the general output is the position coordinates and the corresponding width and height as well as the category, therefore, the position coordinates and the corresponding width and height of the target object are used as state variables:

[0087]

[0088] where x c (k), y c (k) are the current object center coordinates, w(k), h(k) are the corresponding object width and height, is the first derivative of the corresponding variable, i.e. the speed.

[0089] Similarly, the corresponding observation variable is defined as:

[0090]

[0091] The noise matrix Q k , R k is defined as follows

[0092]

[0093] where σ p , σ v , σ m are the corresponding process noise factors and measurement noise factors, which can be initialized according to the actual running situation, and the initialization is set as σ p = σ m , σ v = 0.1 σp More reasonable.

[0094] According to the definition of the above state vector and observation variable, the corresponding transition matrix F and observation matrix H are reasonably derived

[0095]

[0096] Through the above method, the Kalman filter model can be constructed. In the training process, the model finally outputs the position and category information of the current target. Through the Hungarian matching, the matching situation of the detected target result and the current prediction model can be obtained. For the target that cannot be matched, a corresponding Kalman filter can be reinitialized.

[0097] The next frame position of the detected target is predicted by the Kalman filter, and objects outside the picture are removed through the post-processing mechanism. The prediction result of the Kalman filter is converted into a probability map

[0098] The coordinate information in it can generate a probability map through a Gaussian kernel function, and the generation function is as follows:

[0099]

[0100] Where x c , y c is the center of the predicted target, is the variance of the Gaussian distribution, which is proportional to the width w and height h of the target box. G(x,y) is the probability value of each pixel position in the image space.

[0101] Similarly, the probability map corresponding to the upper left corner and the lower right corner of the target box can be generated through the above operation. At this time, σ 2 is determined by the aspect ratio and the area of the target box.

[0102] The prediction probability map can be generated by the above operation, and the corresponding prediction query Q pre d;

[0103] Finally, the fusion query is generated, and the fusion formula is:

[0104] Q final = β·Q cls +(1-β)·Q pred ,

[0105] Wherein, wherein, β is a weight coefficient, used to control the relative importance of the category query and the prediction query.

[0106] S3. The network is trained using the preprocessed real data set and the mixed data set, and the target detection network model is obtained through multiple rounds of training.

[0107] By Q final The multi-layer decoder operation is performed to obtain the final result, and iterative training is continuously performed; in the training process, in order to ensure the effectiveness of the query, the prediction probability map generated in the early stage is fused with the object position probability map of the previous frame after low-noise processing. When the training gradually converges, the object position true noise probability map of the previous frame is no longer added.

[0108] Specifically, continuous training is performed on a server meeting the hardware requirements, generally RTX4090 is used for training, the training round reaches 200 rounds, and every 40 rounds is verified on the verification set and the corresponding model weight file is saved. The initial learning rate is set to 3e-3, and the cosine annealing adjustment strategy is used to gradually reduce it, and the AdamW optimizer is used to accelerate convergence and prevent overfitting. After training, the model with better performance on the verification set is selected as the recognition model.

[0109] S4. By detecting the pipeline and fluid in the video sequence, the corresponding liquid level can be calculated and tracked. In the use of the model detection, a slow exit mechanism is introduced, which will be combined with Figure 3 The slow exit mechanism is explained

[0110] In long-time video sequence detection, noise disturbance may cause missed detection. Therefore, a Kalman filter is added in the detection process, and its specific working condition is as described above. For the undetected object, the next step of the target motion is predicted by the Kalman filter. When the area of the target is 0 or the target frame is close to the edge of the picture, it is exited. Otherwise, it is considered to be occluded or missed in the time sequence with a length of t, and the parameters in the filter are used as the motion standard, and the prediction is performed. When t time is over, it is considered to be disappeared.

[0111] According to the above network structure, the depth of the backbone and the time parameter T of the long-term feature memory module can be adjusted according to actual requirements to reduce the calculation amount and the parameter amount, and the lightweight of the network is realized.

[0112] Corresponding to the foregoing embodiment of the video target detection method suitable for industrial fluid level tracking, the application also provides an embodiment of a video target detection device suitable for industrial fluid level tracking.

[0113] Referring to Figure 4 , the video target detection device provided by the embodiment of the application suitable for industrial fluid level tracking comprises a memory and one or more processors, the memory stores executable code, and the processor executes the executable code to implement the video target detection method suitable for industrial fluid level tracking in the foregoing embodiment.

[0114] The embodiment of the video target detection device for industrial fluid level tracking provided by the present application can be applied to any data processing device, which can be a device or apparatus such as a computer. The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. Taking the software implementation as an example, as a logical device, it is formed by reading the corresponding computer program instructions in the non-volatile memory into the memory for running by the processor of the data processing device where the device is located. From the hardware level, as shown in Figure 4 Figure 4 In addition to the processor, the memory, the network interface, and the non-volatile memory shown in the figure, the data processing device where the device is located in the embodiment can also include other hardware according to the actual functions of the data processing device, and details are not described herein.

[0115] The implementation process of the functions and roles of each unit in the device is specifically described in the implementation process of the corresponding steps in the above method, and details are not described herein.

[0116] For the device embodiment, since it basically corresponds to the method embodiment, the related parts can be referred to the part of the method embodiment. The device embodiment described above is only schematic, and the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or distributed on multiple network units. According to actual needs, part or all of the modules can be selected to achieve the purpose of the present application. Those skilled in the art can understand and implement without creative labor.

[0117] The embodiment of the present application also provides a computer readable storage medium, which stores a program, and the program is executed by a processor to implement the video target detection method for industrial fluid level tracking in the above embodiment.

[0118] ​The computer readable storage medium can be an internal storage unit of any of the aforementioned data processing capable devices, such as a hard disk or a memory. The computer readable storage medium can also be an external storage device of any of the aforementioned data processing capable devices, such as a plug-in hard disk, a Smart Media Card (SMC), an SD card, a Flash Card, etc. Further, the computer readable storage medium can include both an internal storage unit and an external storage device of any of the aforementioned data processing capable devices. The computer readable storage medium is used to store the computer program and other programs and data required by the aforementioned data processing capable devices, and can also be used to temporarily store data that has been output or is about to be output.

[0119] The application further provides a computer program product comprising a computer program which, when executed by a processor, implements the video target detection method for industrial fluid level tracking.

[0120] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the application being indicated by the following claims.

[0121] It is to be understood that the above general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application. The application is not restricted to the precise construction and combinations of parts and steps described above and shown in the accompanying drawings, as such variations and modifications are intended to be included within the scope of the application. The scope of the application is limited only by the following claims.

Claims

1. A video target detection method suitable for industrial fluid level tracking, characterized in that, include: S1. Obtain the dataset and perform annotation and preprocessing; S2. Construct an industrial fluid level target detection network, including a feature extraction network, a long-term feature memory module, and a target detection network. After target detection, use a Kalman filter to predict the next frame. The query in the target detection network is a query based on the object category and a query based on the position prediction of the previous frame obtained by the Kalman filter. The object detection network uses the encoder module of Transformer to globally model the fused feature map. The query part is obtained by fusing object queries based on object categories and queries based on the position prediction of the previous frame. Among them, the object query based on the object category represents the detection target of the corresponding category. During the initialization process, the category label value corresponding to the embedding is used as the query value for initialization. The query based on the position prediction of the previous frame is specifically obtained by generating a probability map for the top left and bottom right corners of the target box based on the Kalman filter prediction results in the previous frame, and then obtaining the predicted query through an encoder. S3. The network is trained multiple times using preprocessed real datasets and mixed datasets to obtain the trained industrial fluid level target detection network. S4. Use the trained target detection model and Kalman filter to perform target detection and next frame prediction on the pipe and liquid parts in the video. A slow exit mechanism is introduced for targets that do not appear, which exits the screen when the linearly predicted fluid leaves the screen or is not detected for a long time.

2. The video target detection method for industrial fluid level tracking according to claim 1, characterized in that, The acquisition of the dataset includes: real data acquisition and pseudo data acquisition. Keyframes of the real data are extracted, and pseudo data is randomly inserted into the real dataset by frame interpolation to generate a hybrid dataset. The real data consists of video data from certain locations and time periods at some test sites in the industrial field, obtained by labeling the collected videos. The pseudo data is generated through network search or computer simulation. The computer simulation uses computer simulation technology to generate simulated images that are consistent with the actual pipe diameter and fluid flow characteristics.

3. The video target detection method for industrial fluid level tracking according to claim 1, characterized in that, The data preprocessing includes: performing size normalization on all data, and applying basic data augmentation operations on different image data within the same video sequence.

4. The video target detection method for industrial fluid level tracking according to claim 1, characterized in that, The feature extraction network specifically uses the residual module in the ResNet50 network as the basic module, initializes it using its pre-trained weights on ImageNet, and uses skip connections to solve the gradient vanishing problem in deep neural network training.

5. A video target detection method for industrial fluid level tracking according to claim 1, characterized in that, The long-term feature memory module consists of the feature maps of the backbone of the previous T frames of images, and is a fixed first-in-first-out queue structure that stores the feature maps of the previous T frames of images. In long-term feature processing, all data are subjected to global pooling, dimensionality reduction and max pooling with the current feature map. The feature maps with the least similarity to the current feature are selected by similarity measurement, and the corresponding features are aggregated into the current feature map.

6. A video target detection method for industrial fluid level tracking according to claim 5, characterized in that, The feature aggregation includes: performing max pooling on the stacked features, fusing the features of the current frame and historical frames using a cross attention mechanism, and then fusing the current feature map, the max-pooled stacked feature map, and the feature map after attention set to obtain the final aggregated features.

7. A video target detection method for industrial fluid level tracking according to claim 1, characterized in that, The fusion formula includes: ; in The final query obtained by merging This is an object query based on object category; For queries based on previous frame position prediction, These are the weighting coefficients.

8. A video target detection method for industrial fluid level tracking according to claim 1, characterized in that, A slow exit mechanism is introduced for targets that do not appear, and the system only exits when the linearly predicted fluid leaves the screen or is not detected for an extended period of time; the specific steps are as follows: For undetected objects, the corresponding coordinate and motion information is found by using a Kalman filter. Temporal prediction is then performed based on the coordinate and motion information. If the target area is close to 0 or the predicted target position is close to the edge of the image, it is confirmed that the object has not been detected. Otherwise, within a time series of length t, it is considered to be obscured or missed, and the parameters in the filter are used as the motion standard. If it is still not detected after time t, it is considered to have disappeared.

9. A video target detection system suitable for industrial fluid level tracking, used to implement the method of any one of claims 1-8, characterized in that, include: The data processing module is used to acquire real data and mixed data, and to label and preprocess the data; The model building module constructs the backbone using a partial ResNet50 convolutional neural network, and the core module for object recognition is constructed using an encoder-decoder module with a Transformer self-attention mechanism. The original position-based query is replaced with a query based on object category and the previous frame position prediction. A long-term feature memory module is introduced into the network to provide richer contextual information for object detection. The model training module initializes the model, sets the corresponding parameters, and performs training. The post-processing module uses a slow exit mechanism to handle cases where no objects are detected.

10. A video target detection device suitable for industrial fluid level tracking, comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that, When the processor executes the executable code, it implements a video target detection method for industrial fluid level tracking as described in any one of claims 1-8.

Images