Liquid leak detection method, apparatus, system, and machine-readable storage medium

Abstract

The application discloses a liquid leakage detection method, device, system and machine readable storage medium, wherein the method comprises: obtaining a difference image according to obtained target video data; determining an input original tensor according to a first color channel corresponding to a first frame image in the target video data and a second color channel corresponding to the difference image; inputting the input original tensor into a detection model to obtain a first tensor; and obtaining a first weight related to a channel and a second weight related to a position according to the first tensor and the detection model; correcting the first tensor according to the first weight and the second weight to obtain a corrected target tensor; and finally determining a detection result according to the target tensor; and the detection model is established based on historical video data. Through the method in the application, the accuracy of the detection result can be improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of liquid leakage detection, and particularly relates to a liquid leakage detection method, device, system and machine readable storage medium. BACKGROUND

[0002] In the process of storing and transporting liquid substances, the transport pipeline or storage equipment may have an abnormal situation of liquid leakage. If leakage occurs, it may have serious consequences. For example, when crude oil leaks, it is easy to cause fire and explosion accidents and cause environmental pollution and harm to the safety of the people. Therefore, in order to reduce the probability of accidents, it is necessary to regularly check the equipment and pipelines for storing and transporting liquids to identify whether the equipment and pipelines have liquid leakage.

[0003] The general detection network model in the prior art is used to realize the identification of the liquid area. Due to the transparency of the liquid itself and the continuous change of the liquid area, the difference between the leaked liquid area and the background is small, and the identification effect is poor. SUMMARY

[0004] In view of the above problems, the present application provides a liquid leakage detection method, device, system and machine readable storage medium to improve the accuracy of the detection result.

[0005] A liquid leakage detection method according to the present application comprises the following steps:

[0006] According to the obtained target video data, the difference between any two target video frames in the target video data is compared, and a difference image is obtained;

[0007] According to the first color channel corresponding to the first frame image in the target video data and the second color channel corresponding to the difference image, an input original tensor is determined;

[0008] The input original tensor is input into a detection model, and the detection result is output; wherein the detection model is established according to historical video data, and the detection result includes position information of the liquid leakage area and confidence corresponding to each liquid leakage area.

[0009] Further, the step of comparing the difference between any two target video frames in the target video data according to the obtained target video data and obtaining a difference image comprises:

[0010] Obtain the target video data to be detected within the target time period;

[0011] selecting at least two target video frames as a first key frame and a second key frame from the target video data according to a preset time interval, the first key frame corresponding to a time point earlier than a time point corresponding to the second key frame;

[0012] obtaining the difference image according to the first key frame and the second key frame;

[0013] When the target video frames are two, the two target video frames are determined as the first key frame and the second key frame.

[0014] When the target video frames are more than two, differences between any two adjacent target video frames are compared, and the two adjacent target video frames with the largest difference are determined as the first key frame and the second key frame.

[0015] Further, when the target video frames are more than two, the step of comparing the differences between any two adjacent target video frames and determining the two adjacent target video frames with the largest difference as the first key frame and the second key frame comprises:

[0016] For any two adjacent first target video frame and second target video frame, a first difference value between a first pixel value of a first pixel point and a second pixel value of a second pixel point is calculated, wherein the first pixel point is any one pixel point on the first target video frame, and the second pixel point is a pixel point on the second target video frame located at a corresponding position of the first pixel point.

[0017] For each first difference value, an absolute value of the first difference value is calculated.

[0018] According to each absolute value, an average value of the absolute values is calculated, and the average value is taken as a difference value between the first target video frame and the second target video frame.

[0019] According to the difference values between any two adjacent target video frames, the two adjacent target video frames with the largest difference value are determined as the first key frame and the second key frame.

[0020] Further, the step of obtaining the difference image according to the first key frame and the second key frame comprises:

[0021] A second difference value between a fourth pixel value of a fourth pixel point and a third pixel value of a third pixel point is calculated, wherein the fourth pixel point is any one pixel point on the second key frame, and the third pixel point is a pixel point on the first key frame located at a corresponding position of the fourth pixel point.

[0022] For each of the second difference values, an absolute value of the second difference value is determined as a fifth pixel value of a pixel point at a corresponding position on the difference image, to obtain the difference image.

[0023] Further,

[0024] The step of inputting the output original tensor into the detection model and outputting the detection result comprises:

[0025] In the detection model: inputting the input original tensor into a convolutional neural network in the detection model to obtain a first tensor; wherein the input original tensor comprises the first color channel and the second color channel;

[0026] According to the first tensor, a first weight and a second weight are obtained;

[0027] According to the first weight and the second weight, the first tensor is corrected to obtain a corrected target tensor;

[0028] According to the target tensor, the detection result is determined.

[0029] Further, the step of obtaining the first weight and the second weight according to the first tensor comprises:

[0030] The first tensor is globally pooled to obtain a second tensor, and the length and width of the second tensor are 1; wherein the second tensor has the same number of channels as the first tensor;

[0031] The second tensor is input into an activation function to output the first weight corresponding to each channel in the first tensor;

[0032] For each channel in the first tensor, the first tensor and the first weight corresponding to the channel are multiplied to obtain a third tensor;

[0033] The third tensor is globally average-pooled to obtain a fourth tensor with a channel number of 1; the fourth tensor is input into a convolutional neural network of the detection model to obtain a second weight corresponding to each position on the fourth tensor; wherein the convolutional neural network comprises an activation function.

[0034] Further, the step of correcting the first tensor according to the first weight and the second weight to obtain a corrected target tensor comprises:

[0035] For the third tensor and the second weight corresponding to each position, the third tensor and the second weight corresponding to the position are point-multiplied to obtain a fifth tensor;

[0036] The fifth tensor obtained is taken as a new first tensor, and the step of performing global pooling on the first tensor is continued until a preset number of times is reached, and the fifth tensor generated last time is taken as the target tensor after modification.

[0037] Further, the first color channel is a composite channel; the second color channel is the composite channel or a gray channel; and the composite channel is a red channel, a green channel and a blue channel.

[0038] Further, after the step of outputting the detection result, the method further comprises the steps of:

[0039] According to the corresponding confidence of each liquid leakage area, a target confidence greater than a first threshold is screened out, and the liquid leakage area corresponding to the target confidence is taken as a candidate liquid leakage area.

[0040] According to the calculated intersection-over-union between any two candidate liquid leakage areas, when the intersection-over-union is greater than a second threshold, the candidate liquid leakage area corresponding to the intersection-over-union is merged to obtain a target liquid leakage area.

[0041] Further, after the step of obtaining the target liquid leakage area, the method further comprises the steps of:

[0042] The position information of the target liquid leakage area is sent to a monitoring terminal for liquid leakage early warning.

[0043] The application also provides a liquid leakage detection device, comprising:

[0044] The acquisition module is configured to compare the difference between any two target video frames in the target video data and acquire a difference image according to the acquired target video data.

[0045] The determination module is configured to determine an input original tensor according to a first color channel corresponding to a first frame image in the target video data and a second color channel corresponding to the difference image.

[0046] The input and output module is configured to input the input original tensor into a detection model and output the detection result, wherein the detection model is established in the model establishment module according to historical video data, and the detection result comprises position information of a liquid leakage area and a confidence corresponding to each liquid leakage area.

[0047] The application also provides a machine-readable storage medium having processor-executable machine-readable instructions stored thereon, wherein the machine-readable instructions are executed by a processor to implement the liquid leakage detection method described above.

[0048] The application further provides a liquid leakage detection system, comprising:

[0049] A video acquisition device is configured to acquire target video data and transmit the target video data to the liquid leakage detection device.

[0050] The liquid leakage detection device is configured to receive the target video data and implement the liquid leakage detection method as described above.

[0051] A monitoring terminal is configured to receive position information of a liquid leakage area and perform early warning.

[0052] The application further provides an electronic device, comprising a processor, a memory and a bus, the memory stores machine readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, and the machine readable instructions are executed by the processor to implement the steps of the liquid leakage detection method as described above.

[0053] The application provides a liquid leakage detection method, device, system and machine readable storage medium, wherein the liquid leakage detection method is to determine a first key frame and a second key frame with the largest difference from target video data, determine an input original tensor according to a difference image formed by a first frame image in the target video data and a difference between the first key frame and the second key frame, compared with the method of determining a tensor corresponding to each frame image according to each frame image in the prior art, the difference between the liquid area and the background in the input original tensor is enhanced, and the saliency feature of the liquid area is improved.

[0054] Further, in the present application, the input original tensor is input into a convolutional neural network to obtain a first tensor, then the first weight corresponding to each channel and the second weight corresponding to each position are obtained from the first tensor, the first tensor is corrected according to the first weight and the second weight to obtain a corrected target tensor, and the detection result is determined according to the target tensor. Compared with the method of directly extracting the liquid area in each frame image through a general detection network model in the prior art, the first weight related to the channel and the second weight related to the position are obtained from the first tensor and the detection model in the present application, the first tensor is corrected to obtain the target tensor through the first weight and the second weight, and then the corrected target tensor is detected to accurately detect the liquid area in the video and improve the accuracy of the detection result.

[0055] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the present application. The objects and other advantages of the present application can be realized and achieved by the structures indicated in the specification, claims and drawings. BRIEF DESCRIPTION OF DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings described below are only some embodiments of the present application, and the ordinary skilled in the art can obtain other drawings according to these drawings without any creative effort.

[0057] Figure 1 A flow chart of a liquid leakage detection method according to an embodiment of the present application is shown;

[0058] Figure 2 A schematic diagram of each first pixel point on a first target video frame according to an embodiment of the present application is shown;

[0059] Figure 3 A schematic diagram of each second pixel point on a second target video frame according to an embodiment of the present application is shown;

[0060] Figure 4 A schematic diagram of each third pixel point on a first key frame according to an embodiment of the present application is shown;

[0061] Figure 5 A schematic diagram of each fourth pixel point on a second key frame according to an embodiment of the present application is shown;

[0062] Figure 6 A schematic diagram of each fifth pixel point on a difference image according to an embodiment of the present application is shown;

[0063] Figure 7 A structural schematic diagram of a first unit in a detection model according to an embodiment of the present application is shown;

[0064] Figure 8 A structural schematic diagram of a liquid leakage detection device according to an embodiment of the present application is shown;

[0065] Figure 9 A structural schematic diagram of an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0066] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the following will combine the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by the ordinary skilled in the art without any creative effort are within the protection scope of the present application.

[0067] In view of the poor recognition effect in detecting the liquid leakage area in the prior art, embodiments of the present application provide a detection method and device, an electronic device and a computer readable storage medium to improve the accuracy of the detection result, which are described below.

[0068] The liquid leakage detection method comprises the following steps:

[0069] Obtain target video data, and compare the difference between any two target video frames in the target video data to obtain a difference image.

[0070] According to the first color channel corresponding to the first frame image in the target video data and the second color channel corresponding to the difference image, an input raw tensor is determined.

[0071] The input raw tensor is input into a detection model to output the detection result, wherein the detection model is established according to historical video data, and the detection result comprises position information of the liquid leakage area and a confidence degree corresponding to each liquid leakage area.

[0072] The liquid detection method of the present application determines the first key frame and the second key frame with the largest difference from the target video data, and determines the input raw tensor according to the first frame image in the target video data and the difference image formed between the first key frame and the second key frame. Compared with the method of determining the tensor corresponding to each frame image in the prior art, the difference between the liquid area and the background in the first tensor is enhanced, and the saliency feature of the liquid area is improved. The detection model of the present application is established based on historical video data. By using a large amount of historical video data to obtain different historical frame images and training the detection model a large number of times, the generalization ability of the model is greatly improved, and the detection effect of liquid leakage is greatly improved when applied to liquid leakage detection.

[0073] Embodiment one:

[0074] In order to facilitate the understanding of the present embodiment, first of all, a liquid leakage detection method disclosed in the present application is described in detail. Figure 1 A flow chart of a liquid leakage detection method provided by the present application is shown in FIG. Figure 1 As shown in the figure, it comprises the following steps:

[0075] S101: According to the obtained target video data, the difference between any two target video frames in the target video data is determined, and the two target video frames with the largest difference are taken as the first key frame and the second key frame.

[0076] In the embodiments of the present application, the target video data can be video data of a to-be-detected object recorded in real time by a camera device, that is, the target video data is real-time generated data. The to-be-detected object can be a liquid conveying device or a liquid storage device, specifically, a device facility such as a pipeline, a storage tank, etc. for storage or transportation filled with liquid. The liquid can be crude oil, water, etc. The target video frame can be any one of the frames of the target video data.

[0077] In the embodiments of the present application, since the relative position of the camera device and the to-be-detected object does not change, and the color, shape, etc. of the to-be-detected object itself does not change, the difference to be detected in the present application generally refers to the difference caused by the area of the leaked liquid. The first key frame and the second key frame have a certain time interval. For example, the first key frame can be an image when the liquid has not leaked, and the second key frame can be an image when the liquid has leaked, and the difference at this time refers to the leaked liquid.

[0078] S102: determining an input raw tensor according to a first color channel corresponding to a first frame image in the target video data and a second color channel corresponding to a difference image; wherein the difference image is an image formed by the difference between the first key frame and the second key frame; the first color channel is a composite channel; the second color channel is a composite channel or a grayscale channel; and the composite channel is a red channel, a green channel and a blue channel.

[0079] In the embodiments of the present application, the first frame image refers to the first frame image in the target video data. The format of each frame image in the target video data in the present application is RGB mode, that is, the format of the first frame image is also RGB mode, so the first frame image contains three first color channels, which are a red channel, a green channel and a blue channel.

[0080] The difference image is an image formed by the difference between the first key frame and the second key frame, wherein the first key frame and the second key frame are both images in the target video data, so the first key frame and the second key frame also contain three color channels.

[0081] In generating the difference image according to the first key frame and the second key frame, the first method is to directly generate the difference image through the first key frame and the second key frame containing three color channels, so that the generated difference image contains three second color channels; the second method is to first perform grayscale processing on the first key frame and the second key frame, and then generate the difference image according to the grayscale processed first key frame and the second key frame, so that the generated difference image contains only one grayscale channel.

[0082] In the process of determining the input original tensor according to the first color channel corresponding to the first frame image in the target video data and the second color channel corresponding to the difference image, specifically, when the difference image contains a composite channel (i.e., three second color channels: red channel, green channel and blue channel), the three first color channels are superimposed (i.e., combined, synthesized) with the three second color channels to generate a target image, at this time, the target image includes 6 color channels, and then the target image is converted into an input original tensor, that is, the input original tensor includes 6 color channels.

[0083] Or it can also be that when the difference image contains a gray channel, the three first color channels are superimposed with the gray channel to generate a target image, at this time, the target image includes 4 color channels, and then the target image is converted into an input original tensor. It should be noted that the tensor (including the input original tensor) mentioned in the present application refers to a high-dimensional vector.

[0084] S103: inputting the input original tensor into the detection model to output a detection result; wherein in the detection model: inputting the input original tensor into a convolutional neural network in the detection model to obtain a first tensor;

[0085] According to the first tensor, a first weight and a second weight are obtained; the color channels of the input original tensor include first color channels and second color channels; the first tensor is corrected according to the first weight and the second weight to obtain a corrected target tensor; the detection result is determined according to the target tensor; the detection result includes position information of a liquid leakage area and confidence corresponding to each liquid leakage area.

[0086] Specifically, the process of determining the detection result according to the target tensor is to re-input the obtained target tensor into the convolutional neural network of the detection model to determine the detection result through classification and regression. In practice, the target video frame in the target video data may be obtained according to the sudden situation (including the entry and exit of personnel / animals, changes in weather, etc.), and the corresponding difference image, that is, the difference image may not appear liquid leakage, and needs to be further confirmed through classification and regression processing. Only in the case of liquid leakage, the detection result related to liquid leakage will be output.

[0087] In the embodiments of the present application, when the first tensor includes N channels, after obtaining the first weight corresponding to each channel in the first tensor, N first weights are obtained.

[0088] In the embodiments of the present application, the first weight represents the importance of each channel in the first tensor, and the second weight represents the importance of each position in the third tensor relative to other positions.

[0089] The first tensor can be modified by the first weight first, and then the modified first tensor can be modified again by the second weight, and then the target tensor is obtained.

[0090] In the present application, the position information of the liquid leakage area can be the coordinates of the liquid leakage, and the confidence degree is used to represent the probability that the detected liquid leakage area is a real leakage area.

[0091] In the present application, the difference image formed by the first key frame and the second key frame with large difference is input into the detection model, so that the detection model extracts effective features about the liquid in the time dimension.

[0092] In one possible implementation, when performing S101, according to the obtained target video data, the difference between any two target video frames in the target video data is determined, and the two target video frames with the largest difference are determined as the first key frame and the second key frame, the following steps can be performed specifically:

[0093] S1011: Obtain target video data in a target time period.

[0094] The target time period can be 10 minutes, half an hour, etc., which is not limited in the present application.

[0095] S1012: According to a preset time interval, at least two target video frames are selected from the target video data.

[0096] The preset time interval is not greater than the target time period. For example, when the target time period is 10 minutes and the preset time interval is 1 minute, 11 target video frames are selected from the target video data.

[0097] S1013: When the target video frame is two, the two target video frames are determined as the first key frame and the second key frame.

[0098] In one specific embodiment, when the preset time interval is equal to the target time period, the two selected target video frames are the first frame image and the last frame image in the target video data. At this time, the first frame image and the last frame image in the target video data are directly determined as the first key frame and the second key frame.

[0099] S1014: When the target video frame is greater than two, according to the difference between any two adjacent target video frames, the two adjacent target video frames with the largest difference are determined as the first key frame and the second key frame.

[0100] For example, when the target video frames are three, namely target video frame A, target video frame B and target video frame C, the difference a between the target video frame A and the target video frame B is determined, and the difference b between the target video frame B and the target video frame C is determined. When the difference a is less than the difference b, the target video frame B and the target video frame C are determined as the first key frame and the second key frame.

[0101] In another possible implementation, when the first key frame and the second key frame are determined from the target video frames, the determination can also be performed by using the frame difference method.

[0102] In a possible implementation, when the target video frames are more than two in the step S1014, the two target video frames with the largest difference between any two adjacent target video frames are determined as the first key frame and the second key frame according to the difference between the two adjacent target video frames. The determination can be performed according to the following steps:

[0103] S10141: calculating a first difference value between a first pixel value of a first pixel point and a second pixel value of a second pixel point for any two adjacent first target video frames and second target video frames, wherein the first pixel point is any one pixel point on the first target video frame, and the second pixel point is a pixel point on the second target video frame which is located at a corresponding position of the first pixel point.

[0104] When the difference between the first target video frame and the second target video frame is calculated, since the number of the first pixel points contained in the first target video frame is the same as the number of the second pixel points contained in the second target video frame, the first pixel value of each first pixel point on the first target video frame can be subtracted from the second pixel value of the second pixel point at the corresponding position on the second target video frame to obtain the first difference value of each first pixel point and the second pixel point at the corresponding position. The number of the first pixel points, the number of the second pixel points and the number of the first difference values are the same.

[0105] For example, Figure 2 A diagram of each first pixel point on the first target video frame provided by the embodiment of the present application is shown as follows, Figure 2 as shown in the figure, each box represents a first pixel point, and the value in each box represents the first pixel value of the first pixel point. The first pixel value of the first pixel point M1 is 152; the first pixel value of the first pixel point M2 is 240; the first pixel value of the first pixel point M3 is 56; and the first pixel value of the first pixel point M4 is 88.

[0106] Figure 3 A diagram of each second pixel point on the second target video frame provided by the embodiment of the present application is shown as follows, Figure 3As shown, each block represents a second pixel point, and the value in each block represents the second pixel value of the second pixel point. The second pixel value of the second pixel point N1 is 254; the second pixel value of the second pixel point N2 is 38; the second pixel value of the second pixel point N3 is 56; and the second pixel value of the second pixel point N4 is 66. The first pixel point M1 and the second pixel point N1 are located at a corresponding position; and the first pixel point M2 and the second pixel point N2 are located at a corresponding position.

[0107] In the calculation of the first difference value C1, specifically, 152 (the first pixel value of the first pixel point M1) - 254 (the second pixel value of the second pixel point N1) = -102 (the first difference value C1). Similarly, in the calculation of the first difference value C2, specifically, 240 (the first pixel value of the first pixel point M2) - 38 (the second pixel value of the second pixel point N2) = 202 (the first difference value C2). In the calculation of the first difference value C3, specifically, 56 (the first pixel value of the first pixel point M3) - 56 (the second pixel value of the second pixel point N3) = 0 (the first difference value C3). In the calculation of the first difference value C4, specifically, 88 (the first pixel value of the first pixel point M4) - 66 (the second pixel value of the second pixel point N4) = 22 (the first difference value C4).

[0108] S10142: For each first difference value, calculate the absolute value of the first difference value.

[0109] In the embodiment of step S10141, the absolute value of the first difference value C1 (i.e. -102) is 102; the absolute value of the first difference value C2 (i.e. 202) is 202; the absolute value of the first difference value C3 (i.e. 0) is 0; and the absolute value of the first difference value C4 (i.e. 22) is 22.

[0110] S10143: According to each absolute value, calculate the average value of the absolute values, and take the average value as the difference value between the first target video frame and the second target video frame.

[0111] In the embodiment of step S10142, in the calculation of the average value of the absolute values, specifically, the average value = (101 + 202 + 0 + 22) / 4 = 81.25. At this time, the difference value between the first target video frame and the second target video frame is 81.25.

[0112] S10144: According to the difference value between any two adjacent target video frames, determine the two adjacent target video frames with the largest difference value as the first key frame and the second key frame.

[0113] Any two adjacent target video frames correspond to a difference value, and two adjacent target video frames with the largest difference value are determined as the first key frame and the second key frame. In the present application, adjacent refers to two adjacent target video frames after the target video frames are sorted in chronological order.

[0114] In a possible implementation, the time point corresponding to the first key frame is earlier than the time point corresponding to the second key frame; before step S102 of determining the input original tensor according to the first color channel corresponding to the first frame image in the target video data and the second color channel corresponding to the difference image, the following steps can also be performed:

[0115] S1021: calculating a second difference value between a fourth pixel value of a fourth pixel point and a third pixel value of a third pixel point; wherein the fourth pixel point is any one pixel point on the second key frame, and the third pixel point is a pixel point on the first key frame located at a corresponding position of the fourth pixel point.

[0116] For example, Figure 4 FIG. 1 shows a schematic diagram of each third pixel point on the first key frame provided by an embodiment of the present application, Figure 5 FIG. 2 shows a schematic diagram of each fourth pixel point on the second key frame provided by an embodiment of the present application, such as Figure 4 and Figure 5 As shown in FIG. 2, when calculating the second difference value, specifically, the fourth pixel value of each fourth pixel point on the second key frame is subtracted from the third pixel value of the third pixel point at the corresponding position on the first key frame to obtain the second difference value of each fourth pixel point and the third pixel point at the corresponding position. The number of third pixel points, the number of fourth pixel points and the number of second difference values are the same.

[0117] When calculating the second difference value, specifically, 86-65=21; 95-95=0; 156-156=0; 226-83=143; 62-26=36; 6-16=-10; 33-44=-11; 96-89=7; 17-17=0. The specific calculation principle can refer to the embodiment of calculating the first difference value in step S10141.

[0118] S1022: for each second difference value, determining the absolute value of the second difference value as a fifth pixel value of a pixel point at a corresponding position on the difference image to obtain the difference image.

[0119] Based on the embodiment in step S1021, Figure 6 FIG. 3 shows a schematic diagram of each fifth pixel point on the difference image provided by an embodiment of the present application, such as Figure 6As shown, in this application, for each second difference value, the absolute value of the second difference value is calculated, and then the absolute value of the second difference value is determined as the fifth pixel value of the pixel point at the corresponding position on the difference image, to obtain the difference image.

[0120] In this application, by subtracting the pixel values of the pixel points at the corresponding positions of the first key frame and the second key frame, the same background region in the first key frame and the second key frame is removed, and only the region different between the two key frames, i.e. the region of difference between the two key frames, is left.

[0121] In a possible implementation, when the first weight and the second weight are obtained from the first tensor in step S103, the following steps can be performed specifically:

[0122] S1031: performing global pooling on the first tensor to obtain a second tensor; wherein the first tensor and the second tensor contain the same number of channels;

[0123] In the embodiments of this application, the detection model includes X units, the first unit is the first unit in the detection model, Figure 7 As shown in the structural schematic diagram of the first unit in the detection model provided by the embodiments of this application, Figure 7 As shown, the first unit includes a channel layer attention mechanism module, a first correction module, a non-local spatial attention mechanism module and a second correction module.

[0124] In this application, the first tensor is input to the channel layer attention mechanism module through the input module, and global pooling is performed on the first tensor in the channel layer attention mechanism module to obtain the second tensor. For example, the image corresponding to the first tensor is H (image length) x W (image width) x T (channel number) one image, and after global pooling, the image corresponding to the obtained second tensor is 1 (image length) x 1 (image width) x T (channel number) one image.

[0125] S1032: inputting the second tensor into an activation function to output the first weight corresponding to each channel of the first tensor.

[0126] In the channel layer attention mechanism module, the second tensor is input into an activation function (Sigmoid function) to obtain the first weight corresponding to each channel. In this application, the channel layer attention mechanism module outputs the first weight corresponding to each channel. That is, the first weight corresponding to each channel in the first tensor is obtained.

[0127] S1033: for each channel in the first tensor, performing a product operation on the first tensor and the first weight corresponding to the channel to obtain a third tensor.

[0128] The first correction module receives the first tensor output by the input module and the first weights corresponding to each channel output by the channel layer attention mechanism module. In the first correction module, for each channel in the first tensor, the first tensor and the first weight corresponding to that channel are multiplied to perform feature correction on each channel in the first tensor, resulting in a third tensor. The image corresponding to the third tensor is an image of H (image length) × W (image width) × T (number of channels).

[0129] S1034: Perform global average pooling on the third tensor to obtain the fourth tensor; the fourth tensor has 1 channel.

[0130] The third tensor output by the first correction module is input into the nonlocal spatial attention mechanism module. In the nonlocal spatial attention mechanism module, global average pooling is performed on the third tensor, and the T channels contained in the third tensor are normalized to obtain the fourth tensor. The image corresponding to the fourth tensor is an image of H (image length) × W (image width) × 1 (number of channels).

[0131] S1035: Input the fourth tensor into the convolutional neural network to obtain the second weight corresponding to each position (on the image) of the third tensor; wherein, the convolutional neural network includes an activation function.

[0132] In the nonlocal spatial attention mechanism module, the fourth tensor is input into the convolutional neural network to obtain the second weights corresponding to each position on the third tensor. The convolutional neural network includes a sigmoid activation function. The nonlocal spatial attention mechanism module outputs the second weights corresponding to each position on the third tensor.

[0133] In one possible implementation, when performing step S103 to correct the first tensor according to the first weight and the second weight to obtain the corrected target tensor, the following steps can be performed:

[0134] S1036: For the third tensor and the second weight corresponding to each position, perform a dot product operation between the third tensor and the second weight corresponding to that position to obtain the fifth tensor.

[0135] like Figure 7 As shown, the second correction module receives the second weights corresponding to each position on the target image output by the non-local spatial attention mechanism module and the third tensor output by the first correction module. In the second correction module, for each position on the target image corresponding to the second weight, the third tensor is multiplied by the second weight corresponding to that position to obtain the fifth tensor.

[0136] S1037: continue to perform the step of performing global pooling on the first tensor to obtain a second tensor until a preset number of times is met, and take the fifth tensor generated last time as the modified target tensor.

[0137] Since the detection model in the present application includes X units, the fifth tensor output by the second correction module of the first unit is taken as the output of the first unit, output through the output module of the first unit, and taken as the input of the next unit in the detection model, that is, the fifth tensor output by the second correction module in the first unit is taken as the new first tensor and input into the input module of the next unit, and the step S1031 of performing global pooling on the first tensor to obtain a second tensor is repeatedly executed until the fifth tensor output by the Xth unit is taken as the modified target tensor through the X units.

[0138] In a possible implementation, when the step S103 of determining the detection result according to the target tensor is performed, the following steps can be performed: the detection model further includes a fully connected layer, the target tensor is input into the fully connected layer, classification is performed through the fully connected layer, and position information of the liquid leakage area and confidence corresponding to each liquid leakage area are output.

[0139] In the present application, the channel layer attention mechanism module is used to extract the channel correlation weight, and each channel represents a detector, so that the channel layer attention mechanism module can determine which channel features have a positive effect on network learning of the detection model. The specific scheme is to perform global pooling to a dimension of 1x1xT (number of channels) first, then extract the features of each channel through a convolutional neural network, and finally pass the features through a sigmoid activation function to obtain a weight coefficient. Multiplying the input features by the weight coefficient obtains the feature correction of the channel attention mechanism.

[0140] The non-local spatial attention mechanism module is used to compensate for the shortcomings of the local receptive field of convolution. The non-local spatial attention mechanism module focuses on the importance of different positions in each feature spectrum relative to other positions, and extracts weights in the H (image length) and W (image width) dimensions. The specific scheme is that, for a given image feature (the third tensor in the present application), first normalize the T channels through global average pooling to obtain a feature of H (image length) x W (image width) x 1, then obtain a spatial feature weight spectrum of each position in the image corresponding to the third tensor through convolution and a sigmoid activation function, and finally perform a point multiplication operation on the third tensor to obtain the final output (the fifth tensor or the target tensor).

[0141] In the present application, the channel layer attention mechanism module and the non-local spatial attention mechanism module are combined in series, which can effectively extract the weight information of the multi-dimensional feature spectrum, so that the detection model pays more attention to the area of interest, i.e., the "liquid leakage area".

[0142] In one specific embodiment of the detection method of the present application, the detection model uses a classic ResNet50 (Residual Network) as the backbone network, uses RetinaNet as the overall framework of the detection model, adds a separable attention mechanism after Stage2 of the ResNet50 structure to redistribute the weights, and feeds the processed semantic and spatiotemporal feature maps into the subsequent deep network results for inference and calculation to obtain the final result (i.e., the detection result in the present application). It should be understood that a person skilled in the art can select different network models according to actual conditions.

[0143] In a possible implementation, after the step S103 of inputting the first tensor into the detection model and outputting the detection result, the following steps can be further performed:

[0144] S1041: According to the corresponding confidence of each liquid leakage area, a target confidence with a confidence greater than a first threshold is screened out, and the liquid leakage area corresponding to the target confidence is taken as a candidate liquid leakage area.

[0145] In the present application, the detection model may output multiple liquid leakage areas, each liquid leakage area corresponding to a confidence.

[0146] S1042: According to the calculated intersection-over-union between any two candidate liquid leakage areas, when the intersection-over-union is greater than a second threshold, the candidate liquid leakage area corresponding to the intersection-over-union is merged to obtain a target liquid leakage area.

[0147] In the present application, there may be overlapping parts between the liquid leakage areas, and the intersection-over-union refers to the degree of overlap between two candidate liquid leakage areas. When the intersection-over-union between two candidate liquid leakage areas is greater than a second threshold, the two candidate liquid leakage areas are merged to form a target liquid leakage area.

[0148] S1043: The position information of the target liquid leakage area is sent to a monitoring terminal for liquid leakage early warning.

[0149] The terminal generally monitored is a hand-held mobile terminal responsible for early warning of the relevant person in charge. The position information (i.e. early warning information) of the target liquid leakage area is sent to the terminal of the person in charge, so that the person in charge can handle it in time according to the received early warning information. The position information (i.e. early warning information) can be sent to the terminal of the person in charge in any one or more of the forms of image, text, and video. The terminal can be a mobile phone, computer, or other device of the person in charge, which is not limited in the present application.

[0150] In a possible implementation, before the step S103 of inputting the first tensor into the detection model and outputting the detection result, the step of establishing the detection model is further included. The detection model in the present application is established through learning and training of historical video data. The establishment can be performed according to the following steps:

[0151] S1001: According to the obtained historical video data, the difference between any two historical video frames in the historical video data is determined, and the two historical video frames with the largest difference are taken as the first historical key frame and the second historical key frame;

[0152] S1002: According to the third color channel corresponding to the first historical frame image in the historical video data and the fourth color channel corresponding to the historical difference image, the first historical tensor is determined; wherein the historical difference image is an image formed by the difference between the first historical key frame and the second historical key frame; the third color channel is a composite channel; the fourth color channel is a composite channel or a gray channel; the composite channel is a red channel, a green channel and a blue channel;

[0153] S1003: The first historical tensor and the leakage correct position information are input into the to-be-trained detection model, and a predicted detection result predicted by the to-be-trained detection model is output;

[0154] S1004: The predicted detection result is compared with the corresponding actual situation to obtain a loss value;

[0155] S1005: The to-be-trained detection model is trained this round using the loss value;

[0156] S1006: When the training reaches a preset number of times, the training is stopped, and the trained to-be-trained detection model is taken as the detection model.

[0157] The training sample (historical video data) provided for the to-be-trained detection model includes various different scene conditions: both scenes with leakage and scenes without leakage. By training the model with a large number of positive and negative samples, the generalization ability of the detection model is improved, i.e. the reliability of the model output result is improved.

[0158] Embodiment two:

[0159] Based on the same technical concept, the embodiment of the present application also provides a liquid leakage detection device, Figure 8 The structural diagram of the detection device provided by the embodiment of the present application is shown in the figure, and the device comprises: Figure 8

[0160] The acquisition module 801 is configured to compare the difference between any two target video frames in the target video data according to the acquired target video data, and acquire a difference image.

[0161] The determination module 802 is configured to determine an input raw tensor according to a first color channel corresponding to a first frame image in the target video data and a second color channel corresponding to the difference image.

[0162] The input and output module 803 is configured to input the input raw tensor into a detection model and output a detection result, wherein the detection model is established in the model establishment module according to historical video data, and the detection result comprises position information of a liquid leakage area and a confidence degree corresponding to each liquid leakage area.

[0163] The first color channel is a composite channel, the second color channel is the composite channel or a gray channel, and the composite channel is a red channel, a green channel and a blue channel.

[0164] In the detection model, the input raw tensor is input into a convolutional neural network in the detection model to acquire a first tensor.

[0165] The first weight and the second weight are acquired according to the first tensor, the input raw tensor comprises the first color channel and the second color channel, the first tensor is corrected according to the first weight and the second weight to obtain a corrected target tensor, and the detection result is determined according to the target tensor.

[0166] Optionally, when the acquisition module 801 determines the difference between any two target video frames in the target video data according to the acquired target video data, and takes the two target video frames with the largest difference as a first key frame and a second key frame, the acquisition module 801 is specifically configured to:

[0167] Acquire target video data in a target time period.

[0168] Select at least two target video frames from the target video data according to a preset time interval.

[0169] When the target video frames are two, the two target video frames are determined as the first key frame and the second key frame.

[0170] ​When the target video frames are more than two, two target video frames with the largest difference between any two adjacent target video frames are determined as the first key frame and the second key frame according to the difference.

[0171] Optionally, the acquisition module 801 is specifically used for:

[0172] For any two adjacent first target video frames and second target video frames, a first difference value between a first pixel value of a first pixel point and a second pixel value of a second pixel point is calculated; wherein the first pixel point is any one pixel point on the first target video frame, and the second pixel point is a pixel point on the second target video frame which is located at a corresponding position with the first pixel point;

[0173] For each first difference value, an absolute value of the first difference value is calculated;

[0174] According to each absolute value, an average value of the absolute values is calculated, and the average value is taken as a difference value between the first target video frame and the second target video frame;

[0175] According to the difference value between any two adjacent target video frames, two target video frames with the largest difference value are determined as the first key frame and the second key frame.

[0176] Optionally, the time point corresponding to the first key frame is earlier than the time point corresponding to the second key frame.

[0177] The acquisition module is specifically used for:

[0178] Before the determination module 802 determines the input original tensor according to the first color channel corresponding to the first frame image in the target video data and the second color channel corresponding to the difference image, a second difference value between a fourth pixel value of a fourth pixel point and a third pixel value of a third pixel point is calculated; wherein the fourth pixel point is any one pixel point on the second key frame, and the third pixel point is a pixel point on the first key frame which is located at a corresponding position with the fourth pixel point;

[0179] For each second difference value, an absolute value of the second difference value is determined as a fifth pixel value of a pixel point at a corresponding position on the difference image, so as to obtain the difference image.

[0180] Optionally, the input and output module 803 is specifically configured to:

[0181] perform global pooling on the first tensor to obtain a second tensor; the second tensor has the same number of channels as the first tensor;

[0182] input the second tensor into an activation function to output a first weight corresponding to each channel of the first tensor; thus, the first weight corresponding to each channel of the first tensor is obtained;

[0183] for each channel in the first tensor, perform a product operation on the first tensor and the first weight corresponding to the channel to obtain a third tensor;

[0184] perform global average pooling on the third tensor to obtain a fourth tensor; the fourth tensor has one channel;

[0185] input the fourth tensor into a convolutional neural network to obtain a second weight corresponding to each position (on an image) of the third tensor; the convolutional neural network layer includes an activation function.

[0186] Optionally, the input and output module 803 is specifically configured to:

[0187] after obtaining the third tensor through the first tensor and the first weight, for the third tensor and the second weight corresponding to each position, perform a dot product operation on the third tensor and the second weight corresponding to the position to obtain a fifth tensor;

[0188] use the obtained fifth tensor as a new first tensor, continue to perform the step of performing global pooling on the first tensor to obtain a second tensor, until a preset number of times is met, and use the fifth tensor generated last time as the target tensor after modification.

[0189] Optionally, the method further comprises:

[0190] a screening module configured to, after the input and output module 803 inputs the input original tensor into the detection model and outputs a detection result, screen out a target confidence having a confidence greater than a first threshold according to the confidence corresponding to each liquid leakage area, and take the liquid leakage area corresponding to the target confidence as a candidate liquid leakage area;

[0191] a merging module configured to, according to a calculated intersection over union between any two candidate liquid leakage areas, when the intersection over union is greater than a second threshold, merge the candidate liquid leakage areas corresponding to the intersection over union to obtain a target liquid leakage area.

[0192] The transmitting module is used to send the location information of the target liquid leak area to the monitoring terminal for early warning. The monitoring terminal is typically a handheld mobile device belonging to the person in charge, serving to provide early warnings to relevant personnel.

[0193] For details on the specific implementation methods, steps, and principles, please refer to the description in Example 1, which will not be repeated here.

[0194] Example 3:

[0195] Based on the same technical concept, embodiments of this application also provide an electronic device. Figure 9 This application provides a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 9 As shown, the electronic device 900 includes a processor 901, a memory 902, and a bus 903. The memory stores machine-readable instructions that can be executed by the processor. When the electronic device is running, the processor 901 communicates with the memory 902 through the bus 903. The processor 901 executes the machine-readable instructions to perform the steps of the liquid leak detection method described in Embodiment 1.

[0196] For details on the specific implementation methods, steps, and principles, please refer to the description in Example 1, which will not be repeated here.

[0197] Example 4:

[0198] Based on the same technical concept, Embodiment 4 of this application also provides a machine-readable storage medium storing machine-readable instructions executable by the processor. When the machine-readable instructions are executed by the processor, they implement the steps of the method described in Embodiment 1.

[0199] Specifically, the machine-readable storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks. When the machine-readable instructions contained in the machine-readable storage medium are executed (which may be computer devices such as personal computers, servers, or network devices), all or part of the steps of the liquid leak detection method described in Embodiment 1 of the present invention are implemented.

[0200] For details on the specific implementation methods, steps, and principles, please refer to the description in Example 1, which will not be repeated here.

[0201] Example 5:

[0202] Based on the same technical concept, the application further provides a liquid leakage detection system, comprising a video acquisition device, a liquid leakage detection device and a monitoring terminal. The video acquisition device acquires target video data and transmits the target video data to the liquid leakage detection device. The liquid leakage detection device receives the target video data and implements the liquid leakage detection method described in the first embodiment. The monitoring terminal receives position information of a liquid leakage area and performs early warning.

[0203] The video acquisition device can transmit the acquired video data to the liquid leakage detection device in real time through a wired or wireless manner. It should be understood that in the application scenario of multiple monitoring points, the video acquisition device and the monitoring terminal can be correspondingly arranged for the monitoring points.

[0204] For the specific method steps and principles of implementation, refer to the description of the first embodiment, which will not be described in detail here.

[0205] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0206] In the several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented by other means. The device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there can be another division manner in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some communication interface, device or unit, and can be electrical, mechanical or other forms.

[0207] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment scheme.

[0208] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.

[0209] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a nonvolatile computer readable storage medium executable by a processor. Based on this understanding, the technical solutions of the present application or the parts of the present application that essentially contribute to the prior art or the parts of the technical solutions can be embodied in the form of software products. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0210] Although the present application is described in detail with reference to the foregoing embodiments, it should be understood by those skilled in the art that the technical solutions described in the foregoing embodiments can be modified, or some of the technical features can be replaced by equivalents; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for detecting liquid leaks, characterized in that, Includes the following steps: Based on the acquired target video data, compare the differences between any two target video frames in the target video data to obtain a difference image; The input raw tensor is determined based on the first color channel corresponding to the first frame image in the target video data and the second color channel corresponding to the difference image. The original input tensor is input into the detection model, and the detection result is output; wherein, the detection model is established based on historical video data, and the detection result includes the location information of the liquid leakage area and the confidence level corresponding to each liquid leakage area; The detection model includes a first unit, which includes a channel layer attention mechanism module, a first correction module, a non-local spatial attention mechanism module, and a second correction module. The process of inputting the original tensor into the detection model and outputting the detection result includes calculating the first weight, the second weight, and correcting the first tensor. The first tensor is input into the channel layer attention mechanism module through the input module. The first tensor is then globally pooled in the channel layer attention mechanism module to obtain the second tensor. In the channel layer attention mechanism module, the second tensor is input into the activation function to obtain the first weight corresponding to each channel; For each channel in the first tensor, the first correction module performs a product operation between the first tensor and the first weight corresponding to that channel to obtain the third tensor; The third tensor output by the first correction module is input into the nonlocal spatial attention mechanism module. In the nonlocal spatial attention mechanism module, global average pooling is performed on the third tensor, and the T channels contained in the third tensor are normalized to obtain the fourth tensor. In the nonlocal spatial attention mechanism module, the fourth tensor is input into the convolutional neural network to obtain the second weight corresponding to each position on the third tensor; The second correction module performs a dot product operation between the third tensor and the second weight corresponding to each position to obtain the fifth tensor. The fifth tensor obtained is used as the new first tensor. The steps are continued to perform global pooling on the first tensor to obtain the second tensor, until the preset number of times is met. The fifth tensor generated last time is used as the corrected target tensor. The detection result is determined based on the target tensor.

2. The liquid leakage detection method according to claim 1, characterized in that, The step of comparing the differences between any two target video frames in the acquired target video data to obtain a difference image includes: Acquire the target video data to be detected within the target time period; According to a preset time interval, at least two target video frames are selected from the target video data as a first key frame and a second key frame, wherein the time point corresponding to the first key frame is earlier than the time point corresponding to the second key frame. The difference image is obtained based on the first keyframe and the second keyframe; When there are two target video frames, the two target video frames are determined as the first key frame and the second key frame. When there are more than two target video frames, the difference between any two adjacent target video frames is compared, and the two adjacent target video frames with the largest difference are determined as the first key frame and the second key frame.

3. The liquid leakage detection method according to claim 2, characterized in that, When there are more than two target video frames, the step of comparing the differences between any two adjacent target video frames and determining the two adjacent target video frames with the largest differences as the first keyframe and the second keyframe includes: For any two adjacent first target video frames and second target video frames, calculate the first difference between the first pixel value of the first pixel point and the second pixel value of the second pixel point; wherein, the first pixel point is any pixel point on the first target video frame, and the second pixel point is a pixel point on the second target video frame that is located at the corresponding position of the first pixel point. For each of the first differences, calculate the absolute value of the first difference; Based on each absolute value, the average value of the absolute values ​​is calculated, and the average value is used as the difference value between the first target video frame and the second target video frame. Based on the difference value between any two adjacent target video frames, the two adjacent target video frames with the largest difference value are determined as the first keyframe and the second keyframe.

4. The liquid leakage detection method according to claim 2, characterized in that, The step of obtaining the difference image based on the first keyframe and the second keyframe includes: Calculate the second difference between the fourth pixel value of the fourth pixel point and the third pixel value of the third pixel point; wherein, the fourth pixel point is any pixel point on the second keyframe, and the third pixel point is the pixel point on the first keyframe that is located at the corresponding position of the fourth pixel point; For each of the second differences, the absolute value of the second difference is determined as the fifth pixel value of the corresponding pixel point on the difference image, so as to obtain the difference image.

5. The liquid leakage detection method according to claim 1, characterized in that, The step of inputting the original output tensor into the detection model and outputting the detection result includes: In the detection model: the original input tensor is input into the convolutional neural network in the detection model to obtain a first tensor; wherein, the original input tensor includes the first color channel and the second color channel; The first weight and the second weight are obtained based on the first tensor; The first tensor is modified according to the first weight and the second weight to obtain the modified target tensor; The detection result is determined based on the target tensor.

6. The liquid leakage detection method according to claim 5, characterized in that, The steps of obtaining the first weight and the second weight based on the first tensor include: The first tensor is globally pooled to obtain a second tensor with a length and width of 1; wherein the second tensor contains the same number of channels as the first tensor. The second tensor is input into the activation function, and the first weight corresponding to each channel in the first tensor is output. For each channel in the first tensor, the first tensor is multiplied by the first weight corresponding to that channel to obtain the third tensor; The third tensor is subjected to global average pooling to obtain a fourth tensor with 1 channel; the fourth tensor is input into the convolutional neural network of the detection model to obtain the second weight corresponding to each position on the fourth tensor; wherein, the convolutional neural network includes an activation function.

7. The liquid leakage detection method according to claim 1, characterized in that, The first color channel is a composite channel; the second color channel is the composite channel or a grayscale channel; the composite channel is a red channel, a green channel, and a blue channel.

8. The liquid leakage detection method according to any one of claims 1 to 7, characterized in that, The step of outputting the detection result is followed by the following step: Based on the confidence level corresponding to each liquid leakage area, target confidence levels with confidence levels greater than a first threshold are selected, and the liquid leakage areas corresponding to the target confidence levels are selected as candidate liquid leakage areas. Based on the calculated crossover ratio between any two candidate liquid leakage areas, when the crossover ratio is greater than a second threshold, the candidate liquid leakage areas corresponding to the crossover ratio are merged to obtain the target liquid leakage area.

9. The liquid leakage detection method according to claim 8, characterized in that, After obtaining the target liquid leakage area, the following steps are also included: The location information of the target liquid leakage area is sent to the monitoring terminal for liquid leakage early warning.

10. A liquid leak detection device, characterized in that, include: The acquisition module is used to acquire target video data; Compare the differences between any two target video frames in the target video data to obtain a difference image; The determination module is used to determine the input raw tensor based on the first color channel corresponding to the first frame image in the target video data and the second color channel corresponding to the difference image; The input / output module is used to input the original input tensor into the detection model and output the detection result; wherein, the detection model is built in the model building module based on historical video data, and the detection result includes the location information of the liquid leakage area and the confidence level corresponding to each liquid leakage area; The detection model includes a first unit, which includes a channel layer attention mechanism module, a first correction module, a non-local spatial attention mechanism module, and a second correction module. The process of inputting the original tensor into the detection model and outputting the detection result includes calculating the first weight, the second weight, and correcting the first tensor. The first tensor is input into the channel layer attention mechanism module through the input module. The first tensor is then globally pooled in the channel layer attention mechanism module to obtain the second tensor. In the channel layer attention mechanism module, the second tensor is input into the activation function to obtain the first weight corresponding to each channel; For each channel in the first tensor, the first correction module performs a product operation between the first tensor and the first weight corresponding to that channel to obtain the third tensor; The third tensor output by the first correction module is input into the nonlocal spatial attention mechanism module. In the nonlocal spatial attention mechanism module, global average pooling is performed on the third tensor, and the T channels contained in the third tensor are normalized to obtain the fourth tensor. In the nonlocal spatial attention mechanism module, the fourth tensor is input into the convolutional neural network to obtain the second weight corresponding to each position on the third tensor; The second correction module performs a dot product operation between the third tensor and the second weight corresponding to each position to obtain the fifth tensor. The fifth tensor obtained is used as the new first tensor. The steps are continued to perform global pooling on the first tensor to obtain the second tensor, until the preset number of times is met. The fifth tensor generated last time is used as the corrected target tensor. The detection result is determined based on the target tensor.

11. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores machine-readable instructions that are executable by a processor, which, when executed by the processor, implement the liquid leak detection method as described in any one of claims 1 to 9.

12. A liquid leak detection system, characterized in that, include: A video acquisition device is used to acquire target video data and transmit it to a liquid leak detection device. A liquid leak detection device, used to receive the target video data and implement the liquid leak detection method as described in any one of claims 1 to 9; The monitoring terminal is used to receive location information of the liquid leak area and issue early warnings.

13. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they implement the steps of the liquid leak detection method as described in any one of claims 1 to 9.

Images