Dripping liquid detection method and device, storage medium and electronic device

Abstract

Embodiments of the present application provide a kind of detection method, device, storage medium and electronic device of dripping liquid, wherein the method comprises: by target detection model, the dripping liquid in first to-be-processed picture is detected, and first target picture is obtained, wherein first target picture includes first detection frame, and first detection frame is used to identify the dripping liquid detected by target detection model;According to the size of the dripping liquid identified by first detection frame and the detection result of the dripping liquid in the adjacent frame picture of first to-be-processed picture, whether the dripping liquid detected is the dripping liquid misdetected by target detection model;In the case that the dripping liquid detected is not the dripping liquid misdetected by target detection model, the second detection frame having mapping relationship with first detection frame is marked in first to-be-processed picture.The dripping liquid detection accuracy rate is not high in the related art, and the problem is solved by the embodiments of the present application.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and more specifically, to a method, apparatus, storage medium, and electronic device for detecting leaking liquid. Background Technology

[0002] During the storage and transportation of liquids, leaks, drips, and spills can occur due to poor management and improper operation. If leaks are not addressed promptly, the scouring effect of the medium can cause them to expand rapidly, resulting in material loss, environmental damage, and, in the case of toxic, hazardous, flammable, or explosive media, potentially leading to serious accidents such as poisoning, fires, and explosions. Therefore, leak detection is crucial. Current technologies identify leaks directly from images, but this method is susceptible to interference from factors such as shadows and lighting, leading to misidentification and reducing the accuracy of leak detection. Thus, related technologies suffer from low accuracy in leak detection.

[0003] There is currently no effective solution to the problem of low accuracy in detecting dripping liquids in related technologies. Summary of the Invention

[0004] This invention provides a method, apparatus, storage medium, and electronic device for detecting dripping liquids, thereby at least solving the problem of low accuracy in detecting dripping liquids in related technologies.

[0005] According to one embodiment of the present invention, a method for detecting dripping liquid is provided, comprising:

[0006] The dripping liquid in the first image to be processed is detected by the target detection model to obtain the first target image. The first target image contains a first detection box, which is used to identify the dripping liquid detected by the target detection model.

[0007] Based on the size of the dripping liquid identified by the first detection box and the detection results of the dripping liquid in adjacent frames of the first image to be processed, it is determined whether the detected dripping liquid is a false detection by the target detection model.

[0008] If the detected dripping liquid is not a false detection by the target detection model, a second detection box with a mapping relationship to the first detection box is marked in the first image to be processed, wherein the second detection box is used to identify the dripping liquid in the first image to be processed.

[0009] According to another embodiment of the present invention, a dripping liquid detection device is also provided, comprising: a detection module, configured to detect dripping liquid in a first image to be processed by means of a target detection model to obtain a first target image, wherein the first target image includes a first detection frame, the first detection frame being used to identify the dripping liquid detected by the target detection model;

[0010] The determination module is used to determine whether the detected dripping liquid is a false detection by the target detection model based on the size of the dripping liquid identified by the first detection box and the detection results of the dripping liquid in adjacent frames of the first image to be processed.

[0011] The marking module is used to mark a second detection box in the first image to be processed that has a mapping relationship with the first detection box when the detected dripping liquid is not falsely detected by the target detection model. The second detection box is used to identify the dripping liquid in the first image to be processed.

[0012] According to yet another embodiment of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, wherein the computer program is configured to perform the steps in any of the above method embodiments when executed.

[0013] According to yet another embodiment of the present invention, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0014] This invention utilizes a target detection model to identify dripping liquid in an image to be processed. The model then verifies the detection results of the target detection model based on the size characteristics of the dripping liquid and the detection results of dripping liquid in adjacent frames of the first image to be processed. If the dripping liquid is not a false detection by the target detection model, it indicates that the image to be processed does indeed contain dripping liquid. A second detection box is then used to mark the dripping liquid at the corresponding position in the image to be processed, thereby reducing false identification of dripping liquid. Therefore, this invention solves the problem of low accuracy in dripping liquid detection in related technologies and achieves the effect of improving the accuracy of dripping liquid detection. Attached Figure Description

[0015] Figure 1 This is a block diagram of the mobile terminal hardware structure of the method for detecting dripping liquid according to an embodiment of the present invention;

[0016] Figure 2 This is a flowchart of a method for detecting dripping liquid according to an embodiment of the present invention;

[0017] Figure 3 This is a schematic diagram of a target detection model detecting dripping liquid according to an embodiment of the present invention;

[0018] Figure 4 This is a schematic diagram of the detection model training process according to an embodiment of the present invention;

[0019] Figure 5 This is a flowchart of a method for detecting dripping liquid according to a specific embodiment of the present invention. Detailed Implementation

[0020] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings and examples.

[0021] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0022] The methods and embodiments provided in this application can be executed on a mobile terminal, computer terminal, or similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a block diagram of the mobile terminal hardware structure of the method for detecting dripping liquid according to an embodiment of the present invention. Figure 1 As shown, a mobile terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0023] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the method for detecting leaking liquid in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0024] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the mobile terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0025] This embodiment provides a method for detecting dripping liquid. Figure 2 This is a flowchart of a method for detecting dripping liquid according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps:

[0026] Step S202: Detect the dripping liquid in the first image to be processed using a target detection model to obtain a first target image. The first target image contains a first detection box, which is used to identify the dripping liquid detected by the target detection model.

[0027] Step S204: Based on the size of the dripping liquid identified by the first detection box and the detection results of the dripping liquid in adjacent frames of the first image to be processed, determine whether the detected dripping liquid is a false detection by the target detection model.

[0028] Step S206: If the detected dripping liquid is not a false detection by the target detection model, a second detection box with a mapping relationship to the first detection box is marked in the first image to be processed, wherein the second detection box is used to identify the dripping liquid in the first image to be processed.

[0029] In this embodiment, when detecting liquid dripping in the first image to be processed, the first image to be processed is input into the target detection model. The target detection model detects the dripping liquid in the first image to be processed to obtain the first target image. The first target image is a mask image. The foreground area on the first image represents the area where the dripping liquid is detected in the first image to be processed by the target detection model, and the area where the dripping liquid is located is marked with a first detection box.

[0030] After obtaining the first target image, the accuracy of the leaking liquid detection by the target detection model is determined based on the size characteristics of the leaking liquid and the detection results of leaking liquid in adjacent frames of the first image to be processed. It is also determined whether the leaking liquid is detected accurately or falsely due to factors such as light or shadow. If the detected leaking liquid is not a false detection, a detection box with a mapping relationship to the first detection box is generated in the first image to be processed to mark the location of the leaking liquid. Generating a second detection box with a mapping relationship to the first detection box means that the position of the first detection box in the first target image corresponds to the position of the generated detection box in the image to be processed.

[0031] Through the above steps, the target detection model first identifies whether there is dripping liquid in the image to be processed. Then, the detection results of the target detection model are verified based on the size characteristics of the dripping liquid and the detection results of dripping liquid in adjacent frames of the first image to be processed. If the dripping liquid is not a false detection by the target detection model, it indicates that the image to be processed does indeed contain dripping liquid. The dripping liquid is then marked with a second detection box at the corresponding position in the image to be processed, thereby reducing the false identification of dripping liquid. Therefore, the problem of low accuracy in dripping liquid detection in related technologies is solved, and the effect of improving the accuracy of dripping liquid detection is achieved.

[0032] In an optional embodiment, the step of detecting dripping liquid in multiple frames of images to be processed using a target detection model includes: reconstructing the first image to be processed using a reconstruction sub-network in the target detection model to obtain a target reconstructed image, wherein the target reconstructed image is an image that does not contain dripping liquid; concatenating the image features of the first image to be processed and the image features of the target reconstructed image to obtain target connection features; performing a discrimination operation on the target connection features using a discriminant sub-network in the target detection model to obtain a target mask output image of the first image to be processed, wherein the target mask output image is used to identify the location of the dripping liquid contained in the first image to be processed; and performing a pooling operation on the target mask output image to obtain the first target image, wherein the first detection box is a detection box determined based on the location of the dripping liquid identified by the target mask output image.

[0033] In this embodiment, Figure 3 This is a schematic diagram of the target detection model detecting dripping liquid according to an embodiment of the present invention, as shown below. Figure 3 As shown, the above target detection model mainly consists of two parts: a reconstruction subnetwork and a discriminant subnetwork. The reconstruction subnetwork is mainly used to remove the dripping liquid from the first image to be processed, and reconstruct a target image without dripping liquid. After connecting the image features of the images before and after reconstruction (the first image to be processed and the target reconstructed image), the target connection features are obtained. The discriminant subnetwork determines the location of the dripping liquid in the first image to be processed by analyzing the target connection features, and outputs the target mask image. The target mask output image is then pooled to obtain a first target image of the same size as the input image to be processed, and the location of the dripping liquid is marked in the first target image using a first detection box.

[0034] In an optional embodiment, before performing the reconstruction operation on the first image to be processed through the reconstruction sub-network in the object detection model, the method further includes:

[0035] A training sample image set and a masked input image set are obtained. Each sample image in the training sample image set is an image that does not contain the leaking liquid. The masked input images in the masked input image set are used to synthesize an input image containing the leaking liquid with the sample images. The masked input images in the masked input image set are randomly generated. The sample images from the training sample image set and the masked input images from the masked input image set are used to synthesize an input image containing the leaking liquid, resulting in an input image set. The detection model to be trained is iteratively trained using the input image set to obtain the target detection model. During training, if the loss value output by the detection model to be trained is less than or equal to a preset threshold, training is terminated, and the detection model at the end of training is determined as the target detection model. If the loss value output by the detection model to be trained is greater than the preset threshold, the model parameters in the detection model to be trained are adjusted.

[0036] In this embodiment, before using the object detection model, it is necessary to train the object detection model first. First, the training samples used to train the object detection model are determined.

[0037] First, a training sample image set and a mask input image set are obtained. The sample images in the training sample image set are a large number of images that do not contain liquid dripping. The mask input images in the mask input image set are randomly generated during training. By combining the mask input images and the training sample images that do not contain liquid dripping, an image containing liquid dripping can be obtained.

[0038] The input image containing the dripping liquid is synthesized using sample images from the training sample image set and masked input images from the masked input image set. This input image set is then used to train the detection model to be trained, resulting in the object detection model.

[0039] Since the objective of the object detection model is to detect dripping liquid in images, it needs to be trained using images containing dripping liquid. However, the images in the training sample image set do not contain dripping liquid. Therefore, images containing dripping liquid are synthesized by combining images from a randomly generated mask input image set with images from the training sample image set. The mask input images are randomly generated, allowing for the synthesis of various images containing dripping liquid from the training sample images. Compared to directly collecting sample images containing dripping liquid, collecting sample images without dripping liquid is easier and yields more data, resulting in better training performance and solving the problem of not being able to collect a large number of dripping liquid images.

[0040] The detection model to be trained is iteratively trained using the input image set. During the iterative training process, the loss value of the detection model is calculated. If the loss value is greater than a preset threshold, the parameters of the detection model are adjusted. If the loss value is less than or equal to the preset threshold, the target detection model is obtained, the training ends and the target detection model is obtained.

[0041] In an optional embodiment, the step of iteratively training the detection model to be trained using the input image set to obtain the target detection model includes:

[0042] The detection model to be trained is trained in the following steps, where i is an integer greater than or equal to 1: The i-th input image used in the i-th training round is obtained, wherein the i-th input image is synthesized by combining the sample image used in the i-th training round from the training sample image set and the mask input image used in the i-th training round from the mask input image set; the i-th input image is reconstructed using the reconstruction sub-network in the detection model to be trained obtained in the (i-1)-th training round to obtain the i-th reconstructed image; the image features of the i-th reconstructed image and the image features of the sample image used in the i-th training round are concatenated to obtain the i-th connection feature; the i-th connection feature is discriminated using the discriminant sub-network in the detection model to be trained obtained in the (i-1)-th training round to obtain the i-th mask output image, wherein the i-th mask output image is used to identify the i-th... The input image contains the location of the leaking liquid; pooling is performed on the i-th masked output image to obtain the i-th target image, wherein the i-th target image includes the i-th detection box, which is determined based on the location of the leaking liquid identified by the i-th masked output image; the i-th loss value of the i-th training round is determined based on the masked input image used in the i-th training round, the i-th masked output image, the sample image used in the i-th training round, and the i-th reconstructed image; if the i-th loss value is less than or equal to the preset threshold, training ends, and the detection model to be trained obtained from the (i-1)-th training round is determined as the target detection model; if the i-th loss value is greater than the preset threshold, the model parameters in the detection model to be trained obtained from the (i-1)-th training round are adjusted to obtain the detection model to be trained from the i-th training round.

[0043] In this embodiment, taking the i-th training iteration as an example, Figure 4 This is a schematic diagram of the detection model training process according to an embodiment of the present invention, such as... Figure 4As shown, during the i-th training, the i-th input image, synthesized from the sample image used in the i-th round and the mask input image used in the i-th round, is input into the reconstruction sub-network of the detection model to be trained in the (i-1)-th round to obtain the i-th reconstructed image. Both the i-th reconstructed image and the sample image used in the i-th round are images that do not contain liquid dripping. The loss value between the i-th reconstructed image and the sample image used in the i-th round is calculated to represent the reconstruction effect of the reconstruction sub-network.

[0044] The image features of the i-th reconstructed image and the i-th input image are concatenated to obtain the i-th concatenation feature. The discriminant sub-network in the detection model to be trained in the (i-1)-th round performs a discrimination operation on the i-th concatenation result to determine the location of the dripping liquid in the i-th input image, and outputs it through the i-th masked output image. The discrimination effect of the discriminant sub-network is represented by calculating the loss value between the i-th masked output image and the masked input image used in the i-th round.

[0045] The i-th target mask image is pooled to obtain the i-th target image with the same size as the i-th input image.

[0046] After obtaining the i-th target image, the i-th loss value for the i-th training round is determined based on the mask input image used in the i-th training round, the i-th mask output image, the sample image used in the i-th training round, and the i-th reconstructed image. If the i-th loss value is less than or equal to a preset threshold, training ends, and the detection model to be trained obtained in the (i-1)-th training round is determined as the target detection model. If the i-th loss value is greater than the preset threshold, the model parameters in the detection model to be trained obtained in the (i-1)-th training round are adjusted to obtain the detection model to be trained in the i-th training round.

[0047] In an optional embodiment, determining the i-th loss value for the i-th training round based on the masked input image used in the i-th training round, the i-th masked output image, the sample image used in the i-th training round, and the i-th reconstructed image includes: inputting the i-th reconstructed image and the sample image used in the i-th training round into a first loss function to obtain a first loss value; inputting the i-th masked output image and the masked input image used in the i-th training round into a second loss function to obtain a second loss value; and determining the sum of the first loss value and the second loss value as the i-th loss value.

[0048] In this embodiment, the first loss function is the loss function used for reconstructing the sub-network, mainly the SSIM loss function, and the second loss function is the loss function used for discriminating the sub-network, mainly the Focal Loss loss function.

[0049] In an optional embodiment, determining whether the detected dripping liquid is a false detection by the target detection model based on the size of the dripping liquid identified by the first detection box and the detection results of dripping liquid in adjacent frames of the first image to be processed includes: determining whether the area of ​​the first detection box meets a preset condition, wherein the preset condition is greater than a preset minimum threshold and less than a preset maximum threshold; if the first detection box meets the preset condition, determining whether there is a detection box in the target detection box set that matches the first detection box, wherein the target detection box set includes detection boxes selected in the second target image whose area is greater than the preset minimum threshold and less than the preset maximum threshold, the second target image is obtained by detecting dripping liquid in the second image to be processed by the target detection model, and the second image to be processed is the previous frame or the next frame of the first image to be processed; if there is a detection box in the target detection box set that matches the first detection box, determining that the detected dripping liquid is not a false detection by the target detection model.

[0050] In this embodiment, when determining whether the target detection model has a false detection, the area of ​​the first detection box that identifies the dripping liquid is used to make a preliminary judgment on whether it is a false detection. If the area of ​​the first detection box is less than a preset minimum threshold or greater than a preset maximum threshold, it is determined that the first detection box is a false detection obtained by the target detection model.

[0051] If the area of ​​the first detection box is greater than a preset minimum threshold and less than a preset maximum threshold, the detection results of the dripping liquid in the second image to be processed in the previous or next frame of the image to be processed are further combined to determine whether the target detection model has a false detection. If there is a detection box in the target detection box set obtained from the second image to be processed that matches the first detection box, then the detected dripping liquid is not a false detection by the target detection model; otherwise, the first detection box is a false detection by the target detection model.

[0052] In an optional embodiment, the slope of the straight line between the target point of each detection box in the target detection box set and the target point of the first detection box is determined to obtain a first slope set; if there is a slope in the first slope set that is within a preset range, it is determined that there is a detection box in the target detection box set that matches the first detection box.

[0053] In this embodiment, the relative position between the first detection box and each detection box in the target detection box set is used to determine whether there is a detection box in the target detection box set that matches the first detection box. Each detection box in the target detection box set is connected to the target point on the first detection box, and the slope of each straight line is calculated to represent the movement trajectory of the detected dripping liquid. If the dripping liquid in the target set box with a slope within a preset range indicates that the dripping liquid follows the law of movement trajectory, such as falling vertically, then if there is a slope within the preset range in the first slope set, it is determined that there is a detection box in the target detection box set that matches the first detection box.

[0054] Obviously, the embodiments described above are only some embodiments of the present invention, and not all embodiments.

[0055] The present invention will be specifically described below with reference to embodiments:

[0056] Figure 5 This is a flowchart of a method for detecting leaking liquid according to a specific embodiment of the present invention, such as... Figure 5 As shown, it includes:

[0057] Step 501, Enter an image;

[0058] Step 502, Unsupervised Network Prediction. The wireless supervised network corresponds to the target detection model. The construction process of the wireless supervised network is mainly divided into two parts. The first part is building a reconstructive sub-network. The structure of this reconstructive sub-network is designed based on the encoder and decoder of DenseNet, reducing the number of network channels and the number of downsampling steps. The second part is building a discriminative sub-network. The structure of this discriminative sub-network is similar to that of the reconstructive sub-network, but the number of network channels and the number of downsampling steps are different to adapt to the discrimination task. An average pooling layer and a max pooling layer are added after the discriminative sub-network to fix the number of channels and dimensions of the output, completing the construction of the unsupervised network structure.

[0059] Step 503: Output the prediction results;

[0060] Step 505: Trajectory analysis of prediction results. The detection boxes suspected of containing water droplets obtained during the unsupervised network prediction process are subjected to trajectory analysis. The analysis mainly focuses on the fact that the trajectory of the liquid droplets should be a vertical fall after the liquid leak occurs. The analysis is supplemented by the constraints such as the size of the detection box (there are certain size limitations for water droplets).

[0061] Step 505: Output drip information.

[0062] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0063] This embodiment also provides a dripping liquid detection device. Figure 5 This is a structural block diagram of a dripping liquid detection device according to an embodiment of the present invention, as shown below. Figure 5 As shown, the device includes:

[0064] Detection module 602 is used to detect the dripping liquid in the first image to be processed by a target detection model to obtain a first target image, wherein the first target image contains a first detection box, and the first detection box is used to identify the dripping liquid detected by the target detection model;

[0065] The determination module 604 is used to determine whether the detected dripping liquid is a false detection by the target detection model based on the size of the dripping liquid identified by the first detection box and the detection results of the dripping liquid in adjacent frames of the first image to be processed.

[0066] The marking module 606 is used to mark a second detection box in the first image to be processed that has a mapping relationship with the first detection box when the detected dripping liquid is not falsely detected by the target detection model. The second detection box is used to identify the dripping liquid in the first image to be processed.

[0067] In an optional embodiment, the above-described apparatus is further configured to: reconstruct the first image to be processed using the reconstruction sub-network in the target detection model to obtain a target reconstructed image, wherein the target reconstructed image is an image that does not contain the dripping liquid; connect the image features of the first image to be processed and the image features of the target reconstructed image to obtain target connection features; perform a discrimination operation on the target connection features using the discriminant sub-network in the target detection model to obtain a target mask output image of the first image to be processed, wherein the target mask output image is used to identify the location of the dripping liquid contained in the first image to be processed; and perform a pooling operation on the target mask output image to obtain the first target image, wherein the first detection box is a detection box determined based on the location of the dripping liquid identified by the target mask output image.

[0068] In an optional embodiment, the above apparatus is further configured to: acquire a training sample image set and a masked input image set, wherein each sample image in the training sample image set is an image that does not contain the leaking liquid, and the masked input images in the masked input image set are used to synthesize an input image containing the leaking liquid with the sample images, and the masked input images in the masked input image set are randomly generated; synthesize an input image containing the leaking liquid using the sample images in the training sample image set and the masked input images in the masked input image set to obtain an input image set; iteratively train the detection model to be trained using the input image set to obtain the target detection model, wherein during the training process, if the loss value output by the detection model to be trained is less than or equal to a preset threshold, the training is terminated, and the detection model to be trained at the time of training termination is determined as the target detection model; if the loss value output by the detection model to be trained is greater than the preset threshold, the model parameters in the detection model to be trained are adjusted.

[0069] In an optional embodiment, the above-described apparatus is further configured to perform the following steps to train the detection model to be trained in the i-th round, where i is an integer greater than or equal to 1: obtaining the i-th input image used in the i-th round of training, wherein the i-th input image is an input image obtained by synthesizing the sample image used in the i-th round of training from the training sample image set and the mask input image used in the i-th round of training from the mask input image set; performing a reconstruction operation on the i-th input image through the reconstruction sub-network in the detection model to be trained obtained in the (i-1)-th round of training to obtain the i-th reconstructed image; connecting the image features of the i-th reconstructed image and the image features of the sample image used in the i-th round of training to obtain the i-th connection feature; and performing a discrimination operation on the i-th connection result through the discriminant sub-network in the detection model to be trained obtained in the (i-1)-th round of training to obtain the i-th mask output image, wherein the i-th mask output image... The image is used to identify the location of the leaking liquid in the i-th input image; a pooling operation is performed on the i-th masked output image to obtain the i-th target image, wherein the i-th target image includes an i-th detection box, which is determined based on the location of the leaking liquid identified by the i-th masked output image; the i-th loss value of the i-th training round is determined based on the masked input image used in the i-th training round, the i-th masked output image, the sample image used in the i-th training round, and the i-th reconstructed image; if the i-th loss value is less than or equal to the preset threshold, training ends, and the detection model to be trained obtained from the (i-1)-th training round is determined as the target detection model; if the i-th loss value is greater than the preset threshold, the model parameters in the detection model to be trained obtained from the (i-1)-th training round are adjusted to obtain the detection model to be trained from the i-th training round.

[0070] In an optional embodiment, the above-described apparatus is further configured to: input the i-th reconstructed image and the sample image used in the i-th training round into a first loss function to obtain a first loss value; input the i-th masked output image and the masked input image used in the i-th training round into a second loss function to obtain a second loss value; and determine the sum of the first loss value and the second loss value as the i-th loss value.

[0071] In an optional embodiment, the above-described apparatus is further configured to: determine whether the area of ​​the first detection box satisfies a preset condition, wherein the preset condition is greater than a preset minimum threshold and less than a preset maximum threshold; if the first detection box satisfies the preset condition, determine whether there is a detection box in the target detection box set that matches the first detection box, wherein the target detection box set includes detection boxes selected from the second target image whose area is greater than the preset minimum threshold and less than the preset maximum threshold, the second target image is obtained by detecting dripping liquid in the second image to be processed by the target detection model, and the second image to be processed is the previous frame or the next frame of the first image to be processed; if there is a detection box in the target detection box set that matches the first detection box, determine that the detected dripping liquid is not a false detection by the target detection model.

[0072] In an optional embodiment, the above-described apparatus is further configured to: determine the slope of the straight line between the target point of each detection box in the target detection box set and the target point of the first detection box, respectively, to obtain a first slope set; and if there is a slope in the first slope set that is within a preset range, determine that there is a detection box in the target detection box set that matches the first detection box.

[0073] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0074] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to perform the steps in any of the above method embodiments when executed.

[0075] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0076] Embodiments of the present invention also provide an electronic device including a memory and a processor, the memory storing a computer program and the processor being configured to run the computer program to perform the steps in any of the above method embodiments.

[0077] In one exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0078] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0079] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0080] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for detecting leaking liquid, characterized in that, include: The dripping liquid in the first image to be processed is detected by the target detection model to obtain the first target image. The first target image contains a first detection box, which is used to identify the dripping liquid detected by the target detection model. Based on the size of the dripping liquid identified by the first detection box and the detection results of the dripping liquid in adjacent frames of the first image to be processed, it is determined whether the detected dripping liquid is a false detection by the target detection model. If the detected dripping liquid is not a false detection by the target detection model, a second detection box with a mapping relationship to the first detection box is marked in the first image to be processed, wherein the second detection box is used to identify the dripping liquid in the first image to be processed; The method further includes, before detecting the dripping liquid in the first image to be processed using a target detection model to obtain the first target image: The detection model to be trained is trained in the following steps until the i-th loss value of the i-th training round is less than or equal to a preset threshold, then the training ends, and the detection model to be trained obtained in the (i-1)-th training round is determined as the target detection model, where i is an integer greater than or equal to 1: Obtain the i-th input image used in the i-th round of training, wherein the i-th input image is obtained by combining the sample image used in the i-th round of training from the training sample image set and the mask input image used in the i-th round of training from the mask input image set. Each sample image in the training sample image set is an image that does not contain the leaking liquid. The mask input images in the mask input image set are used to combine with the sample images to form an input image that contains the leaking liquid. The mask input images in the mask input image set are randomly generated. The reconstruction sub-network in the detection model to be trained, obtained from the (i-1)th round of training, is used to reconstruct the i-th input image to obtain the i-th reconstructed image. The image features of the i-th reconstructed image and the image features of the sample images used in the i-th round of training are concatenated to obtain the i-th concatenation feature; The discriminant subnetwork in the detection model to be trained, obtained through the (i-1)th round of training, performs a discrimination operation on the i-th connection feature to obtain the i-th mask output image, wherein the i-th mask output image is used to identify the location of the leaking liquid contained in the i-th input image; A pooling operation is performed on the i-th mask output image to obtain the i-th target image, wherein the i-th target image includes the i-th detection box, which is determined based on the location of the leaking liquid identified by the i-th mask output image; The i-th loss value is determined based on the masked input image used in the i-th training round, the i-th masked output image, the sample image used in the i-th training round, and the i-th reconstructed image; If the i-th loss value is greater than the preset threshold, adjust the model parameters in the detection model to be trained obtained in the (i-1)-th round of training to obtain the detection model to be trained in the i-th round of training.

2. The method according to claim 1, characterized in that, The detection of dripping liquid in multiple frames of images to be processed using a target detection model includes: The first image to be processed is reconstructed by the reconstruction sub-network in the target detection model to obtain a target reconstruction image, wherein the target reconstruction image is an image that does not contain the dripping liquid; The image features of the first image to be processed and the image features of the target reconstructed image are concatenated to obtain the target concatenation features; The target connection features are discriminated by the discriminant subnetwork in the target detection model to obtain the target mask output image of the first image to be processed, wherein the target mask output image is used to identify the location of the dripping liquid contained in the first image to be processed. The target mask output image is pooled to obtain the first target image, wherein the first detection box is determined based on the location of the dripping liquid as identified by the target mask output image.

3. The method according to claim 2, characterized in that, Before performing the reconstruction operation on the first image to be processed through the reconstruction sub-network in the object detection model, the method further includes: Obtain the training sample image set and the masked input image set; The input image set is obtained by combining the sample images in the training sample image set and the masked input image in the masked input image set to form an input image containing the dripping liquid. The input image set is used to iteratively train the detection model to be trained to obtain the target detection model. During the training process, if the loss value output by the detection model to be trained is less than or equal to a preset threshold, the training ends and the detection model to be trained at the end of the training is determined as the target detection model. If the loss value output by the detection model to be trained is greater than the preset threshold, the model parameters in the detection model to be trained are adjusted.

4. The method according to claim 1, characterized in that, The step of determining the i-th loss value for the i-th training round based on the masked input image used in the i-th training round, the i-th masked output image, the sample image used in the i-th training round, and the i-th reconstructed image includes: The i-th reconstructed image and the sample images used in the i-th round of training are input into the first loss function to obtain the first loss value; The i-th mask output image and the mask input image used in the i-th round of training are input into the second loss function to obtain the second loss value; The sum of the first loss value and the second loss value is determined as the i-th loss value.

5. The method according to claim 1, characterized in that, Based on the size of the dripping liquid identified by the first detection box and the detection results of dripping liquid in adjacent frames of the first image to be processed, it is determined whether the detected dripping liquid is a false detection by the target detection model, including: Determine whether the area of ​​the first detection box meets a preset condition, wherein the preset condition is greater than a preset minimum threshold and less than a preset maximum threshold; If the first detection box meets the preset conditions, it is determined whether there is a detection box in the target detection box set that matches the first detection box. The target detection box set includes detection boxes in the second target image whose area is greater than the preset minimum threshold and less than the preset maximum threshold. The second target image is obtained by detecting the dripping liquid in the second image to be processed by the target detection model. The second image to be processed is the previous frame or the next frame of the first image to be processed. If a detection box matching the first detection box exists in the target detection box set, it is determined that the detected dripping liquid is not a false detection by the target detection model.

6. The method according to claim 5, characterized in that, Determining whether a detection box matching the first detection box exists in the target detection box set includes: The slope of the line connecting the target point of each detection box in the target detection box set to the target point of the first detection box is determined to obtain the first slope set. If a slope within a preset range exists in the first slope set, it is determined that a detection box matching the first detection box exists in the target detection box set.

7. A device for detecting leaking liquid, characterized in that, include: The detection module is used to detect the dripping liquid in the first image to be processed by a target detection model to obtain a first target image, wherein the first target image contains a first detection box, and the first detection box is used to identify the dripping liquid detected by the target detection model; The determination module is used to determine whether the detected dripping liquid is a false detection by the target detection model based on the size of the dripping liquid identified by the first detection box and the detection results of the dripping liquid in adjacent frames of the first image to be processed. A marking module is used to mark a second detection box in the first image to be processed that has a mapping relationship with the first detection box when the detected dripping liquid is not falsely detected by the target detection model. The second detection box is used to identify the dripping liquid in the first image to be processed. The device is further configured to perform the following steps before detecting the leaking liquid in the first image to be processed by the target detection model to obtain the first target image: training the detection model to be trained for the i-th round until the i-th loss value of the i-th round of training is less than or equal to a preset threshold, ending the training, and determining the detection model to be trained obtained in the (i-1)-th round of training as the target detection model, where i is an integer greater than or equal to 1: obtaining the i-th input image used in the i-th round of training, wherein the i-th input image is an input image obtained by synthesizing the sample image used in the i-th round of training from the training sample image set and the mask input image used in the i-th round of training from the mask input image set, wherein each sample image in the training sample image set is an image that does not contain the leaking liquid, and the mask input image in the mask input image set is used to synthesize an input image containing the leaking liquid with the sample image, and the mask input image in the mask input image set is randomly generated; reconstructing the i-th input image through the reconstruction sub-network in the detection model to be trained obtained in the (i-1)-th round of training to obtain the i-th target image. Reconstruct the image; concatenate the image features of the i-th reconstructed image with the image features of the sample images used in the i-th training round to obtain the i-th connection feature; perform a discrimination operation on the i-th connection feature through the discriminant subnetwork in the detection model to be trained obtained in the (i-1)-th training round to obtain the i-th masked output image, wherein the i-th masked output image is used to identify the location of the leaking liquid contained in the i-th input image; perform a pooling operation on the i-th masked output image to obtain the i-th target image, wherein the i-th... Each target image includes an i-th detection box, which is determined based on the location of the leaking liquid as identified by the i-th masked output image. An i-th loss value is determined based on the masked input image used in the i-th training round, the i-th masked output image, the sample image used in the i-th training round, and the i-th reconstructed image. If the i-th loss value is greater than a preset threshold, the model parameters in the detection model to be trained obtained from the (i-1)-th training round are adjusted to obtain the detection model to be trained from the i-th training round.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the method described in any one of claims 1 to 6.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method described in any one of claims 1 to 6.

Images