A Method and System for Detecting Crude Oil Leakage in Complex Scenarios Based on Computer Vision

Abstract

The present invention relates to a method and system for detecting crude oil leakage in complex scenarios based on computer vision. The method includes obtaining original image data in various scenarios; extracting features from the obtained original image data based on a scene conversion model; generating scene conversion image data; constructing a training data set; constructing a crude oil leakage detection model; training the crude oil leakage detection model; and inputting the image data to be detected into the trained crude oil leakage detection model to obtain corresponding detection results. By adding a scene conversion module in the data processing process, the present invention can convert the image data in sunny scenarios into different complex scenarios, generate scene conversion feature information in different complex scenarios, achieve the purpose of expanding the amount of model training data, and at the same time, can accurately identify the location of crude oil leakage in video surveillance images in complex scenarios, with a relatively high recognition accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of crude oil leakage detection, and particularly to a method and system for detecting crude oil leakage in complex scenarios based on computer vision. Background Art

[0002] In the technical field of using computer images to detect crude oil leakage, a trained deep neural network model can accurately detect objects in images. However, most of the existing crude oil leakage detection methods mainly detect single scenarios (such as sunny days), and the models used for detection are also designed for such single scenarios. Although the existing crude oil leakage detection methods have good detection effects for single scenarios, when the scenario changes, due to the lack of data for these scenarios during model training, the generalization ability and detection accuracy of the model will be poor.

[0003] In addition, due to the difficulty of data collection in complex scenarios (such as cloudy days, rainy days, haze, snowy days, etc.), therefore, how to convert the existing image data in sunny scenarios into image data in different complex scenarios to expand the data volume for model training, and how to improve the generalization ability of the model are technical problems that need to be solved urgently.

[0004] Therefore, the present invention provides a method and system for detecting crude oil leakage in complex scenarios based on computer vision. Summary of the Invention

[0005] Based on this, it is necessary to provide a method and system for detecting crude oil leakage in complex scenarios based on computer vision for the above technical problems.

[0006] To achieve the above object, the technical solution of the present invention is as follows.

[0007] According to one aspect of the present invention, a method for detecting crude oil leakage in complex scenarios based on computer vision is provided, including: acquiring original image data in multiple scenarios, where the multiple scenarios include sunny scenarios and complex scenarios; extracting features from the acquired original image data based on a scene conversion model to generate scene feature information in sunny scenarios and scene feature information in complex scenarios; based on a scene conversion module, converting the scene feature information in sunny scenarios to different complex scenarios to generate scene conversion feature information in different complex scenarios, and generating scene conversion image data based on the scene conversion feature information; optimizing the generated scene conversion image data using an adversarial loss function, and constructing a training data set based on the original image data in sunny scenarios and the scene conversion image data; constructing a crude oil leakage detection model based on a lightweight mixture of experts network, a feature extraction network, a multi-scale feature fusion network, and a detection network; inputting the training data set into the crude oil leakage detection model for training to obtain a trained crude oil leakage detection model; and inputting the image data to be detected into the trained crude oil leakage detection model to obtain corresponding detection results.

[0008] Optionally, the complex scenarios include cloudy scenarios, rainy scenarios, haze scenarios, and snowy scenarios.

[0009] Optionally, the extracting features from the acquired original image data based on a scene conversion model to generate scene feature information in sunny scenarios and scene feature information in complex scenarios is specifically calculated as follows.

[0010] (1)

[0011] (?)

[0012] In formulas (1)-(2): represents the original image data in sunny scenarios, represents the original image data in complex scenarios, represents the encoder in sunny scenarios, represents the encoder in complex scenarios, represents the scene feature information in sunny scenarios, represents the scene feature information in complex scenarios.

[0013] Optionally, the converting the scene feature information in sunny scenarios to different complex scenarios based on a scene conversion module to generate scene conversion feature information in different complex scenarios is specifically calculated as follows.

[0014] (3)

[0015] In formula (3): represents the scene conversion module, Represents scene transition feature information.

[0016] Based on the scene transition feature information, scene transition image data is generated, and the specific calculation formula is as follows.

[0017] (4)

[0018] In formula (4): Represents an image generator, Represents the scene transition image data.

[0019] Optionally, an adversarial loss function is used to optimize the generated scene transition image data, and the specific calculation formula is as follows.

[0020] (5)

[0021] (6)

[0022] (7)

[0023] (8)

[0024] In formulas (5)-(8): Represents the loss function for optimizing the generator, Represents the discriminator, Represents the mathematical expectation, Represents the Perceptual Loss function, Represents the number of feature layers of the pre-trained VGG19, Represents the feature layer of the pre-trained VGG19, Represents the Sytle Loss function, Represents the function for calculating the mean, Represents the function for calculating the standard deviation, and Represents in the loss function and The weight parameters.

[0025] Optionally, inputting the training data set into the crude oil leakage detection model for training to obtain the trained crude oil leakage detection model specifically includes: inputting the training data set into the lightweight mixture of experts network for image enhancement processing to generate enhanced feature information under multiple scenarios, and performing fusion processing on the enhanced feature information under multiple scenarios to generate fused enhanced feature information; inputting the fused enhanced feature information into the feature extraction network for feature extraction to extract feature maps of multiple different scales; inputting the feature maps of multiple different scales into the multi-scale feature fusion network for feature enhancement to generate multi-scale fused feature maps; inputting the multi-scale fused feature maps into the detection network for detection processing to generate corresponding detection results; using and the binary cross-entropy loss function to optimize the crude oil leakage detection model until the crude oil leakage detection model converges to obtain the trained crude oil leakage detection model.

[0026] Optionally, inputting the training data set into the lightweight mixture of experts network for image enhancement processing to generate enhanced feature information under multiple scenarios, and the specific calculation formula is as follows.

[0027] (9)

[0028] (10)

[0029] (11)

[0030] (12)

[0031] (13)

[0032] In formulas (9)-(13): represents the training data set, represents the shallow feature extraction network, represents the shallow feature map, represents the feature enhancement network under cloudy days, represents the feature enhancement network under rainy days, represents the feature enhancement network under hazy days, represents the feature enhancement network under snowy days, represents the enhanced feature information under cloudy days, represents the enhanced feature information under rainy days, represents the enhanced feature information under hazy days, represents the enhanced feature information under snowy days;

[0033] Fuse the enhanced feature information in multiple scenarios to generate fused enhanced feature information. The specific calculation formula is as follows.

[0034] (14)

[0035] In formula (14): represents the adaptive fusion module, represents the fused enhanced feature information;

[0036] Input the fused enhanced feature information into the feature extraction network for feature extraction, and extract multiple feature maps of different scales. The specific calculation formula is as follows.

[0037] (15)

[0038] In formula (15): represents the feature extraction network, , and respectively represent multiple feature maps of different scales. Among them, has the same scale as the original image data, has a scale that is half of the original image data scale, has a scale that is one-fourth of the original image data scale.

[0039] Input the multiple feature maps of different scales into the multi-scale feature fusion network for feature enhancement to generate multiple scale fusion feature maps. The specific calculation formula is as follows.

[0040] (16)

[0041] In formula (16): represents the multi-scale feature fusion network, , and respectively represent the scale fusion feature maps generated by using the multi-scale feature fusion network to fuse , and after fusion processing.

[0042] Input the multiple scale fusion feature maps into the detection network for detection processing to generate the corresponding detection results. The specific calculation formula is as follows.

[0043] (17)

[0044] In formula (17): represents the detection network, , and respectively represent the detection results corresponding to the detections of , and by the detection network. Among them, the detection results include the upper left corner coordinate data of the object to be detected in the image data, the width data of the object to be detected, the height data of the object to be detected, and the category information of the object to be detected. Based on the upper left corner coordinate data, width data, and height data, the predicted rectangular detection frame corresponding to the object to be detected is obtained.

[0045] Use and the binary cross-entropy loss function to optimize the crude oil leakage detection model until the crude oil leakage detection model converges, and the trained crude oil leakage detection model is obtained. The specific calculation formula is as follows.

[0046] (18)

[0047] (19)

[0048] (20)

[0049] (21)

[0050] (22)

[0051] (23)

[0052] In formulas (18)-(23): represents the predicted rectangular detection frame of the object to be detected predicted by the detection network, represents the true rectangular detection frame of the object to be detected, represents the calculation result of the degree of overlap between the predicted rectangular detection frame and the true rectangular detection frame by calculating the intersection of the predicted rectangular detection frame and the true rectangular detection frame divided by their union. c represents the diagonal length data of the minimum circumscribed rectangle of the predicted rectangular detection frame and the true rectangular detection frame, v represents the similarity measurement result of the aspect ratio, represents the width data of the true rectangular detection frame, represents the height data of the true rectangular detection frame, w represents the width data of the predicted rectangular detection frame, h represents the height data of the predicted rectangular detection frame, represents the calculation result of the distance between the centers of the measured rectangular detection frame and the true rectangular detection frame, represents the weight coefficient, represents the predicted category value of the object to be detected, represents the true category value of the object to be detected, n represents the number of samples, represents the Sigmoid function, represents the total loss function, represents the CIoU Loss , and both represent the binary cross - entropy loss function of BCE Loss. Among them, is used to evaluate the accuracy of the crude oil leakage detection model in predicting the existence of the target, is used to evaluate the accuracy of the crude oil leakage detection model in predicting the classification of the target category, , and respectively represent , and the corresponding weight coefficients.

[0053] According to the second aspect of the present invention, there is provided a crude oil leakage detection system under complex scenarios based on computer vision, including: an image acquisition module for acquiring original image data under multiple scenarios, where the multiple scenarios include sunny scenarios and complex scenarios; a scene feature information extraction module for extracting features from the acquired original image data based on a scene conversion model to generate scene feature information under sunny scenarios and scene feature information under complex scenarios; a scene conversion module for converting the scene feature information under sunny scenarios into different complex scenarios based on the scene conversion module to generate scene conversion feature information under different complex scenarios, and generating scene conversion image data based on the scene conversion feature information; a training dataset construction module for optimizing the generated scene conversion image data using an adversarial loss function and constructing a training dataset based on the original image data under sunny scenarios and the scene conversion image data; a crude oil leakage detection model construction module for constructing a crude oil leakage detection model based on a lightweight mixture of experts network, a feature extraction network, a multi - scale feature fusion network, and a detection network; a crude oil leakage detection model training module for inputting the training dataset into the crude oil leakage detection model for training to obtain a trained crude oil leakage detection model; a detection module for inputting the image data to be detected into the trained crude oil leakage detection model to obtain a corresponding detection result.

[0054] According to the third aspect of the present invention, there is provided an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above - mentioned method are implemented.

[0055] According to the fourth aspect of the present invention, there is provided a computer - readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps of the above - mentioned method are implemented.

[0056] The advantages and beneficial effects of the present invention are as follows: A method and system for detecting crude oil leakage in complex scenarios based on computer vision provided by the present invention can convert image data in sunny scenarios into different complex scenarios during the data processing process, generate scene conversion feature information in different complex scenarios, and achieve the purpose of expanding the amount of model training data. At the same time, by adding a lightweight mixture of experts network during the construction of the crude oil leakage detection model, lightweight feature enhancement processing can be performed on image data in different scenarios respectively, effectively enhancing the generalization ability of the crude oil leakage detection model. In addition, compared with existing crude oil leakage detection methods, this method can accurately identify the location of crude oil leakage in video surveillance images in complex scenarios, with a relatively high recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 It is a flowchart of a method for detecting crude oil leakage in complex scenarios based on computer vision in an embodiment of the present invention.

[0058] Figure 2 It is a schematic diagram of the result of a system for detecting crude oil leakage in complex scenarios based on computer vision in an embodiment of the present invention.

[0059] Figure 3 It is a schematic diagram of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0060] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below through specific embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0061] Embodiment 1

[0062] Refer to the attached Figure 1 , a method for detecting crude oil leakage in complex scenarios based on computer vision, the method includes the following steps.

[0063] S1. Obtain original image data in multiple scenarios, where the multiple scenarios include sunny scenarios and complex scenarios.

[0064] In this embodiment, the complex scenarios include but are not limited to cloudy scenarios, rainy scenarios, haze scenarios, snowy scenarios, etc. It should be understood that the image data in different scenarios in the original image data do not need to be paired.

[0065] S2. Based on the scene conversion model, perform feature extraction on the obtained original image data to generate scene feature information in sunny scenarios and scene feature information in complex scenarios.

[0066] In this embodiment, a scene conversion model is used to extract features from the original image data in multiple scenarios obtained, generating scene feature information in sunny scenarios and scene feature information in complex scenarios. The specific calculation formulas are as follows.

[0067] (1)

[0068] (2)

[0069] In formulas (1)-(2): represents the original image data in sunny scenarios, represents the original image data in complex scenarios, represents the encoder in sunny scenarios, represents the encoder in complex scenarios, represents the scene feature information in sunny scenarios, represents the scene feature information in complex scenarios.

[0070] S3. Based on the scene conversion module, convert the scene feature information in sunny scenarios to different complex scenarios, generating scene conversion feature information in different complex scenarios, and generate scene conversion image data based on the scene conversion feature information.

[0071] In this embodiment, the scene conversion module is used to ensure that the scene feature information is not lost, and convert the scene feature information in to

[0072] to generate the converted scene conversion feature information, and generate a piece of scene conversion image data after scene conversion based on the converted scene conversion feature information. Furthermore, the

[0073] scene feature information (i.e., the content information in the image data) in

[0074] (3)

[0075] In formula (3): represents the scene conversion module, represents the scene conversion feature information. Based on the scene conversion feature information, generate scene conversion image data. The specific calculation formula is as follows.

[0076] (4)

[0077] In formula (4): represents an image generator represents the scene conversion image data

[0078] Furthermore, the scene feature information contained in the scene conversion image data is the same as the scene feature information contained in the original image data under sunny scenes. The sunny scene is converted to other complex scenes, such as: converting the sunny scene to a cloudy scene, converting the sunny scene to a rainy scene, etc.

[0079] S4. Optimize the generated scene conversion image data using an adversarial loss function, and construct a training dataset based on the original image data and the scene conversion image data under sunny scenes

[0080] In this embodiment, the generated scene conversion image data is optimized using an adversarial loss function, and the specific calculation formula is as follows

[0081] (5)

[0082] (6)

[0083] (7)

[0084] (8)

[0085] In formulas (5)-(8): represents the loss function for optimizing the generator represents the discriminator represents the mathematical expectation represents the Perceptual Loss perceptual loss function represents the number of feature layers of the pre-trained VGG19 represents the feature layer of the pre-trained VGG19 represents the Sytle Loss style loss function represents the function for calculating the mean represents the function for calculating the standard deviation and represent in the loss function and are the weight parameters

[0086] Furthermore, use the original image data under sunny scenes and the scene conversion image data obtained through the scene conversion process to construct a training dataset for training the crude oil leakage detection model

[0087] S5. Construct a crude oil leakage detection model based on a lightweight mixture of experts network, a feature extraction network, a multi-scale feature fusion network, and a detection network.

[0088] In this embodiment, the crude oil leakage detection model includes a lightweight mixture of experts network, a feature extraction network, a multi-scale feature fusion network, and a detection network.

[0089] S6. Input the training data set into the crude oil leakage detection model for training to obtain a trained crude oil leakage detection model.

[0090] In this embodiment, inputting the training data set into the crude oil leakage detection model for training to obtain a trained crude oil leakage detection model specifically includes the following steps.

[0091] S61. Input the training data set into the lightweight mixture of experts network for image enhancement processing to generate enhanced feature information in multiple scenarios, and perform fusion processing on the enhanced feature information in multiple scenarios to generate fused enhanced feature information.

[0092] Further, the calculation formula of step S61 is as follows.

[0093] (9)

[0094] (10)

[0095] (11)

[0096] (12)

[0097] (13)

[0098] (14)

[0099] In formulas (9)-(14): represents the training data set, represents the shallow feature extraction network, represents the shallow feature map, represents the feature enhancement network in the cloudy scenario, represents the feature enhancement network in the rainy scenario, represents the feature enhancement network in the haze scenario, represents the feature enhancement network in the snowy scenario, represents the enhanced feature information in the cloudy scenario, represents the enhanced feature information in the rainy scenario, represents the enhanced feature information in the haze scenario, Represents enhanced feature information in a snowy scene, Represents an adaptive fusion module, Represents the fused enhanced feature information.

[0100] S62. Input the fused enhanced feature information into a feature extraction network for feature extraction to extract multiple feature maps of different scales.

[0101] Furthermore, the calculation formula for step S62 is as follows.

[0102] (15)

[0103] In formula (15): Represents the feature extraction network, 、 and respectively represent multiple feature maps of different scales, where The scale of is the same as the scale of the original image data, The scale of is one-half of the scale of the original image data, The scale of is one-quarter of the scale of the original image data.

[0104] S63. Input the multiple feature maps of different scales into a multi-scale feature fusion network for feature enhancement to generate multiple scale-fused feature maps.

[0105] Furthermore, the calculation formula for step S63 is as follows.

[0106] (16)

[0107] In formula (16): Represents the multi-scale feature fusion network, 、 and respectively represent the scale-fused feature maps generated corresponding to the fusion processing of 、 and using the multi-scale feature fusion network.

[0108] S64. Input the multiple scale-fused feature maps into a detection network for detection processing to generate corresponding detection results.

[0109] Furthermore, the calculation formula for step S64 is as follows.

[0110] <� (17)

[0111] In formula (17): Represents the detection network, 、 and respectively represent the detection results generated after the detection network performs detection processing on , and . Among them, the detection results include the upper left corner coordinate data of the object to be detected in the image data, the width data of the object to be detected, the height data of the object to be detected, and the category information of the object to be detected. Based on the upper left corner coordinate data, width data, and height data, a predicted rectangular detection frame corresponding to the object to be detected is obtained.

[0112] S65. Use and the binary cross-entropy loss function to optimize the crude oil leakage detection model until the crude oil leakage detection model converges, and obtain the trained crude oil leakage detection model.

[0113] Furthermore, the calculation formula of step S65 is as follows.

[0114] (18)

[0115] (19)

[0116] (20)

[0117] (21)

[0118] (22)

[0119] (23)

[0120] In formulas (18)-(23): represents the predicted rectangular detection frame of the object to be detected predicted by the detection network, represents the true rectangular detection frame of the object to be detected, represents the calculation result of the degree of overlap between the predicted rectangular detection frame and the true rectangular detection frame by calculating the intersection of the predicted rectangular detection frame and the true rectangular detection frame divided by their union. c represents the diagonal length data of the minimum circumscribed rectangle of the predicted rectangular detection frame and the true rectangular detection frame, and v represents the similarity measurement result of the aspect ratio, represents the width data of the true rectangular detection frame, represents the height data of the true rectangular detection frame, w represents the width data of the predicted rectangular detection frame, and h represents the height data of the predicted rectangular detection frame, represents the calculation result of the distance between the centers of the measured rectangular detection frame and the true rectangular detection frame, represents the weight coefficient, represents the predicted category value of the object to be detected, represents the true class value of the object to be detected, and n represents the number of samples. represents the Sigmoid function. represents the total loss function. represents CIoU Loss , and both represent the binary cross-entropy loss function of BCE Loss. Among them, is used to evaluate the accuracy of the crude oil leakage detection model in predicting the existence of the target. is used to evaluate the accuracy of the crude oil leakage detection model in predicting the classification of the target category. , and respectively represent , and the corresponding weight coefficients.

[0121] Furthermore, represents the calculation of dividing the intersection of the predicted rectangular detection frame and the true rectangular detection frame by their union to obtain the calculation result of the coincidence degree between the predicted rectangular detection frame and the true rectangular detection frame. It should be understood that the higher the calculation result of IoU, the higher the coincidence degree between the predicted rectangular detection frame and the true rectangular detection frame, that is, the more accurate the prediction effect of the crude oil leakage detection model. On the contrary, the lower the calculation result of IoU, the lower the coincidence degree between the predicted rectangular detection frame and the true rectangular detection frame, that is, the worse the prediction effect of the crude oil leakage detection model.

[0122] Furthermore, is used to evaluate the accuracy of the crude oil leakage detection model in predicting the existence of the target (i.e., whether there is crude oil leakage). It mainly focuses on the positioning ability of the crude oil leakage detection model for objects in the image and the detection ability of the existence of the target. While is used to evaluate the accuracy of the crude oil leakage detection model in predicting the classification of the target category. It mainly focuses on the recognition ability of the crude oil leakage detection model for the target category.

[0123] S7. Input the image data to be detected into the trained crude oil leakage detection model to obtain the corresponding detection results.

[0124] In summary, the present invention proposes a method for detecting crude oil leakage in complex scenarios based on computer vision. The method uses a scene conversion module that can convert image data in sunny scenarios to different complex scenarios, generating scene conversion feature information in different complex scenarios. Based on the scene conversion feature information, scene conversion image data is generated, and a training dataset for training a crude oil leakage detection model is constructed using the original image data in sunny scenarios and the generated scene conversion image data after conversion. A crude oil leakage detection model is constructed for this training dataset. The crude oil leakage detection model includes a lightweight mixture of experts network, a feature extraction network, a multi-scale feature fusion network, and a detection network. After the crude oil leakage detection model is trained, the trained crude oil leakage detection model is deployed in combination with video surveillance, which can be used to detect crude oil leakage in real time, and then an accurate recognition result of the crude oil leakage location in the video surveillance image can be obtained.

[0125] Embodiment 2

[0126] Based on the above Embodiment 1, this embodiment provides a system 200 for detecting crude oil leakage in complex scenarios based on computer vision. Please refer to Figure 2 and is used to implement the steps of the method for detecting crude oil leakage in complex scenarios based on computer vision described in the above Embodiment 1. The system 200 mainly includes: an image acquisition module 210, a scene feature information extraction module 220, a scene conversion module 230, a training dataset construction module 240, a crude oil leakage detection model construction module 250, a crude oil leakage detection model training module 260, and a detection module 270.

[0127] The image acquisition module 210 is used to acquire original image data in multiple scenarios, and the multiple scenarios include sunny scenarios and complex scenarios.

[0128] The scene feature information extraction module 220 is used to extract features from the acquired original image data based on the scene conversion model, generating scene feature information in sunny scenarios and scene feature information in complex scenarios.

[0129] The scene conversion module 230 is used to, based on the scene conversion module, convert the scene feature information in sunny scenarios to different complex scenarios, generating scene conversion feature information in different complex scenarios, and generating scene conversion image data based on the scene conversion feature information.

[0130] The training dataset construction module 240 is used to optimize the generated scene conversion image data using an adversarial loss function, and construct a training dataset based on the original image data in sunny scenarios and the scene conversion image data.

[0131] The crude oil leakage detection model construction module 250 is used to construct a crude oil leakage detection model based on a lightweight hybrid expert network, a feature extraction network, a multi-scale feature fusion network, and a detection network.

[0132] The crude oil leakage detection model training module 260 is used to input a training data set into the crude oil leakage detection model for training to obtain a trained crude oil leakage detection model.

[0133] The detection module 270 is used to input the image data to be detected into the trained crude oil leakage detection model to obtain a corresponding detection result.

[0134] Embodiment III

[0135] Based on the above Embodiment I, this embodiment further provides an electronic device. Please refer to the attached Figure 3 , Figure 3 The electronic device shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present disclosure.

[0136] As Figure 3 shown, the electronic device may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other through a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0137] Generally, the following devices may be connected to the I / O interface 305: an input device 306 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, etc., an output device 307 including, for example, a liquid crystal display (LCD), a speaker, etc., a storage device 308 including, for example, a magnetic tape, a hard disk, etc., and a communication device 309. The communication device 309 can allow the electronic device to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 3 shows an electronic device having various devices, it should be understood that it is not required to implement or have all the shown devices. Instead, more or fewer devices may be implemented or had. Figure 3 Each block shown in

[0138] In particular, according to some embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product that includes a computer program carried on a computer-readable medium, and the computer program includes program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 309, or installed from a storage device 308, or installed from a ROM 302. When the computer program is executed by a processing device 301, the above-mentioned functions defined in the methods of some embodiments of the present disclosure are performed.

[0139] Embodiment 4

[0140] Based on the above Embodiment 1, this embodiment further provides a computer-readable storage medium that stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0141] It should be noted that the computer-readable medium in some embodiments of the present disclosure can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, a computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, and the computer-readable signal medium can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0142] In this embodiment, the client and the server can communicate using any currently known or future-developed network protocol such as HTTP (HyperText Transfer Protocol), and can be interconnected with digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed network.

[0143] The above computer-readable medium can be included in the above device or exist separately without being assembled into the electronic device. The above computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device is caused to: obtain training data, convert the training data to obtain initial data; determine an initial rule base based on the initial data, and optimize the parameters of the initial rule base to obtain a target rule base; calculate activation weights for the rules in the target rule base according to a preset activation weight calculation formula; and determine abnormal information according to the test data and the activation weights.

[0144] Computer program code for performing the operations of some embodiments of the present disclosure can be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., by using an Internet service provider to connect through the Internet).

[0145] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0146] The units described in some embodiments of the present disclosure can be implemented in software or in hardware. The described units can also be provided in a processor. For example, it can be described as: a processor includes a data acquisition unit, a rule determination unit, a weight calculation unit, and an anomaly determination unit. Among them, the names of these units do not constitute a limitation on the unit itself in some cases. For example, the data acquisition unit can also be described as "the unit for acquiring training data".

[0147] The functions described above herein can be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that can be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), etc.

[0148] Obviously, those skilled in the art should understand that the above-described implementation manners of the steps of the present invention can be executed in a manner different from that of the present invention, and the simulation methods and experimental devices include but are not limited to the above description. The above-described steps of the present invention can be executed in a different order than that here in some cases, and the steps shown or described above can be executed separately. Therefore, the present invention is not limited to any specific combination of hardware and software.

[0149] The above content is a further detailed description of the present invention in combination with specific implementation manners, and it cannot be determined that the specific implementation of the present invention is only limited to these descriptions. For those of ordinary skill in the technical field to which the present invention pertains, without departing from the concept of the present invention, several simple deductions or substitutions can still be made, and all should be regarded as belonging to the protection scope of the present invention.

Claims

1. A method for detecting crude oil leakage in complex scenarios based on computer vision, characterized in that, Including: Obtain original image data in multiple scenarios, where the multiple scenarios include sunny scenarios and complex scenarios; Based on the scene conversion model, perform feature extraction on the obtained original image data to generate scene feature information in sunny scenarios and scene feature information in complex scenarios; Based on the scene conversion module, convert the scene feature information in sunny scenarios to different complex scenarios to generate scene conversion feature information in different complex scenarios, and based on the scene conversion feature information, generate scene conversion image data; Optimize the generated scene conversion image data using an adversarial loss function, and construct a training dataset based on the original image data and the scene conversion image data in sunny scenarios; Based on a lightweight mixture of experts network, a feature extraction network, a multi-scale feature fusion network, and a detection network, construct a crude oil leakage detection model; Input the training dataset into the crude oil leakage detection model for training to obtain the trained crude oil leakage detection model; Input the image data to be detected into the trained crude oil leakage detection model to obtain the corresponding detection result; The complex scenarios include cloudy scenarios, rainy scenarios, haze scenarios, and snowy scenarios; The step of inputting the training dataset into the crude oil leakage detection model for training to obtain the trained crude oil leakage detection model specifically includes: Input the training dataset into the lightweight mixture of experts network for image enhancement processing to generate enhanced feature information in multiple scenarios, and perform fusion processing on the enhanced feature information in multiple scenarios to generate fused enhanced feature information; input the fused enhanced feature information into the feature extraction network for feature extraction to extract feature maps of multiple different scales; input the feature maps of multiple different scales into the multi-scale feature fusion network for feature enhancement to generate multi-scale fused feature maps; input the multi-scale fused feature maps into the detection network for detection processing to generate the corresponding detection results; use the object detection regression loss function and the binary cross-entropy loss function to optimize the crude oil leakage detection model until the crude oil leakage detection model converges to obtain the trained crude oil leakage detection model; The step of inputting the training dataset into the lightweight mixture of experts network for image enhancement processing to generate enhanced feature information in multiple scenarios, and the specific calculation formula is as follows: （9） （10） （11） （12） （13） In formulas (9)-(13): represents the training data set, represents the shallow feature extraction network, represents the shallow feature map, represents the feature enhancement network under cloudy sky scenes, represents the feature enhancement network under rainy sky scenes, represents the feature enhancement network under haze scenes, represents the feature enhancement network under snowy sky scenes, represents the enhanced feature information under cloudy sky scenes, represents the enhanced feature information under rainy sky scenes, represents the enhanced feature information under haze scenes, represents the enhanced feature information under snowy sky scenes; The step of performing fusion processing on the enhanced feature information in multiple scenarios to generate fused enhanced feature information, and the specific calculation formula is as follows: （14） In formula (14): represents the adaptive fusion module, represents the fused enhanced feature information; The step of inputting the fused enhanced feature information into the feature extraction network for feature extraction to extract feature maps of multiple different scales, and the specific calculation formula is as follows: （15） In formula (15): represents the feature extraction network, , and respectively represent feature maps of multiple different scales, where has the same scale as the original image data, has a scale that is one-half of the original image data scale, has a scale that is one-fourth of the original image data scale; The step of inputting the feature maps of multiple different scales into the multi-scale feature fusion network for feature enhancement to generate multi-scale fused feature maps, and the specific calculation formula is as follows: （16） In formula (16): represents a multi-scale feature fusion network,[[]] , and respectively represent the scale fusion feature maps generated corresponding to the fusion processing of , and after fusion processing; The step of inputting the multi-scale fused feature maps into the detection network for detection processing to generate the corresponding detection results, and the specific calculation formula is as follows: （17） In formula (17): represents the detection network, , and respectively represent the detection results generated after using the detection network to perform detection processing on , and Among them, the detection results include the upper left corner coordinate data of the object to be detected in the image data, the width data of the object to be detected, the height data of the object to be detected, and the category information of the object to be detected. Based on the upper left corner coordinate data, width data, and height data, a predicted rectangular detection frame corresponding to the object to be detected is obtained; The step of using the object detection regression loss function and the binary cross-entropy loss function to optimize the crude oil leakage detection model until the crude oil leakage detection model converges to obtain the trained crude oil leakage detection model, and the specific calculation formula is as follows: （18） （19） （20） （21） （22） （23） In formulas (18)-(23): represents the predicted rectangular detection box of the object to be detected predicted by the detection network, represents the ground truth rectangular detection box of the object to be detected, represents the calculation result of the overlap degree between the predicted rectangular detection box and the ground truth rectangular detection box by dividing the intersection of the predicted rectangular detection box and the ground truth rectangular detection box by their union. c represents the length data of the diagonal of the minimum bounding rectangle of the predicted rectangular detection box and the ground truth rectangular detection box, and v represents the similarity measurement result of the aspect ratio, represents the width data of the ground truth rectangular detection box, represents the height data of the ground truth rectangular detection box, w represents the width data of the predicted rectangular detection box, and h represents the height data of the predicted rectangular detection box, represents the calculation result of the distance between the centers of the measured rectangular detection box and the ground truth rectangular detection box, represents the weight coefficient, represents the predicted class value of the object to be detected, represents the ground truth class value of the object to be detected, and n represents the number of samples, represents the Sigmoid function, represents the total loss function, represents the CIoU Loss object detection regression loss function, and both represent the BCE Loss binary cross-entropy loss function, where, is used to evaluate the accuracy of the prediction of the presence of the target by the oil spill detection model, is used to evaluate the accuracy of the prediction of the target class classification by the oil spill detection model, 、 and respectively represent 、 and corresponding weight coefficients, is the loss function for the bounding box regression task, is the binary cross-entropy loss function.

2. The method for detecting crude oil leakage in complex scenarios based on computer vision according to claim 1, wherein, The feature extraction is performed on the obtained original image data based on the scene conversion model to generate scene feature information in sunny scenes and scene feature information in complex scenes. The specific calculation formulas are as follows: （1） （2） In formulas (1)-(2): represents the original image data in sunny weather scenarios, represents the original image data in complex scenarios, represents the encoder in sunny weather scenarios, represents the encoder in complex scenarios, represents the scene feature information in sunny weather scenarios, represents the scene feature information in complex scenarios.

3. A method for detecting crude oil leakage in complex scenarios based on computer vision according to claim 2, characterized in that, Based on the scene conversion module, the scene feature information in sunny scenes is converted into different complex scenes to generate scene conversion feature information in different complex scenes. The specific calculation formulas are as follows: （3） In formula (3): represents the scene conversion module, represents the scene conversion feature information; based on the scene conversion feature information, scene conversion image data is generated, and the specific calculation formula is as follows: （4） In formula (4): represents an image generator, represents scene conversion image data.

4. A method for detecting crude oil leakage in complex scenarios based on computer vision according to claim 3, characterized in that, The generated scene conversion image data is optimized using an adversarial loss function. The specific calculation formulas are as follows: （5） （6） （7） （8） In formulas (5)-(8): represents the loss function of the optimized generator, represents the discriminator, represents the mathematical expectation, represents the Perceptual Loss function, represents the number of feature layers of the pre-trained VGG19, represents the feature layer of the pre-trained VGG19, represents the Style Loss function, represents the function for calculating the mean, represents the function for calculating the standard deviation, and represent in the loss function and are the weight parameters of.

5. An oil spill detection system in complex scenarios based on computer vision, characterized in that, It includes: An image acquisition module for acquiring original image data in multiple scenes, where the multiple scenes include sunny scenes and complex scenes; A scene feature information extraction module for performing feature extraction on the acquired original image data based on the scene conversion model to generate scene feature information in sunny scenes and scene feature information in complex scenes; A scene conversion module for converting the scene feature information in sunny scenes into different complex scenes based on the scene conversion module to generate scene conversion feature information in different complex scenes, and generating scene conversion image data based on the scene conversion feature information; A training dataset construction module for optimizing the generated scene conversion image data using an adversarial loss function and constructing a training dataset based on the original image data and the scene conversion image data in sunny scenes; A crude oil leakage detection model construction module for constructing a crude oil leakage detection model based on a lightweight mixture of experts network, a feature extraction network, a multi-scale feature fusion network, and a detection network; A crude oil leakage detection model training module for inputting the training dataset into the crude oil leakage detection model for training to obtain a trained crude oil leakage detection model; A detection module for inputting the image data to be detected into the trained crude oil leakage detection model to obtain the corresponding detection result; The complex scenes include cloudy scenes, rainy scenes, haze scenes, and snowy scenes; The process of inputting the training dataset into the crude oil leakage detection model for training to obtain a trained crude oil leakage detection model specifically includes: Inputting the training dataset into the lightweight mixture of experts network for image enhancement processing to generate enhanced feature information in multiple scenes, and performing fusion processing on the enhanced feature information in multiple scenes to generate fused enhanced feature information; inputting the fused enhanced feature information into the feature extraction network for feature extraction to extract multiple feature maps of different scales; inputting the multiple feature maps of different scales into the multi-scale feature fusion network for feature enhancement to generate multiple scale-fused feature maps; inputting the multiple scale-fused feature maps into the detection network for detection processing to generate the corresponding detection result; optimizing the crude oil leakage detection model using an object detection regression loss function and a binary cross-entropy loss function until the crude oil leakage detection model converges to obtain a trained crude oil leakage detection model; The process of inputting the training dataset into the lightweight mixture of experts network for image enhancement processing to generate enhanced feature information in multiple scenes. The specific calculation formulas are as follows: （9） （10） （11） （12） （13） In formulas (9)-(13): represents the training data set, represents the shallow feature extraction network, represents the shallow feature map, represents the feature enhancement network under cloudy sky scenes, represents the feature enhancement network under rainy sky scenes, represents the feature enhancement network under haze scenes, represents the feature enhancement network under snowy sky scenes, represents the enhanced feature information under cloudy sky scenes, represents the enhanced feature information under rainy sky scenes, represents the enhanced feature information under haze scenes, represents the enhanced feature information under snowy sky scenes; The process of performing fusion processing on the enhanced feature information in multiple scenes to generate fused enhanced feature information. The specific calculation formulas are as follows: （14） In formula (14): represents the adaptive fusion module, represents the fused enhanced feature information; Input the fused enhanced feature information into the feature extraction network for feature extraction, and extract feature maps of multiple different scales. The specific calculation formula is as follows: （15） In formula (15): represents the feature extraction network, , and respectively represent feature maps of multiple different scales, where has the same scale as the original image data, has a scale that is half of the original image data scale, has a scale that is one-fourth of the original image data scale; Input the feature maps of multiple different scales into the multi-scale feature fusion network for feature enhancement, and generate multi-scale fused feature maps. The specific calculation formula is as follows: （16） In formula (16): represents a multi-scale feature fusion network,[[]] , and respectively represent the scale fusion feature maps generated after the fusion processing of , and using the multi-scale feature fusion network; Input the multi-scale fused feature maps into the detection network for detection processing, and generate corresponding detection results. The specific calculation formula is as follows: （17） In formula (17): represents a detection network, , and respectively represent the detection results generated after performing detection processing on , and where the detection results include the upper left corner coordinate data of the object to be detected in the image data, the width data of the object to be detected, the height data of the object to be detected, and the category information of the object to be detected. Based on the upper left corner coordinate data, the width data, and the height data, a predicted rectangular detection frame corresponding to the object to be detected is obtained; Use the object detection regression loss function and the binary cross-entropy loss function to optimize the crude oil leakage detection model until the crude oil leakage detection model converges, and obtain the trained crude oil leakage detection model. The specific calculation formula is as follows: （18） （19） （20） （21） （22） （23） In formulas (18)-(23): represents the predicted rectangular detection box of the object to be detected predicted by the detection network, represents the ground-truth rectangular detection box of the object to be detected, represents the calculation result of the overlap degree between the predicted rectangular detection box and the ground-truth rectangular detection box by dividing the intersection of the predicted rectangular detection box and the ground-truth rectangular detection box by their union. c represents the diagonal length data of the minimum bounding rectangle of the predicted rectangular detection box and the ground-truth rectangular detection box, and v represents the similarity measurement result of the aspect ratio, represents the width data of the ground-truth rectangular detection box, represents the height data of the ground-truth rectangular detection box, w represents the width data of the predicted rectangular detection box, and h represents the height data of the predicted rectangular detection box, represents the calculation result of the distance between the centers of the measured rectangular detection box and the ground-truth rectangular detection box, represents the weight coefficient, represents the predicted class value of the object to be detected, represents the ground-truth class value of the object to be detected, and n represents the number of samples, represents the Sigmoid function, represents the total loss function, represents the CIoU Loss , and both represent the BCELoss binary cross-entropy loss function, where, is used to evaluate the accuracy of the prediction of the presence of the target by the oil spill detection model, is used to evaluate the accuracy of the prediction of the target class classification by the oil spill detection model, , and respectively represent , and corresponding weight coefficients, is the loss function for the bounding box regression task, is the binary cross-entropy loss function.

6. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 4.

Images