An oil spill detection method and system based on reinforcement learning and improved optimization algorithm

Abstract

This invention discloses an oil spill detection method and system based on reinforcement learning and an improved optimization algorithm, relating to the field of marine oil spill detection technology. The method includes: preprocessing acquired sea surface radar images; extracting regions of interest from the preprocessed sea surface radar images using a Q-learning reinforcement learning algorithm to obtain an oil slick region map; obtaining an optimal segmentation threshold using an improved intelligent optimization algorithm, and segmenting the oil slick region map using the optimal segmentation threshold to obtain a segmented image based on the optimal segmentation threshold; and post-processing the segmented image based on the optimal segmentation threshold to obtain the final oil slick segmented image. This invention provides more accurate segmentation results for oil spill images, laying a reliable foundation for subsequent oil slick identification.

Description

Technical Field

[0001] This invention relates to the field of marine oil spill detection technology, and in particular to an oil spill detection method and system based on reinforcement learning and improved optimization algorithms. Background Technology

[0002] Marine oil spills are a major challenge for global marine ecological protection, often stemming from ship operational errors, hull damage, and collisions. Oil spills form oil slicks that block water-air exchange, destroy marine habitats, and severely impact marine industries such as fishing, tourism, and transportation. Against this backdrop, the development of efficient and accurate marine oil spill detection technologies is urgently needed.

[0003] Current marine oil spill detection technologies primarily rely on remote sensing, which has significant limitations in application. Synthetic Aperture Radar (SAR) depends on wind and wave conditions and is easily confused with targets such as biofilms, leading to a high false alarm rate. Optical and infrared technologies are greatly affected by weather and lighting conditions, resulting in limited identification capabilities. Technologies such as laser fluorescence are costly and susceptible to interference. These methods mostly rely on static features or fixed thresholds for decision-making, making it difficult to dynamically adapt to complex and changing sea conditions and target characteristics. They also lack the ability to detect and accurately segment early-stage, small-volume oil spills, limiting the real-time performance and reliability of monitoring systems. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides an oil spill detection method and system based on reinforcement learning and improved optimization algorithms.

[0005] According to one aspect of the present invention, an oil spill detection method based on reinforcement learning and improved optimization algorithm is proposed, the method comprising:

[0006] Preprocess the acquired radar images of the sea surface;

[0007] The Q-learning reinforcement learning algorithm is used to extract regions of interest from preprocessed sea surface radar images to obtain oil slick area maps.

[0008] An improved intelligent optimization algorithm is used to obtain the optimal segmentation threshold, and the oil film region map is segmented using the optimal segmentation threshold to obtain a segmented image based on the optimal segmentation threshold.

[0009] Post-processing is performed on the segmented image based on the optimal segmentation threshold to obtain the final oil film segmentation image.

[0010] Furthermore, the preprocessing includes grayscale conversion and Gaussian filtering.

[0011] Furthermore, the step of using the Q-learning reinforcement learning algorithm to extract regions of interest from the preprocessed sea surface radar image to obtain an oil slick area map includes:

[0012] Image pixels are classified into three categories: background, sea surface, and other objects; the state space includes the image's intensity features and texture features.

[0013] During training, for each image pixel, the current state is first determined by combining the pixel's intensity features and texture features. Then select an action. In other words, the category label of a pixel is selected according to an ε-greedy strategy. The execution logic of the ε-greedy strategy is as follows: when the randomly generated probability value is less than the exploration rate ε, a category is randomly selected from all possible categories as the current action; when the randomly generated probability value is greater than or equal to ε, the current state is queried. For the corresponding row in the Q-table, select the category with the largest Q-value in that row as the action; then calculate the reward value. The calculation formula is:

[0014] ;

[0015] In the formula, This represents the coefficient used to amplify the effect of feature matching; Indicates the strength matching reward; Indicates a reward for good texture matching; Represents space consistency rewards; and These represent the weights corresponding to the intensity matching reward and the texture matching reward, respectively.

[0016] Then update the Q value according to the following formula:

[0017] ;

[0018] In the formula, This represents the Q value before the update; This represents the updated Q value; This represents the parameter used to control the magnitude of each Q-value update; is the discount factor used to measure the weight of future rewards; A represents the set of all actions the agent takes in any state. Indicates the execution of an action The next state after that; The Q value for the next state. This represents the optimal Q-value for the next state;

[0019] For each image pixel, the action that maximizes its Q value is selected to determine the pixel category, thereby completing the extraction of the region of interest from the preprocessed sea surface radar image and obtaining the oil film area map.

[0020] Furthermore, the intensity matching reward The texture matching degree reward The aforementioned spatial consistency reward The calculation formula is as follows:

[0021] ;

[0022] In the formula, The expected intensity of the preset category; Represents pixels The intensity value;

[0023] ;

[0024] In the formula, The desired texture for the preset category; Represents pixels Neighborhood texture;

[0025] ;

[0026] In the formula, Represents pixels 4-area pixels, Indicates the number of effective neighbors. For indicator functions; Represents pixels 4-area pixel The intensity value.

[0027] Furthermore, obtaining the optimal segmentation threshold using the improved intelligent optimization algorithm includes:

[0028] The fitness function in the improved intelligent optimization algorithm is:

[0029] ;

[0030] In the formula, The current iteration number is T, and the maximum iteration number is T. Inter-class variance is used to measure the degree of separation between the foreground and background. , Forward weighting, As background weight, Foreground mean The background mean. This is the global mean. For threshold reward items, k is the decay coefficient, used to control the rate at which rewards decay. This is a proportional penalty term used to constrain the reasonable distribution of the foreground region. , Weighted by the ideal prospect; Basic weighting coefficients; This refers to the proportional weighting coefficient;

[0031] The improved intelligent optimization algorithm employs a phased position update strategy, as detailed below:

[0032] When t 0.3 At time T, the position update formula is:

[0033] ;

[0034] In the formula, Indicates the position of the i-th seed after the update; Indicates the optimal position of the seed; This indicates the position of the i-th seed before the update; Indicates the adjustment parameter; This represents the scaling factor, used to control the search step size. ;

[0035] When 0.3 T t 0.7 At time T, the position update formula is:

[0036] ;

[0037] When t 0.7 At time T, the position update formula is:

[0038] ;

[0039] In the formula, , This is a parameter used for local fine-tuning. This represents a standard normal distribution.

[0040] Furthermore, the method of obtaining the optimal segmentation threshold using the improved intelligent optimization algorithm also includes:

[0041] In each iteration, after obtaining the current seed position, a random negative perturbation is applied to the position, i.e.:

[0042] ;

[0043] In the formula, Indicates the disturbance term. ; This represents the i-th seed position after perturbation.

[0044] Furthermore, obtaining the optimal segmentation threshold using the improved intelligent optimization algorithm also includes: selecting the optimal solution using an improved greedy selection strategy, that is, if one of the following three conditions is met, the corresponding seed position is taken as the optimal position:

[0045] 1) ;2) ;3) ;

[0046] In the formula, This represents the fitness value corresponding to the i-th seed position after phased updates and negative perturbations; This represents the original position of the i-th seed before the current iteration update. The corresponding fitness value.

[0047] Furthermore, the post-processing of the segmented image based on the optimal segmentation threshold to obtain the final oil film segmentation image includes: performing morphological operations on the segmented image based on the optimal segmentation threshold to eliminate isolated noise regions, and combining connected component area filtering technology to remove interference regions to obtain the final oil film segmentation image.

[0048] Furthermore, after obtaining the oil film area map, the oil film area map is processed and optimized, including: for the extracted region of interest, a morphological hole filling operation is used to repair the discontinuous parts within the region of interest to generate a ROI mask; the ROI mask is logically operated with the preprocessed sea surface radar image to obtain the optimized oil film area map.

[0049] According to another aspect of the present invention, an oil spill detection system based on reinforcement learning and an improved optimization algorithm is proposed. The system is implemented based on the aforementioned oil spill detection method based on reinforcement learning and an improved optimization algorithm. The system includes:

[0050] The preprocessing module is configured to preprocess the acquired sea surface radar images in pairs;

[0051] The region of interest extraction module is configured to use the Q-learning reinforcement learning algorithm to extract the region of interest from the preprocessed sea surface radar image and obtain an oil film area map.

[0052] The optimal segmentation module is configured to obtain the optimal segmentation threshold using an improved intelligent optimization algorithm, and to segment the oil film region map using the optimal segmentation threshold to obtain a segmented image based on the optimal segmentation threshold.

[0053] The oil film detection module is configured to post-process the segmented image based on the optimal segmentation threshold to obtain the final oil film segmentation image.

[0054] The embodiments of the present invention have the following technical effects:

[0055] This invention proposes an oil spill detection method and system based on reinforcement learning and an improved optimization algorithm. It utilizes Q-learning reinforcement learning to obtain accurate oil slick area maps and improves existing intelligent optimization algorithms by modifying the fitness function, implementing phased dynamic updates, and introducing guided greedy selection. This achieves a shift from blind search to targeted intelligent optimization, resulting in a faster, more stable, and more accurate adaptive optimal segmentation threshold for complex marine oil spill SAR or optical images. This invention provides more accurate segmentation results for oil spill images, laying a reliable foundation for subsequent oil slick identification. Attached Figure Description

[0056] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0057] Figure 1 This is a flowchart of an oil spill detection method based on reinforcement learning and improved optimization algorithm provided in an embodiment of the present invention.

[0058] Figure 2 This is an example of an oil film region map obtained using the Q-learning reinforcement learning algorithm in an embodiment of the present invention.

[0059] Figure 3 This is an example of a ROI mask generated by optimizing an oil film region map in an embodiment of the present invention.

[0060] Figure 4 This is an example of an image obtained by performing logical operations between the ROI mask and the original radar grayscale image in an embodiment of the present invention.

[0061] Figure 5 This is an example of image segmentation based on the optimal segmentation threshold in an embodiment of the present invention.

[0062] Figure 6 This is an example of the final oil film segmentation image in an embodiment of the present invention. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0064] This invention proposes an oil spill detection method and system based on reinforcement learning and improved optimization algorithms, aiming to improve the accuracy and robustness of oil spill target identification in complex scenarios.

[0065] Figure 1 This is a flowchart of an oil spill detection method based on reinforcement learning and improved optimization algorithms provided in an embodiment of the present invention. See also... Figure 1 An oil spill detection method based on reinforcement learning and improved optimization algorithms includes:

[0066] S1. Preprocess the acquired radar images of the sea surface;

[0067] S2. Use the Q-learning reinforcement learning algorithm to extract the region of interest from the preprocessed sea surface radar image to obtain the oil slick area map;

[0068] S3. Obtain the optimal segmentation threshold using an improved intelligent optimization algorithm, and segment the oil film region map using the optimal segmentation threshold to obtain a segmented image based on the optimal segmentation threshold.

[0069] S4. Post-process the segmented image based on the optimal segmentation threshold to obtain the final oil film segmentation image.

[0070] The method begins with S1, in which the acquired radar images of the sea surface are preprocessed.

[0071] According to an embodiment of the present invention, the preprocessing includes grayscale conversion and Gaussian filtering, and the sea surface radar image is subjected to Gaussian filtering to remove noise in order to improve image quality.

[0072] Then, in S2, the Q-learning reinforcement learning algorithm is used to extract the region of interest from the preprocessed sea surface radar image to obtain the oil film area map.

[0073] According to an embodiment of the present invention, the Q-learning reinforcement learning algorithm uses images collected during training from oil spill events caused by pipeline malfunctions during tanker unloading. These images were continuously collected during professional oil spill monitoring patrols. The data acquisition platform is an X-band navigation radar operating in horizontal polarization mode, capable of real-time monitoring with an effective detection range of 0.75 nautical miles. The training image set consists of radar images with a resolution of 1024x1024.

[0074] Q-learning is a model-free reinforcement learning algorithm. Its goal is to enable an agent to learn to choose the optimal action through interaction with its environment to maximize long-term cumulative rewards. The training process of the Q-learning reinforcement learning algorithm is as follows.

[0075] 1) Algorithm Initialization: In image segmentation tasks, the Q-learning algorithm achieves pixel-level classification by constructing a state-action-value mapping. Pixels are classified into three categories: background, sea surface, and other objects; the state space is composed of intensity features (8-dimensional) and texture features (4-dimensional), with a total of 32 possible combinations. The table is initialized as follows:

[0076] (1)

[0077] In the formula, Q(s, () indicates the category to be selected under state s The expected value, initially a random number between 0 and 0.1; This represents a 32-row, 3-column matrix.

[0078] 2) Training Process: The core of training is updating the Q-table using the Bellman equation. Each round's specific process includes: First, randomly selecting pixels (x, y) from the image to simulate the agent's "random exploration environment"; then performing a single-step loop, each step executing: First, determining the current state by combining the pixel's intensity and texture features; then selecting an action, choosing the pixel's category label according to an ε-greedy strategy. The execution logic of the ε-greedy strategy is as follows: When the randomly generated probability value is less than the exploration rate ε, the algorithm chooses "explore," that is, randomly selecting a category from all possible categories as the current action; when the randomly generated probability value is greater than or equal to ε, the algorithm chooses "exploit," that is, querying the row of the Q-table corresponding to the current state s and selecting the category with the largest Q-value in that row as the action; finally, calculating the reward value, which is a comprehensive strength matching reward. Texture matching reward Spatial Consistency Reward Among them, the strength matching reward Used to measure current pixel intensity and category Expected intensity degree of proximity:

[0079] (2)

[0080] In the formula, The expected intensity of the preset category; Represents pixels The intensity value;

[0081] Texture matching bonus Used to measure the texture and category of the current pixel's neighborhood. Expected texture degree of proximity:

[0082] (3)

[0083] In the formula, The desired texture for the preset category; Represents pixels Neighborhood texture;

[0084] Spatial Consistency Reward To encourage class continuity between adjacent pixels, it is calculated based on the intensity difference of a 4-neighborhood as follows:

[0085] (4)

[0086] In the formula, Represents pixels The four areas Indicates the number of effective neighbors. For indicator functions; Represents pixels 4-area pixel The intensity value;

[0087] Total Rewards This is a weighted combination of the three rewards, as shown below:

[0088] (5)

[0089] In the formula, and These represent the weights corresponding to the intensity matching reward and texture matching reward, respectively; coefficients Used to amplify the impact of feature matching.

[0090] Then update the Q value according to the following formula:

[0091] (6)

[0092] In the formula, This represents the Q value before the update; This represents the updated Q value; This is used to control the magnitude of each Q-value update; the smaller the value, the more stable the learning. In this embodiment... =0.2; This is a discount factor used to measure the weight of future rewards. The larger the value, the more the model focuses on long-term returns. In this embodiment... =0.8; A represents the set of all actions of the agent in any state; To perform the action The next state after that; The Q value for the next state. The optimal Q value for the next state represents an estimate of the long-term cumulative reward.

[0093] The Q-learning reinforcement learning algorithm is used to extract regions of interest from preprocessed sea surface radar images as follows. For each image pixel, the action (category label) that maximizes its Q value is selected to determine the pixel category, thus completing the full image segmentation. The decision strategy is as follows:

[0094] (7)

[0095] In the formula, The optimal policy function refers to the optimal action to be taken in state s.

[0096] This step uses a Q-Learning model to extract regions of interest (ROIs). This model constructs a state space composed of pixel intensity features and texture features, and divides the classification results into three categories (background, sea surface, and other objects) as the action space. After 100 training iterations to optimize the converged Q-table, the Q-learning model generates initial segmentation results, such as... Figure 2 As shown.

[0097] Furthermore, after obtaining the oil film area map, it can be processed and optimized, including: selecting a grayscale threshold range of 100-200 from the aforementioned segmentation results for binarization, extracting the target area of ​​the oil film on the sea surface, and using morphological hole filling to repair discontinuous parts within the area, generating a ROI mask, as shown in the figure. Figure 3 As shown; logical operations are performed between the above binary mask and the original radar grayscale image, and the result is as follows. Figure 4 As shown in the figure. This method provides an accurate region of interest for subsequent analysis and application of oil film boundaries.

[0098] Then, in S4, the improved intelligent optimization algorithm is used to obtain the optimal segmentation threshold, and the oil film area map is segmented using the optimal segmentation threshold to obtain a segmented image based on the optimal segmentation threshold.

[0099] According to embodiments of the present invention, an intelligent optimization algorithm is used to solve the segmentation threshold problem of oil spill targets within a region of interest (ROI), including initialization, fitness function calculation, and location update mechanisms. The intelligent optimization algorithm can be a genetic algorithm or other optimization algorithms.

[0100] 1) Initialization: Randomly generate an initial population within the solution space, and determine the position of each seed individual (candidate oil spill segmentation threshold). All are within the preset threshold range The internal order follows a uniform distribution. and These are the upper and lower bounds of the threshold, respectively.

[0101] 2) The embodiments of the present invention have made the following improvements to the fitness function, namely, in the traditional inter-class variance... Based on this, a multi-factor weighted fitness function was constructed. fitness function for:

[0102] (8)

[0103] In the formula, The current iteration number is T, and the maximum iteration number is T. Inter-class variance is used to measure the degree of separation between the foreground and background. , Forward weighting, As background weight, Foreground mean The background mean. This is the global mean. For threshold reward items, k is the decay coefficient, used to control the rate of reward decay; in this embodiment, k is 30. This is a proportional penalty term used to constrain the reasonable distribution of the foreground region. , Weighted by the ideal prospect; Basic weighting coefficients; This is the proportional weighting coefficient.

[0104] 3) Position Update Strategy: This embodiment of the invention abandons the traditional intelligent optimization algorithm's "fixed step size, single update formula" mode, and improves it to a staged position update. It adopts a segmented update strategy, dynamically adjusts the bias intensity, and combines Gaussian perturbation to enhance randomness, ensuring exploration with a small threshold while avoiding getting trapped in local optima. Specifically, as follows:

[0105] 31) Early stage (t) 0.3 T), the position update formula is:

[0106] (9)

[0107] In the formula, Indicates the position of the i-th seed after the update; Indicates the optimal position of the seed; This indicates the position of the i-th seed individual before the update; In this embodiment, the adjustment parameter is indicated. =0.7; This represents the scaling factor, used to control the search step size. Its decays exponentially with the number of iterations, in order to achieve "extensive exploration in the early stage and refined development in the later stage".

[0108] 32) Mid-term (0.3) T t 0.7 T), the position update formula is:

[0109] (10)

[0110] In the formula, In this embodiment, the adjustment parameter is indicated. =0.7.

[0111] 33) Later stage (t) 0.7 T), the position update formula is:

[0112] (11)

[0113] In the formula, This is a parameter used for local fine-tuning. This represents a standard normal distribution.

[0114] Furthermore, to further strengthen the small threshold bias, in each iteration, after obtaining the current seed position, a random negative perturbation is applied to the position, i.e.:

[0115] (12)

[0116] In the formula, Indicates the disturbance term. ; This represents the i-th seed position after applying a random negative perturbation.

[0117] Furthermore, the following boundary constraints are added to the individual solutions (i.e., individual positions):

[0118] (13)

[0119] In the formula, This represents the current position of the seed after applying boundary constraints. During the iterative optimization process, the algorithm uses boundary constraints to ensure the validity of the solution.

[0120] 4) Adopt an improved greedy selection strategy to select the optimal solution (i.e. the optimal position): Improve the traditional greedy selection strategy that "only uses fitness as the standard" to a greedy selection strategy with multiple constraints.

[0121] The optimal solution is determined by satisfying one of the following three conditions:

[0122] 1) ;2) ;3) ;

[0123] In the formula, This represents the fitness value corresponding to the i-th seed position after phased updates and negative perturbations; This represents the original position of the i-th seed before the current iteration update. The corresponding fitness value.

[0124] During iterative optimization, the algorithm reinforces its preference for smaller thresholds through an improved greedy selection strategy. New solutions are acceptable under conditions such as better fitness or similar solutions with smaller thresholds. When updating the global optimum, smaller thresholds are prioritized while ensuring solution quality, thus guiding the search direction. The flexible greedy selection mechanism ensures that the algorithm avoids getting trapped in local optima.

[0125] This step utilizes an improved intelligent optimization algorithm to segment the oil film target in the ROI. The improved algorithm autonomously finds the optimal segmentation threshold through a multi-iterative evolutionary mechanism. Based on this optimal segmentation threshold, the grayscale image of the ROI (i.e., the oil film region map) is binarized to obtain the optimal segmentation result, as shown below. Figure 5 As shown.

[0126] Then, in S5, the segmented image based on the optimal segmentation threshold is post-processed to obtain the final oil film segmentation image.

[0127] According to an embodiment of the present invention, in order to improve the segmentation quality, post-processing is performed on the segmented image based on the optimal segmentation threshold, including: using morphological operations to eliminate isolated noise regions, and combining connected component area filtering technology to remove interference regions, which significantly enhances the integrity and continuity of the target region contour, and finally obtains an accurate oil film segmentation image, such as... Figure 6 As shown.

[0128] In the process of calculating the optimal segmentation threshold, this invention improves the intelligent optimization algorithm by improving the fitness function, implementing phased dynamic updates, and introducing guided greedy selection. This achieves a transformation from "blind search" to "directed intelligent optimization," enabling faster, more stable, and more accurate adaptive optimal segmentation thresholds for complex marine oil spill SAR or optical images. This results in more accurate segmentation results and lays a reliable foundation for subsequent oil film identification.

[0129] This invention also proposes an oil spill detection system based on reinforcement learning and an improved optimization algorithm. The system is implemented based on the oil spill detection method based on reinforcement learning and an improved optimization algorithm described in the above embodiments. The system includes:

[0130] The preprocessing module is configured to preprocess the acquired sea surface radar images in pairs;

[0131] The region of interest extraction module is configured to use the Q-learning reinforcement learning algorithm to extract the region of interest from the preprocessed sea surface radar image and obtain an oil film area map.

[0132] The optimal segmentation module is configured to obtain the optimal segmentation threshold using an improved intelligent optimization algorithm, and to segment the oil film region map using the optimal segmentation threshold to obtain a segmented image based on the optimal segmentation threshold.

[0133] The oil film detection module is configured to post-process the segmented image based on the optimal segmentation threshold to obtain the final oil film segmentation image.

[0134] The function of the oil spill detection system based on reinforcement learning and improved optimization algorithm described in this embodiment of the invention can be explained by the aforementioned oil spill detection method based on reinforcement learning and improved optimization algorithm. Therefore, for the parts not described in detail in the system embodiment, please refer to the above method embodiment, and they will not be repeated here.

[0135] It should be noted that the terminology used in this invention is for describing specific embodiments only and is not intended to limit the scope of this application. As shown in this specification, unless the context clearly indicates otherwise, words such as "a," "an," "an," and / or "the" do not specifically refer to the singular and may include the plural. The terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, or apparatus that includes said element.

[0136] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention.

Claims

1. An oil spill detection method based on reinforcement learning and improved optimization algorithms, characterized in that, include: Preprocess the acquired radar images of the sea surface; The Q-learning reinforcement learning algorithm is used to extract regions of interest from preprocessed sea surface radar images to obtain oil slick area maps; including: Image pixels are classified into three categories: background, sea surface, and other objects; the state space includes the image's intensity features and texture features. During training, for each image pixel, the current state is first determined by combining the pixel's intensity features and texture features. Then select an action. In other words, the category label of a pixel is selected according to an ε-greedy strategy. The execution logic of the ε-greedy strategy is as follows: when the randomly generated probability value is less than the exploration rate ε, a category is randomly selected from all possible categories as the current action; when the randomly generated probability value is greater than or equal to ε, the current state is queried. For the corresponding row in the Q-table, select the category with the largest Q-value in that row as the action; then calculate the reward value. The calculation formula is: ； In the formula, This represents the coefficient used to amplify the effect of feature matching; Indicates the strength matching reward; Indicates a reward for good texture matching; Represents space consistency rewards; and These represent the weights corresponding to the intensity matching reward and the texture matching reward, respectively. Then update the Q value according to the following formula: ； In the formula, This represents the Q value before the update; This represents the updated Q value; This represents the parameter used to control the magnitude of each Q-value update; is the discount factor used to measure the weight of future rewards; A represents the set of all actions the agent takes in any state. Indicates the execution of an action The next state after that; The Q value for the next state. This represents the optimal Q-value for the next state; For each image pixel, the action that maximizes its Q value is selected to determine the pixel category, thereby completing the extraction of the region of interest from the preprocessed sea surface radar image and obtaining the oil film area map. An improved intelligent optimization algorithm is used to obtain an optimal segmentation threshold, and the oil film region map is segmented using the optimal segmentation threshold to obtain a segmented image based on the optimal segmentation threshold; the step of obtaining the optimal segmentation threshold using the improved intelligent optimization algorithm includes: The fitness function in the improved intelligent optimization algorithm is: ； In the formula, The current iteration number is T, and the maximum iteration number is T. Inter-class variance is used to measure the degree of separation between the foreground and background. , Forward weighting, As background weight, Foreground mean The background mean. The global mean; For threshold reward items, k is the decay coefficient, used to control the rate at which rewards decay. This is a proportional penalty term used to constrain the reasonable distribution of the foreground region. , Weighted by the ideal prospect; Basic weighting coefficients; This refers to the proportional weighting coefficient; The improved intelligent optimization algorithm employs a phased position update strategy, as detailed below: When t 0.3 At time T, the position update formula is: ； In the formula, Indicates the position of the i-th seed after the update; Indicates the optimal position of the seed; This indicates the position of the i-th seed before the update; Indicates the adjustment parameter; This represents the scaling factor, used to control the search step size. ; When 0.3 T t 0.7 At time T, the position update formula is: ； When t 0.7 At time T, the position update formula is: ； In the formula, , This is a parameter used for local fine-tuning. Represents a standard normal distribution; Post-processing is performed on the segmented image based on the optimal segmentation threshold to obtain the final oil film segmentation image.

2. The oil spill detection method based on reinforcement learning and improved optimization algorithm according to claim 1, characterized in that, The preprocessing includes grayscale conversion and Gaussian filtering.

3. The oil spill detection method based on reinforcement learning and improved optimization algorithm according to claim 1, characterized in that, The intensity matching reward The texture matching degree reward The aforementioned spatial consistency reward The calculation formula is as follows: ； In the formula, The expected intensity of the preset category; Represents pixels The intensity value; ； In the formula, The desired texture for the preset category; Represents pixels Neighborhood texture; ； In the formula, Represents pixels 4-area pixels, Indicates the number of effective neighbors. For indicator functions; Represents pixels 4-area pixel The intensity value.

4. The oil spill detection method based on reinforcement learning and improved optimization algorithm according to claim 1, characterized in that, The method of obtaining the optimal segmentation threshold using the improved intelligent optimization algorithm also includes: In each iteration, after obtaining the current seed position, a random negative perturbation is applied to the position, i.e.: ； In the formula, Indicates the disturbance term. , , This is a parameter used for local fine-tuning. Represents a standard normal distribution; This represents the i-th seed position after perturbation.

5. The oil spill detection method based on reinforcement learning and improved optimization algorithm according to claim 4, characterized in that, The method of obtaining the optimal segmentation threshold using the improved intelligent optimization algorithm further includes: selecting the optimal solution using an improved greedy selection strategy, that is, if one of the following three conditions is met, the corresponding seed position is taken as the optimal position: 1） ；2） ；3） ； In the formula, This represents the fitness value corresponding to the i-th seed position after phased updates and negative perturbations; This represents the original position of the i-th seed before the current iteration update. The corresponding fitness value.

6. The oil spill detection method based on reinforcement learning and improved optimization algorithm according to claim 1, characterized in that, The post-processing of the segmented image based on the optimal segmentation threshold to obtain the final oil film segmentation image includes: performing morphological operations on the segmented image based on the optimal segmentation threshold to eliminate isolated noise regions, and combining connected component area filtering technology to remove interference regions to obtain the final oil film segmentation image.

7. An oil spill detection method based on reinforcement learning and improved optimization algorithm according to any one of claims 1-6, characterized in that, After obtaining the oil film area map, the oil film area map is processed and optimized, including: for the extracted region of interest, morphological hole filling operation is used to repair the discontinuous parts in the region of interest to generate a ROI mask; the ROI mask is logically operated with the preprocessed sea surface radar image to obtain the optimized oil film area map.

8. An oil spill detection system based on reinforcement learning and improved optimization algorithms, characterized in that, The system is implemented based on the oil spill detection method based on reinforcement learning and improved optimization algorithm according to any one of claims 1-7; the system includes: The preprocessing module is configured to preprocess the acquired sea surface radar images in pairs; The region of interest extraction module is configured to use the Q-learning reinforcement learning algorithm to extract the region of interest from the preprocessed sea surface radar image and obtain an oil film area map. The optimal segmentation module is configured to obtain the optimal segmentation threshold using an improved intelligent optimization algorithm, and to segment the oil film region map using the optimal segmentation threshold to obtain a segmented image based on the optimal segmentation threshold. The oil film detection module is configured to post-process the segmented image based on the optimal segmentation threshold to obtain the final oil film segmentation image.

Images