Infrared image segmentation method and system based on maximum between-class variance method

Abstract

The application discloses an infrared image segmentation method and system based on the maximum between-class variance method, and relates to the technical field of image segmentation, and comprises the following steps: collecting infrared sample images under different motion states by a water robot, and recording motion state parameters; analyzing the absolute difference between garbage step edges and water area pixel gradient amplitude, setting effective constraints and forming an invalid state parameter set; judging whether the newly collected motion state parameter is effective, if effective, continuing to collect target infrared images, and calculating the gray level change difference value of each pixel; determining the direction of the most severe gray level change in the pixel neighborhood, extracting the gray level profile sequence, judging whether a gray level step occurs, traversing all the pixel points, screening effective target pixels and background pixels, counting the gradient amplitude of the effective target pixels, and performing the maximum between-class variance method to divide the garbage area, so that the detection accuracy is significantly improved.

Description

Technical Field

[0001] This invention relates to the field of image segmentation technology, specifically to an infrared image segmentation method and system based on the maximum inter-class variance method. Background Technology

[0002] In the field of water surface debris cleanup and environmental monitoring, aquatic robots have gradually become an important tool. Cleaning up water surface debris is of great significance for water body environmental protection, and real-time and accurate identification of debris areas on the water surface is a key step to achieve efficient cleanup. Infrared imaging technology has great application potential in debris area detection because it can detect water surface temperature distribution and reflection characteristics. However, due to the complexity of water surface reflection and environmental conditions, water surface infrared images usually have drastic changes in gradient amplitude, which makes it difficult for traditional image segmentation methods to effectively distinguish debris areas from the water surface background.

[0003] Currently, common technical solutions mainly involve setting conventional segmentation thresholds to segment water surface garbage areas in infrared images, or using deep learning-based methods to classify or semantically segment infrared images. However, these methods still face many challenges when processing water surface infrared images. For example, the water surface environment is complex and changeable, and garbage edge detection is easily interfered with. Performing segmentation tasks in environments with strong interference will lead to segmentation task failure and waste a lot of computing resources. Therefore, there is an urgent need for an image segmentation method that can determine whether an image acquisition and segmentation task is being performed in the current environment, and can effectively eliminate water surface interference in dynamic water environments to improve the accuracy and robustness of garbage region segmentation.

[0004] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to provide an infrared image segmentation method and system based on the maximum inter-class variance method, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: An infrared image segmentation method based on the Otsu's method includes the following steps: Infrared sample images of multiple motion states were collected by an aquatic robot, and the robot's motion state parameters were recorded simultaneously at the time of image acquisition. For each infrared sample image, the absolute difference of the gradient magnitude of each pixel in the garbage step edge and the remaining water surface area was analyzed. For any motion state parameter, set a valid constraint based on the magnitude of its corresponding absolute difference. Record the motion state parameters that do not satisfy the valid constraints as invalid state parameters. Iterate through all motion state parameters to form a set of invalid state parameters. The motion state parameters of the water robot in the water area to be treated are collected. It is determined whether the motion state parameters fall into the set of invalid state parameters. If they do, the infrared image acquisition is stopped. Otherwise, the target infrared image of the surface of the water area to be treated is collected, and the grayscale change difference value of each pixel in the target infrared image is calculated. For any pixel in the target infrared image, determine the direction of the most drastic gray-level change in the neighborhood of the pixel, extract the gray-level values ​​of several pixels including the pixel along the direction to form a gray-level profile sequence of the pixel, calculate the degree of difference between adjacent gray-level values ​​in the gray-level profile sequence, and determine whether a gray-level step occurs in the gray-level profile sequence based on the degree of difference. Traverse all pixels in the target infrared image, identify pixels that undergo gray-level steps within the gray-level profile sequence, filter effective target pixels and background pixels, and statistically analyze the gradient magnitude distribution of effective target pixels to perform the maximum inter-class variance method to divide the garbage area on the surface of the water to be treated.

[0007] Furthermore, the motion state parameters include the three-axis acceleration values ​​of the water robot at the moment of infrared image acquisition and the light intensity, which together constitute a four-dimensional motion state parameter vector. The system controls the water robot to operate in various different motion states. In each motion state, infrared sample images are continuously acquired at the corresponding moment. At the same time as acquiring each infrared sample image, the four-dimensional motion state parameter vector corresponding to that acquisition moment is recorded through the inertial measurement unit.

[0008] Furthermore, the contour of the waste step edge is determined by manual annotation for any infrared sample image, wherein the waste step edge specifically refers to the edge where the waste contacts the water surface; The calculation of the gradient magnitude of the garbage step edge pixel includes, in any infrared sample image, for any garbage step edge pixel, calculating the first-order central difference of its gray value in the horizontal and vertical directions, taking the sum of the squares of the first-order central differences in the two directions based on the first-order central difference of the garbage step edge pixel, and taking the square root of the sum of squares to obtain the pixel gradient magnitude of the garbage step edge pixel. Traverse all garbage step edge pixels and determine the pixel gradient magnitude of all garbage step edge pixels in the infrared sample image; Determine the pixel gradient magnitude of each pixel in the remaining water surface area image within the infrared sample image, calculate the absolute difference between it and the pixel gradient magnitude of each garbage step edge in the infrared sample image, and select the smallest absolute difference for any pixel in the remaining water surface area image as the absolute difference between the pixel gradient magnitude of that pixel and the pixel gradient magnitude of the garbage step edge.

[0009] Furthermore, for any motion state parameter, if the absolute difference under that motion state parameter simultaneously satisfies the following conditions, then the motion state parameter is considered not to satisfy the valid constraints. Condition 1: All absolute differences under this motion state parameter have an absolute difference that is no greater than a preset difference threshold; Condition 2: The number of absolute differences is not greater than the preset difference threshold and is not less than the preset interference threshold; Traverse all motion state parameters to form an invalid state parameter set. The invalid state parameter set specifically includes sub-ranges of each state parameter. Each sub-range has a lower limit based on the minimum value of the corresponding invalid state parameter, and an upper limit set to a preset multiple of the lower limit.

[0010] Furthermore, the water robot is placed in the water area to be treated, and the current motion state parameters are collected. If any motion state parameter falls into the set of invalid state parameters, the infrared image acquisition is stopped. Otherwise, acquire the target infrared image of the surface of the water area to be processed. For any pixel in the target infrared image, the specific method for determining the direction of the most drastic grayscale change is as follows: take the first difference value of the pixel in the horizontal and vertical directions as the ordinate and abscissa respectively, calculate the arctangent function value corresponding to the ratio of the ordinate to the abscissa, take the arctangent function value as the direction angle, sample the pixel in the image space along the straight line indicated by the direction angle, and extract the grayscale values ​​of several consecutive pixels including the pixel to form the grayscale profile sequence.

[0011] Furthermore, for any pixel in the target infrared image, in its grayscale profile sequence, the difference between the grayscale values ​​of two adjacent pixels in the grayscale profile sequence is calculated, and the difference result is used as the first-order difference value. The entire grayscale profile sequence is traversed in this manner to obtain a first-order difference sequence composed of first-order difference values. Extract the maximum and second largest absolute values ​​from the first-order difference sequence. If the ratio of the maximum absolute value to the second largest absolute value is not greater than a preset step significance ratio threshold, then it is determined that the pixel has not experienced a gray-level step in the gray-level profile sequence. Otherwise, extract the pixel corresponding to the maximum absolute value in the first-order difference sequence. Taking this pixel as the center, calculate the average gray value of all pixels on both sides of this pixel in the gray-scale profile sequence, as well as the coefficient of variation of the gray values ​​of the pixels on both sides. If the absolute difference between the average gray values ​​on both sides is not less than the preset average threshold, and the coefficient of variation of the gray values ​​of the pixels on both sides is not greater than the preset equilibrium threshold, then it is determined that a gray-scale step has occurred in the gray-scale profile sequence; otherwise, it is determined that no gray-scale step has occurred.

[0012] Furthermore, the specific method for filtering effective target pixels and background pixels is as follows: traverse all pixels in the target infrared image, determine the pixels corresponding to gray-level steps in all gray-level profile sequences, and take them as effective target pixels, while the gray-level values ​​of the remaining pixels are taken as background pixels. The logic behind the Otsu's method is as follows: A histogram of gradient magnitude distributions for all valid target pixels is plotted. The gradient magnitude with the lowest frequency in the histogram is taken as a reference point, which is then used as the starting point of the candidate interval. The maximum gradient magnitude among all valid target pixels is taken as the ending point of the candidate interval. Within this candidate interval, the inter-class variance corresponding to each candidate gradient magnitude is calculated, and the candidate gradient magnitude corresponding to the maximum inter-class variance is taken as the optimal segmentation threshold. Pixels with gradient magnitudes of effective target pixels that are not less than the optimal segmentation threshold are designated as garbage step edge pixels; otherwise, they are designated as background region pixels. This method is used to divide the garbage region on the surface of the water body to be processed.

[0013] This invention also provides an infrared image segmentation system based on the Otsu's method, wherein the infrared image segmentation system based on Otsu's method is used to execute the above-described infrared image segmentation method based on Otsu's method, and includes: The sample data acquisition module is used to acquire infrared sample images of multiple motion states through the water robot, and synchronously record the robot motion state parameters at the time of image acquisition. For each infrared sample image, the absolute difference of the gradient amplitude of each pixel in the garbage step edge and the remaining water surface area is analyzed. The working feature filtering module is used to set effective constraints for any motion state parameter based on its corresponding absolute difference magnitude, record the motion state parameters that do not meet the effective constraints as invalid state parameters, and traverse all motion state parameters to form a set of invalid state parameters. The grayscale difference analysis module is used to collect the motion state parameters of the water robot in the water area to be treated, determine whether its motion state parameters fall into the invalid state parameter set, if they fall into the invalid state parameter set, stop infrared image acquisition, otherwise collect the target infrared image of the surface of the water area to be treated, and calculate the grayscale change difference value of each pixel in the target infrared image. The grayscale step detection module is used to determine the direction of the most drastic grayscale change in the neighborhood of any pixel in the target infrared image, extract the grayscale values ​​of several pixels including the pixel along the direction to form a grayscale profile sequence of the pixel, calculate the degree of difference between adjacent grayscale values ​​in the grayscale profile sequence, and determine whether a grayscale step has occurred in the grayscale profile sequence based on the degree of difference. The image segmentation module is used to traverse all pixels in the target infrared image, identify pixels that have gray-level steps in the gray-level profile sequence, filter effective target pixels and background pixels, and statistically analyze the gradient magnitude distribution of effective target pixels in order to perform the maximum inter-class variance method to divide the garbage area on the surface of the water to be treated.

[0014] Compared with the prior art, the beneficial effects of the present invention are: This invention fully utilizes infrared sample images under different motion states. By analyzing the absolute difference in pixel gradient amplitude between the garbage step edge and the water surface region, the boundary between garbage and water surface is effectively identified, ensuring the accuracy of image segmentation and avoiding recognition errors caused by changes in lighting or water wave reflection. Based on this analysis, an effective working environment is determined. By establishing a set of invalid state parameters, image acquisition is ensured under specific robot motion states, thereby avoiding image distortion and noise interference caused by changes in motion states. When the working environment is not met, the acquisition task is stopped, saving ineffective resource waste. This invention also accurately identifies pixels with gray-level abrupt changes by analyzing the gray-level change profile of each pixel in the target infrared image. Furthermore, by statistically analyzing the gradient magnitude distribution of effective target pixels, it further optimizes the application effect of the maximum inter-class variance method. It clearly distinguishes between garbage areas and the water surface background, forming a clear segmentation result. Compared with traditional threshold segmentation or edge detection methods, this solution demonstrates higher robustness and reliability when handling complex backgrounds. Especially in dynamic water surface environments, the accuracy of garbage detection is significantly improved, thereby achieving efficient garbage collection and management. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the overall method flow of the present invention; Figure 2 This is a schematic diagram of image recognition and segmentation according to the present invention; Figure 3 This is a schematic diagram of the overall system structure of the present invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0017] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0018] Example: Please see Figures 1-2 The present invention provides a technical solution: An infrared image segmentation method based on the Otsu's method includes the following steps: Step 1: Collect infrared sample images of multiple motion states using an aquatic robot, and simultaneously record the robot's motion state parameters at the time of image acquisition. For each infrared sample image, analyze the absolute difference in the gradient magnitude of each pixel in the garbage step edge and the remaining water surface area.

[0019] The motion state parameters include the three-axis acceleration values ​​of the water robot at the moment of infrared image acquisition and the light intensity, which together constitute a four-dimensional motion state parameter vector. A three-axis accelerometer, such as a MEMS accelerometer, is installed on the aquatic robot. This sensor can measure the robot's acceleration in real time, including acceleration in the forward, backward, left, right, and up / down directions. The robot control system reads the data from the accelerometer periodically. Whenever an infrared image is acquired, the three-axis acceleration value output by the sensor is recorded simultaneously to ensure synchronization between the acceleration data and the infrared image acquisition time. A timestamp mechanism can be used to timestamp the acceleration data when the image is acquired so that it can be accurately matched in subsequent analysis. A light intensity sensor, such as a photoresistor or photodiode, is installed on the robot. This sensor can measure the ambient light intensity in real time. Similar to the method of acquiring acceleration data, when the robot acquires an infrared image, the output value of the light intensity sensor is read and the light intensity data at that moment is recorded. A timestamp mechanism is used to ensure that the time of the light intensity data is consistent with the time of infrared image acquisition; thus forming a four-dimensional motion state parameter vector.

[0020] The system controls the water robot to operate in various different motion states. In each motion state, infrared sample images are continuously acquired at the corresponding moment. At the same time as acquiring each infrared sample image, the four-dimensional motion state parameter vector corresponding to that acquisition moment is recorded through the inertial measurement unit.

[0021] The contour of the waste step edge is determined by manual annotation for any infrared sample image. The waste step edge specifically refers to the edge where the waste comes into contact with the water surface. Calculating the gradient magnitude of pixels at the step edge of a garbage dump involves, specifically, within any infrared sample image, calculating the first-order central difference of the grayscale value of any pixel at the step edge of a garbage dump along the horizontal and vertical directions. Based on this first-order central difference, the sum of the squares of the first-order central differences in the two directions is taken, and the square root of this sum is obtained to obtain the pixel gradient magnitude of the pixel at the step edge of the garbage dump. The specific formula used to calculate the pixel gradient magnitude is as follows: In the formula, Intra-infrared sample image coordinates The pixel gradient magnitude below Intra-infrared sample image coordinates First-order central difference along the horizontal direction, Intra-infrared sample image coordinates First-order central difference along the vertical direction, This refers to the x-coordinate index of the pixel at the garbage step edge within the infrared sample image. The ordinate index of the pixel at the step edge of the garbage in the infrared sample image; It should be noted that when When the value is small, the gray value of the pixel and its neighborhood is almost constant, indicating that the pixel is in a region with uniform thermal radiation, which corresponds to the absorption and reflection area inside the water background or the garbage target in the infrared image. when When the value is large, the pixel is located at the position where the gray value changes abruptly. In the infrared image, this corresponds to the boundary between solid waste and the water surface. At this point, the difference in emissivity and heat capacity causes a significant gray value jump between adjacent pixels. This method determines how the grayscale value changes at the boundary between solid waste and the water surface, and distinguishes the gradient change of grayscale value from the gradient change caused by the edge of water waves or the influence of light; thereby further improving the accuracy of waste identification and eliminating the influence of water waves and light.

[0022] The formula used to calculate the first-order central difference is as follows: In the formula, coordinates The grayscale value of the pixel at the edge of the garbage step. coordinates The grayscale value of the pixel at the edge of the garbage step. coordinates The grayscale value of the pixel at the edge of the garbage step. coordinates Gray values ​​of pixels at the garbage step edge; Traverse all garbage step edge pixels and determine the pixel gradient magnitude of all garbage step edge pixels in the infrared sample image; Determine the pixel gradient magnitude of each pixel in the remaining water surface area image within the infrared sample image, calculate the absolute difference between it and the pixel gradient magnitude of each garbage step edge in the infrared sample image, and select the smallest absolute difference for any pixel in the remaining water surface area image as the absolute difference between the pixel gradient magnitude of that pixel and the pixel gradient magnitude of the garbage step edge.

[0023] It should be noted that by comparing the gradient magnitudes of the water surface region and the step edge of the trash, pixels in the water surface that are similar to the edge features of the trash can be identified. This helps to understand the interaction between the water surface and the trash, as well as the distribution characteristics of the trash, in more detail. In image processing, edges are usually the most important features. By calculating the absolute difference and finding the minimum value, the accuracy of edge detection can be effectively enhanced, enabling the algorithm to find the water surface feature that is closest to the edge of the trash.

[0024] Step 2: For any motion state parameter, set a valid constraint based on the magnitude of its corresponding absolute difference. Record the motion state parameters that do not meet the valid constraints as invalid state parameters. Iterate through all motion state parameters to form a set of invalid state parameters.

[0025] For any motion state parameter, if the absolute difference under the motion state parameter simultaneously satisfies the following conditions, then the motion state parameter is considered not to satisfy the valid constraints. Condition 1: All absolute differences under this motion state parameter have an absolute difference that is no greater than a preset difference threshold; Condition 2: The number of absolute differences is not greater than the preset difference threshold and is not less than the preset interference threshold; It should be noted that, in condition 1, among all the absolute differences under the motion state parameter, there is an absolute difference that is no greater than the preset difference threshold; this means that in a specific motion state, at least one water surface pixel has a gradient magnitude that is small in difference from the gradient magnitude of the garbage step edge, showing a strong similarity between them. That is, under the current water surface condition, the existing interference may lead to the interference being judged as garbage, resulting in detection errors. Therefore, it is used as one of the judgment conditions for invalid state parameters. Condition 2 limits the number of anomalies, indicating that in this motion state, there is a relatively large number of pixels that are similar to the garbage step edge feature. This enhances the effectiveness of the motion state parameter, with more samples supporting its judgment ability in a specific state. It can effectively exclude state parameters that occur by chance and do not conform to the overall trend. It helps to reduce the impact of noise on the results, thereby improving the stability and reliability of the motion state parameter.

[0026] The difference threshold is used to determine the absolute difference between pixels in the water surface area and pixels at the garbage step edge; if an absolute difference is less than or equal to this threshold, the point is considered to have similar features, and it is generally set between 2 and 10. The interference threshold refers to the number of water surface pixels that are similar to the edge features of garbage in a certain motion state. By understanding the interference thresholds used in other studies in similar scenarios, we can use them as a reference basis for setting the threshold. Depending on the size of the monitored area, the threshold is generally set between 10 and 50.

[0027] All motion state parameters are iterated to form an invalid state parameter set. This invalid state parameter set specifically includes sub-ranges for each state parameter. Each sub-range has its lower limit set by the minimum value of the corresponding invalid state parameter, and its upper limit set by a preset multiple of the lower limit. By setting the upper limit to a preset multiple of the lower limit, the range of state parameters can be linearly expanded; typically, it is set to 3-5 times to indicate more intense motion.

[0028] Step 3: Collect the motion state parameters of the water robot in the water area to be treated, and determine whether its motion state parameters fall into the invalid state parameter set. If they do, stop the infrared image acquisition. Otherwise, collect the target infrared image of the surface of the water area to be treated, and calculate the grayscale change difference value of each pixel in the target infrared image.

[0029] The aquatic robot is placed in the water area to be treated, and its current motion state parameters are collected. These motion state parameters are collected in real time during the operation of the aquatic robot. If any motion state parameter falls into the invalid state parameter set, infrared image acquisition is stopped. If the current motion state parameter matches the invalid state parameter set, it indicates that there may be large waves or light reflection in the water environment at this time, which will negatively affect the quality of the infrared image. For example, when the waves are too high, the reflection and interference of the water surface may cause the edges of the target in the image to become blurred, affecting the subsequent recognition and segmentation effect, and seriously affecting image recognition and segmentation. The calculation of the grayscale difference value of each pixel in the target infrared image is the same as the calculation of the first-order central difference mentioned above, and will not be repeated here.

[0030] Step 4: For any pixel in the target infrared image, determine the direction of the most drastic gray-level change in the neighborhood of that pixel, extract the gray-level values ​​of several pixels including that pixel along that direction to form a gray-level profile sequence of that pixel, calculate the degree of difference between adjacent gray-level values ​​in the gray-level profile sequence, and determine whether a gray-level step occurs in the gray-level profile sequence based on the degree of difference.

[0031] Otherwise, acquire the target infrared image of the surface of the water area to be processed. For any pixel in the target infrared image, the specific method for determining the direction of the most drastic grayscale change is as follows: take the first difference value of the pixel in the horizontal and vertical directions as the ordinate and abscissa respectively, calculate the arctangent function value corresponding to the ratio of the ordinate to the abscissa, take the arctangent function value as the direction angle, sample the pixel in the image space along the straight line indicated by the direction angle, and extract the grayscale values ​​of several consecutive pixels including the pixel to form the grayscale profile sequence. The formula used to determine the direction of the most dramatic grayscale change is as follows: In the formula, Intra-infrared coordinates of the target The direction angle where the grayscale change is most drastic; and These are the indices of the x and y coordinates of the pixels within the target's infrared image, respectively. It should be noted that the coordinates horizontal difference at the point Mathematically, this is equivalent to the partial derivative of the grayscale field at that point along the horizontal direction, i.e., the instantaneous rate of change of the grayscale value with respect to the column coordinates; vertical difference It is equivalent to the partial derivative of the gray field at that point along the vertical direction, that is, the instantaneous rate of change of the gray value with the change of the row coordinate; Direction angle The physical meaning of the gray value is precisely the direction in which the gray value increases the fastest in space within the neighborhood of the pixel. That is, starting from the pixel, moving a unit distance along this direction will result in the largest increase in gray value. The gray value has the strongest tendency to monotonically increase or decrease with spatial coordinates. Extracting the gray profile sequence along this direction as the basis for subsequent classification can maximize the highlighting of the most essential spatial morphological information of the pixel.

[0032] Centered on the current pixel, the grayscale profile sequence is constructed by selecting several consecutive pixels before and after the direction of the most drastic grayscale change. Generally, 2-5 pixels are selected before and after the current pixel to construct the grayscale profile sequence.

[0033] For any pixel in the target infrared image, in its grayscale profile sequence, the difference between the grayscale values ​​of two adjacent pixels in the grayscale profile sequence is calculated, and the difference result is used as the first-order difference value. The entire grayscale profile sequence is traversed in this way to obtain a first-order difference sequence composed of first-order difference values. Extract the maximum and second largest absolute values ​​from the first-order difference sequence. If the ratio of the maximum absolute value to the second largest absolute value is not greater than a preset step significance ratio threshold, then it is determined that the pixel has not experienced a gray-level step in the gray-level profile sequence. It should be noted that when there is a physical property step boundary between solid waste and water in the grayscale profile, at the location where the step occurs, the grayscale value changes significantly within a distance of 1 to 3 pixels. This is characterized by a first-order difference sequence, meaning that the absolute value of the difference corresponding to this jump is much larger than the second largest value, and the ratio between the two is relatively large. The grayscale step is determined by a ratio, which is independent of the overall contrast and grayscale dynamic range of the image. Regardless of whether the contrast is high or low, the significant difference between the maximum slope and the second largest slope always exists in the step boundary scene, while the finite difference between the maximum slope and the second largest slope always exists in the wave convex scene. Therefore, this judgment logic naturally has the ability to resist illumination fluctuations and environmental contrast changes, without the need to frequently adjust the threshold parameters according to different environments. Specifically, the step significance ratio threshold can be tested on multiple garbage samples to calculate the ratio of the maximum value to the second largest value, and a reasonable distribution analysis of this ratio can be performed. A certain proportion of quantiles, such as the 95th percentile, can be selected as the reference value for the step significance ratio threshold to ensure that most effective grayscale steps can be correctly identified.

[0034] Otherwise, extract the pixel corresponding to the maximum absolute value in the first-order difference sequence. Taking this pixel as the center, calculate the average gray value of all pixels on both sides of this pixel in the gray-scale profile sequence, as well as the coefficient of variation of the gray values ​​of the pixels on both sides. If the absolute difference between the average gray values ​​on both sides is not less than the preset average threshold, and the coefficient of variation of the gray values ​​of the pixels on both sides is not greater than the preset equilibrium threshold, then it is determined that a gray-scale step has occurred in the gray-scale profile sequence; otherwise, it is determined that no gray-scale step has occurred.

[0035] The coefficient of variation is defined as the ratio of the standard deviation to the average value of all pixel grayscale values ​​within the specified portion. Its physical meaning is the degree of dispersion of each pixel grayscale value within the specified portion relative to its average value. The smaller the coefficient of variation, the more uniform and less volatile the grayscale values ​​within the specified portion. The larger the coefficient of variation, the greater the fluctuation or tilt within the specified portion. To determine whether the grayscale step is a change from the water surface to solid waste, the grayscale values ​​of the water surface are relatively uniform. Similarly, after the step to the solid waste surface, the grayscale values ​​of the solid waste surface are also uniform within a short distance and a limited number of pixels. If the coefficient of variation is too large after the step, it indicates that there may be wave or lighting interference, and the edge pixels of the step from the water surface to the waste have not been truly identified. At the same time, the absolute difference between the average gray values ​​on both sides is set to be no less than a preset average threshold, which reflects the difference between the gray values ​​of the water surface and the gray values ​​of the solid waste surface, further constraining the judgment of gray value step.

[0036] The preset average threshold is used to determine the absolute difference between the average gray values ​​on both sides, so as to confirm the significant difference between the gray values ​​of the water surface and the gray values ​​of the solid waste surface. For different objects, such as water surface and solid waste, their gray values ​​usually have significant differences. The gray values ​​can be measured by multiple experiments to calculate the difference between the average gray values ​​of the water surface and solid waste, which can be used as a reference for the preset average threshold. The average threshold is then set accordingly. The equalization threshold is used to evaluate the coefficient of variation of the gray values ​​of pixels on both sides to determine whether there is interference causing gray value abrupt changes. The coefficient of variation reflects the dispersion of pixel gray values. The smaller the coefficient, the more uniform the gray values ​​are, which can distinguish between the gray values ​​of water surface and the gray values ​​of garbage surface. By analyzing multiple sets of data, the coefficient of variation of gray values ​​under normal conditions, such as water surface or solid garbage surface, can be calculated to obtain the distribution characteristics of the coefficient. The coefficient of variation is then set specifically based on the distribution characteristics and expert experience.

[0037] Step 5: Traverse all pixels in the target infrared image, identify pixels that have gray-level steps in the gray-level profile sequence, filter effective target pixels and background pixels, and statistically analyze the gradient magnitude distribution of effective target pixels to perform the maximum inter-class variance method to divide the garbage area on the surface of the water to be treated.

[0038] The specific method for filtering effective target pixels and background pixels is as follows: traverse all pixels in the target infrared image, determine the pixels corresponding to gray-level steps in all gray-level profile sequences, and take them as effective target pixels. The gray values ​​of the remaining pixels are taken as background pixels. The pixels corresponding to gray-level steps are the midpoints of the gray-level profile sequences where gray-level steps occur. The logic behind the Otsu's method is as follows: A histogram of gradient magnitude distributions for all valid target pixels is plotted. The gradient magnitude with the lowest frequency in the histogram is taken as a reference point, which is then used as the starting point of the candidate interval. The maximum gradient magnitude among all valid target pixels is taken as the ending point of the candidate interval. Within this candidate interval, the inter-class variance corresponding to each candidate gradient magnitude is calculated, and the candidate gradient magnitude corresponding to the maximum inter-class variance is taken as the optimal segmentation threshold. The specific formula used to calculate the inter-class variance is as follows: In the formula, Indicates the segmentation threshold is At that time, the inter-class variance values ​​between the foreground and background images are obtained through segmentation. The foreground and background images represent images of solid waste and water surface areas, respectively. Indicates the segmentation threshold The frequency of occurrence in the histogram, The proportion of foreground pixels to the total image size. The proportion of background pixels to the image. The average gradient magnitude of the foreground pixels. The average gradient magnitude of the background pixels; For the segmentation threshold variable; It should be noted that the main purpose of selecting the threshold is to achieve optimal image segmentation in order to effectively separate the foreground (solid debris) from the background (water surface). By maximizing the inter-class variance, it can be ensured that the selected threshold maximizes the difference between the foreground and the background, thereby improving the accuracy of segmentation. The formula is based on the conventional inter-class variance segmentation method. However, since solid waste occupies a small proportion of pixels, the histogram of pixel gradient magnitudes in water surface images does not exhibit a clear bimodal or unimodal trend. Therefore, using the traditional maximum inter-class variance (MAX) method for image segmentation will introduce bias. Generally, the optimal segmentation threshold always appears at the trough of a bimodal distribution or the edge of a unimodal distribution. Therefore, an improved method is proposed based on the traditional MAX method, incorporating a weight when calculating the variance. Based on maximizing the variance of the foreground and background, the optimal segmentation threshold is selected from points with very low probability of occurrence. Pixels with gradient magnitudes of effective target pixels that are not less than the optimal segmentation threshold are designated as garbage step edge pixels; otherwise, they are designated as background region pixels. This method is used to divide the garbage region on the surface of the water body to be processed.

[0039] Please see Figure 3The present invention also provides an infrared image segmentation system based on the maximum inter-class variance method. This system is used to execute the aforementioned infrared image segmentation method based on the maximum inter-class variance method, and includes: The sample data acquisition module is used to acquire infrared sample images of multiple motion states through the water robot, and synchronously record the robot motion state parameters at the time of image acquisition. For each infrared sample image, the absolute difference of the gradient amplitude of each pixel in the garbage step edge and the remaining water surface area is analyzed. The working feature filtering module is used to set effective constraints for any motion state parameter based on its corresponding absolute difference magnitude, record the motion state parameters that do not meet the effective constraints as invalid state parameters, and traverse all motion state parameters to form a set of invalid state parameters. The grayscale difference analysis module is used to collect the motion state parameters of the water robot in the water area to be treated, determine whether its motion state parameters fall into the invalid state parameter set, if they fall into the invalid state parameter set, stop infrared image acquisition, otherwise collect the target infrared image of the surface of the water area to be treated, and calculate the grayscale change difference value of each pixel in the target infrared image. The grayscale step detection module is used to determine the direction of the most drastic grayscale change in the neighborhood of any pixel in the target infrared image, extract the grayscale values ​​of several pixels including the pixel along the direction to form a grayscale profile sequence of the pixel, calculate the degree of difference between adjacent grayscale values ​​in the grayscale profile sequence, and determine whether a grayscale step has occurred in the grayscale profile sequence based on the degree of difference. The image segmentation module is used to traverse all pixels in the target infrared image, identify pixels that have gray-level steps in the gray-level profile sequence, filter effective target pixels and background pixels, and statistically analyze the gradient magnitude distribution of effective target pixels in order to perform the maximum inter-class variance method to divide the garbage area on the surface of the water to be treated.

[0040] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0041] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0042] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0043] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. An infrared image segmentation method based on the Otsu's method, characterized in that, The specific steps include: Infrared sample images of multiple motion states were collected by an aquatic robot, and the robot's motion state parameters were recorded simultaneously at the time of image acquisition. For each infrared sample image, the absolute difference of the gradient magnitude of each pixel in the garbage step edge and the remaining water surface area was analyzed. For any motion state parameter, set a valid constraint based on the magnitude of its corresponding absolute difference. Record the motion state parameters that do not satisfy the valid constraints as invalid state parameters. Iterate through all motion state parameters to form a set of invalid state parameters. The motion state parameters of the water robot in the water area to be treated are collected. It is determined whether the motion state parameters fall into the set of invalid state parameters. If they do, the infrared image acquisition is stopped. Otherwise, the target infrared image of the surface of the water area to be treated is collected, and the grayscale change difference value of each pixel in the target infrared image is calculated. For any pixel in the target infrared image, determine the direction of the most drastic gray-level change in the neighborhood of the pixel, extract the gray-level values ​​of several pixels including the pixel along the direction to form a gray-level profile sequence of the pixel, calculate the degree of difference between adjacent gray-level values ​​in the gray-level profile sequence, and determine whether a gray-level step occurs in the gray-level profile sequence based on the degree of difference. Traverse all pixels in the target infrared image, identify pixels that undergo gray-level steps within the gray-level profile sequence, filter effective target pixels and background pixels, and statistically analyze the gradient magnitude distribution of effective target pixels to perform the maximum inter-class variance method to divide the garbage area on the surface of the water to be treated.

2. The infrared image segmentation method based on the Otsu's method according to claim 1, characterized in that, The motion state parameters include the three-axis acceleration values ​​of the water robot at the moment of infrared image acquisition and the light intensity, which together constitute a four-dimensional motion state parameter vector. The system controls the water robot to operate in various different motion states. In each motion state, infrared sample images are continuously acquired at the corresponding moment. At the same time as acquiring each infrared sample image, the four-dimensional motion state parameter vector corresponding to that acquisition moment is recorded through the inertial measurement unit.

3. The infrared image segmentation method based on the Otsu's method according to claim 2, characterized in that, The contour of the waste step edge is determined by manual annotation for any infrared sample image. The waste step edge specifically refers to the edge where the waste comes into contact with the water surface. The calculation of the gradient magnitude of the garbage step edge pixel includes, in any infrared sample image, for any garbage step edge pixel, calculating the first-order central difference of its gray value in the horizontal and vertical directions, taking the sum of the squares of the first-order central differences in the two directions based on the first-order central difference of the garbage step edge pixel, and taking the square root of the sum of squares to obtain the pixel gradient magnitude of the garbage step edge pixel. Traverse all garbage step edge pixels and determine the pixel gradient magnitude of all garbage step edge pixels in the infrared sample image; Determine the pixel gradient magnitude of each pixel in the remaining water surface area image within the infrared sample image, calculate the absolute difference between it and the pixel gradient magnitude of each garbage step edge in the infrared sample image, and select the smallest absolute difference for any pixel in the remaining water surface area image as the absolute difference between the pixel gradient magnitude of that pixel and the pixel gradient magnitude of the garbage step edge.

4. The infrared image segmentation method based on the Otsu's method according to claim 3, characterized in that, For any motion state parameter, if the absolute difference under the motion state parameter simultaneously satisfies the following conditions, then the motion state parameter is considered not to satisfy the valid constraints. Condition 1: All absolute differences under this motion state parameter have an absolute difference that is no greater than a preset difference threshold; Condition 2: The number of absolute differences is not greater than the preset difference threshold and is not less than the preset interference threshold; Traverse all motion state parameters to form an invalid state parameter set. The invalid state parameter set specifically includes sub-ranges of each state parameter. Each sub-range has a lower limit based on the minimum value of the corresponding invalid state parameter, and an upper limit set to a preset multiple of the lower limit.

5. The infrared image segmentation method based on the Otsu's method according to claim 4, characterized in that, The water robot is placed in the water area to be treated, and the current motion state parameters are collected. If any motion state parameter falls into the set of invalid state parameters, the infrared image acquisition is stopped. Otherwise, acquire the target infrared image of the surface of the water area to be processed. For any pixel in the target infrared image, the specific method for determining the direction of the most drastic grayscale change is as follows: take the first-order difference value of the pixel in the horizontal and vertical directions as the ordinate and abscissa respectively, calculate the arctangent function value corresponding to the ratio of the ordinate to the abscissa, take the arctangent function value as the direction angle, sample the pixel in the image space along the straight line indicated by the direction angle, and extract the grayscale values ​​of several consecutive pixels including the pixel to form the grayscale profile sequence.

6. The infrared image segmentation method based on the Otsu's method according to claim 5, characterized in that, For any pixel in the target infrared image, in its grayscale profile sequence, the difference between the grayscale values ​​of two adjacent pixels in the grayscale profile sequence is calculated, and the difference result is used as the first-order difference value. The entire grayscale profile sequence is traversed in this way to obtain a first-order difference sequence composed of first-order difference values. Extract the maximum and second largest absolute values ​​from the first-order difference sequence. If the ratio of the maximum absolute value to the second largest absolute value is not greater than a preset step significance ratio threshold, then it is determined that the pixel has not experienced a gray-level step in the gray-level profile sequence. Otherwise, extract the pixel corresponding to the maximum absolute value in the first-order difference sequence. Taking this pixel as the center, calculate the average gray value of all pixels on both sides of this pixel in the gray-scale profile sequence, as well as the coefficient of variation of the gray values ​​of the pixels on both sides. If the absolute difference between the average gray values ​​on both sides is not less than the preset average threshold, and the coefficient of variation of the gray values ​​of the pixels on both sides is not greater than the preset equilibrium threshold, then it is determined that a gray-scale step has occurred in the gray-scale profile sequence; otherwise, it is determined that no gray-scale step has occurred.

7. The infrared image segmentation method based on the Otsu's method according to claim 6, characterized in that, The specific method for filtering effective target pixels and background pixels is as follows: traverse all pixels in the target infrared image, determine the pixels corresponding to gray-level steps in all gray-level profile sequences, and take them as effective target pixels. The gray-level values ​​of the remaining pixels are taken as background pixels. The logic behind performing the maximum inter-class variance method is as follows: statistically analyze the gradient magnitude distribution histogram of all effective target pixels, take the gradient magnitude with the lowest frequency in the histogram as the reference base point, take the reference base point as the starting point of the candidate interval, and take the maximum gradient magnitude among all effective target pixels as the ending point of the candidate interval. Within the candidate interval, calculate the inter-class variance value corresponding to each candidate gradient magnitude, and take the candidate gradient magnitude corresponding to the maximum value of the inter-class variance as the optimal segmentation threshold. Pixels with gradient magnitudes of effective target pixels that are not less than the optimal segmentation threshold are designated as garbage step edge pixels; otherwise, they are designated as background region pixels. This method is used to divide the garbage region on the surface of the water body to be processed.

8. An infrared image segmentation system based on the Otsu's method, used to execute the infrared image segmentation method based on the Otsu's method according to any one of claims 1-7, characterized in that, include: The sample data acquisition module is used to acquire infrared sample images of multiple motion states through the water robot, and synchronously record the robot motion state parameters at the time of image acquisition. For each infrared sample image, the absolute difference of the gradient amplitude of each pixel in the garbage step edge and the remaining water surface area is analyzed. The working feature filtering module is used to set effective constraints for any motion state parameter based on its corresponding absolute difference magnitude, record the motion state parameters that do not meet the effective constraints as invalid state parameters, and traverse all motion state parameters to form a set of invalid state parameters. The grayscale difference analysis module is used to collect the motion state parameters of the water robot in the water area to be treated, determine whether its motion state parameters fall into the invalid state parameter set, if they fall into the invalid state parameter set, stop infrared image acquisition, otherwise collect the target infrared image of the surface of the water area to be treated, and calculate the grayscale change difference value of each pixel in the target infrared image. The grayscale step detection module is used to determine the direction of the most drastic grayscale change in the neighborhood of any pixel in the target infrared image, extract the grayscale values ​​of several pixels including the pixel along the direction to form a grayscale profile sequence of the pixel, calculate the degree of difference between adjacent grayscale values ​​in the grayscale profile sequence, and determine whether a grayscale step has occurred in the grayscale profile sequence based on the degree of difference. The image segmentation module is used to traverse all pixels in the target infrared image, identify pixels that have gray-level steps in the gray-level profile sequence, filter effective target pixels and background pixels, and statistically analyze the gradient magnitude distribution of effective target pixels in order to perform the maximum inter-class variance method to divide the garbage area on the surface of the water to be treated.

Images