A positioning detection method and system for ship signal flags

Abstract

The present invention discloses a positioning detection method for ship signal flags, which relates to the technical field of image detection and recognition. It includes: obtaining an image of a target ship and performing color restoration processing on the k-th frame image in the image; performing moving target segmentation processing on the k-th frame image to segment the moving targets in the k-th frame image; performing morphological opening operation on the k-th frame image; detecting the shape of each separated moving target in the k-th frame image to determine candidate signal flags; performing RGB color separation on the k-th frame image, enhancing a preset color, calculating the area proportion of the enhanced preset color, and identifying the signal flag from the candidate signal flags according to the area proportion result. The positioning detection method and system provided by the present invention are applicable to the detection and recognition of ship signal flags, realize the extraction of ship signal flag targets, and improve the reliability and stability of ship signal flag detection.

Description

Technical Field

[0001] The present invention relates to the technical field of image detection and recognition, and particularly to a method and system for positioning and detecting ship signal flags. Background Art

[0002] Ship signal flags are an internationally common ship communication system, which is specifically used for communication between ships or between ships and the shore. This communication method can cross language barriers and indicate the intentions of ships.

[0003] However, ship signal flag communication is interfered by various complex factors in the real environment, and the observation of ship signal flags requires a large amount of manual and time investment. In the case of personnel fatigue or absence, it is easy to miss important communication signals or fail to accurately understand the meaning of signal flags. Summary of the Invention

[0004] The present invention provides a method, system, readable storage medium and computer device for positioning and detecting ship signal flags, in order to solve the problems that the recognition of ship signal flags in the prior art is vulnerable to interference and it is easy to miss and misunderstand important communication signals.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0006] A method for positioning and detecting ship signal flags, comprising:

[0007] Obtain an image of a target ship, and perform color restoration processing on the k-th frame image in the image based on an automatic white balance algorithm with a dynamic threshold;

[0008] Perform moving target segmentation processing on the k-th frame image after color restoration processing to segment the moving targets in the k-th frame image;

[0009] Perform morphological opening operation on the k-th frame image after moving target segmentation to separate each moving target from each other;

[0010] Detect the shape of each separated moving target in the k-th frame image through a shape detection algorithm, and determine candidate signal flags according to the shape detection results;

[0011] Perform RGB color separation on the k-th frame image after shape detection, enhance a preset color, calculate the area ratio of the enhanced preset color, and identify signal flags from the candidate signal flags according to the area ratio results.

[0012] In order to solve the above technical problems, the present invention can also adopt the following technical solutions:

[0013] A system for positioning and detecting ship signal flags, comprising:

[0014] A color restoration unit for obtaining an image of a target ship and performing color restoration processing on the k-th frame image in the image based on an automatic white balance algorithm with a dynamic threshold;

[0015] A moving target segmentation unit for performing moving target segmentation processing on the k-th frame image after color restoration processing to segment the moving targets in the k-th frame image;

[0016] A moving target separation unit for performing morphological opening operation on the k-th frame image after moving target segmentation to separate each of the moving targets from each other;

[0017] A shape detection unit for detecting the shape of each separated moving target in the k-th frame image through a shape detection algorithm and determining candidate signal flags according to the shape detection results;

[0018] A color detection unit for performing RGB color separation on the k-th frame image after shape detection, enhancing a preset color, calculating the area ratio of the enhanced preset color, and identifying a signal flag from the candidate signal flags according to the area ratio result.

[0019] To solve the above technical problems, the present invention can also adopt the following technical solutions:

[0020] A readable storage medium storing at least one program, where the at least one program, when executed, is used to implement the positioning and detection method of the ship signal flag described in the above technical solution.

[0021] To solve the above technical problems, the present invention can also adopt the following technical solutions:

[0022] A computer device, the computer device includes: a processor and a memory, the memory is used to store at least one program, and the processor is used to read the at least one program to implement the positioning and detection method of the ship signal flag described in the above technical solution.

[0023] The positioning and detection method and system provided by the present invention are applicable to the detection and recognition of ship signal flags. Aiming at the technical difficulty that the color characteristics of ship signal flags are easily affected by light, an image preprocessing method is first proposed. By introducing an automatic white balance algorithm based on dynamic thresholds, the image contrast is improved, the original color of the image is restored, and the influence of light is weakened. Then, through the moving target segmentation technology and the morphological opening operation method, the target candidate areas where the ship signal flags may exist are obtained. Then, the shape features of the target candidate areas are extracted by a positioning and detection method that combines shape and color information. After obtaining the shape candidate areas, the RGB color ratio method is used to highlight the characteristic colors of the signal flags. By fusing shape and color segmentation, the interference objects with only the same color or only the same shape in the image are filtered out, thereby realizing the extraction of ship signal flag targets. Compared with the existing image recognition and detection schemes with poor adaptability to light changes, this method improves the robustness of the entire system to changes in light conditions by introducing the idea of color invariance in the image preprocessing stage. The moving target segmentation method is used to solve the problem that the gray distribution of the signal flags is uneven and difficult to segment. The morphological processing method is used to obtain the candidate target areas according to the connection characteristics between the flags and the hull. The detection and positioning method that fuses the characteristic shape and characteristic color information of the signal flags can achieve a higher accuracy than a single method, greatly reducing the amount of manual participation in the process of communication between ships using signal flags, avoiding signal omission caused by personnel fatigue or absence problems, and improving the reliability and stability of ship signal flag detection.

[0024] Advantages of additional aspects of the present invention will be partly given in the following description, partly become obvious from the following description, or be understood through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a schematic flow chart provided for an embodiment of the positioning and detection method of the present invention;

[0026] Figure 2 It is a schematic diagram of the Douglas-Peucker algorithm approximating a known curve;

[0027] Figure 3 It is a schematic diagram of the Douglas-Peucker algorithm detection provided for an embodiment of the positioning and detection method of the present invention;

[0028] Figure 4 It is a schematic flow chart of ship signal flag detection based on shape information provided for an embodiment of the positioning and detection method of the present invention;

[0029] Figure 5 It is a schematic flow chart of ship signal flag detection based on color information provided for an embodiment of the positioning and detection method of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0030] The following clearly and completely describes the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the protection scope of the present invention.

[0031] As Figure 1 shown, it is a schematic flowchart provided by an embodiment of the positioning detection method of the present invention. The positioning detection method of the ship signal flag includes:

[0032] S1, obtain an image of the target ship, and perform color restoration processing on the k-th frame image in the image based on the automatic white balance algorithm with a dynamic threshold.

[0033] It should be noted that the color information of the ship signal flag has a great impact on the subsequent detection stage. If the correct color cannot be obtained, it is difficult to ensure the accuracy of the subsequent detection and recognition results. The color information is sensitive to changes in illumination. Therefore, in the preprocessing stage, an automatic white balance algorithm based on a dynamic threshold is used to weaken the influence of illumination on the image, restore the original color of the signal flag, and improve the quality of the image frame input to the subsequent positioning detection stage.

[0034] Among them, k is any frame in the image.

[0035] S2, perform moving target segmentation processing on the k-th frame image after color restoration processing, and segment out the moving targets in the k-th frame image.

[0036] It should be noted that the ship signal flag is usually composed of a combination of multiple characteristic colors, and it is difficult to achieve good segmentation results by the global threshold segmentation method. For example, since the background is basically unchanged in the ocean scene, the ship and the signal flag on it are moving objects in the scene. Therefore, moving target segmentation technology can be used to extract the foreground targets, differentiate the current image frame from the background model, and calculate the area that deviates from the background beyond a certain threshold as the moving target.

[0037] S3, perform morphological opening operation on the k-th frame image after moving target segmentation to separate each moving target from each other.

[0038] It should be noted that the ship signal flag is only connected to the ship's hull by a flagpole or a rope, and the connection part is very narrow compared to other foreground images. Therefore, in the preprocessing stage, the white balance algorithm can be mainly used to restore the image color, and the morphological opening operation can be used to separate the ship signal flag from the hull to form independent regions for subsequent processing.

[0039] It should be understood that morphological methods include several common operators such as dilation, erosion, opening, and closing. Among them, dilation and erosion operators are the basis of all composite morphological analyses. The dilation operator expands image information, thickens lines, and fills gaps and pores in the image. The erosion operator can converge image information, thin particles and lines in the image, and enlarge gaps and pores.

[0040] Morphological opening operation suppresses the contours in the image and makes them smooth through a serial composite transformation of first eroding and then dilating. The opening operation can remove isolated points on the image edge, eliminate thin protrusions on the contour at the same time, break narrow connections, and make the target contour incomplete. For the direction of ship signal flag detection, it can just break the connection between the flag and the ship part. Therefore, morphological opening operation can be selected in the last step of preprocessing to separate the signal flag from the ship image, facilitating subsequent shape and color detection.

[0041] For example, the parameters of the morphological opening operation can be adjusted through empirical values to separate the ship signal flag from the hull.

[0042] S4. Detect the shape of each separated moving target in the k-th frame image through a shape detection algorithm, and determine candidate signal flags according to the shape detection results.

[0043] It should be noted that ship signal flags have four specific shapes: rectangle, swallowtail, triangle, and trapezoid. Therefore, non-target objects that do not have these four specific shapes are excluded through a shape-based detection method, achieving the purpose of removing non-target areas and reducing the amount of subsequent operations. However, it is difficult to separate interfering objects with the same shape as the target only relying on shape information. Therefore, after shape detection, a color-based detection method is introduced to highlight the region of interest through the specific color of the signal flag, further removing interfering objects and improving the real-time performance of the system.

[0044] For example, the Douglas-Peucker algorithm can be used for detection.

[0045] S5. Perform RGB color separation on the k-th frame image after shape detection, enhance the preset color, calculate the area ratio of the preset color after enhancement processing, and identify the signal flag from the candidate signal flags according to the area ratio result.

[0046] It should be noted that ship signal flags have five specific colors: red, yellow, blue, white, and black. Red, yellow, blue, and black can be used as preset colors for identification. Ship signal flags are designed with a unified standard for shape and size, and at the same time, distinct and easily detectable colors are purposefully selected. This method fully considers this characteristic of ship signal flags and combines shape operators and color operators to locate and detect the target. The series structure of this shape analysis and color analysis makes full use of the unique shape and color characteristics of ship signal flags, which can effectively improve the accuracy of target location detection and enhance the robustness of the system.

[0047] It should be understood that the area ratio refers to the ratio of the area of each color to the area of the candidate signal flag where it is located.

[0048] For example, assume a signal flag has two colors, red and black. Then, the area ratio of red in the candidate signal flag and the area ratio of black in the candidate signal flag can be obtained respectively. Assume that the area ratio of red exceeds 40%, and the area ratio of black exceeds 40%. Then, this candidate signal flag can be considered as a signal flag.

[0049] The location detection method provided in this embodiment is applicable to the detection and identification of ship signal flags. Aiming at the technical difficulty that the color characteristics of ship signal flags are easily affected by light, an image preprocessing method is first proposed. By introducing an automatic white balance algorithm based on dynamic thresholds, the image contrast is improved, the original color of the image is restored, the influence of light is weakened. Then, through the moving target segmentation technology and the morphological opening operation method, the target candidate areas where ship signal flags may exist are obtained. Then, the shape features of the target candidate areas are extracted by the location detection method combining shape and color information. After obtaining the shape candidate areas, the RGB color ratio method is used to highlight the characteristic colors of the signal flags. By fusing shape and color segmentation, the interference objects with only the same color or only the same shape in the image are filtered out, thus realizing the extraction of ship signal flag targets. Compared with the existing image recognition and detection schemes with poor adaptability to light changes, this method improves the robustness of the entire system to changes in light conditions by introducing the idea of color invariance in the image preprocessing stage, uses the moving target segmentation method to solve the problem that the gray distribution of the signal flag is uneven and difficult to segment, uses the morphological processing method according to the connection characteristics between the flag and the hull to obtain the candidate target areas, and the detection and location method that fuses the characteristic shape and characteristic color information of the signal flag can achieve better accuracy than a single method, greatly reducing the amount of manual participation in the process of using signal flags for communication between ships, avoiding signal missed detection caused by personnel fatigue or absence problems, and improving the reliability and stability of ship signal flag detection.

[0050] Optionally, in some possible implementation manners, the automatic white balance algorithm based on dynamic thresholds performs color restoration processing on the k-th frame image in the video, specifically including:

[0051] Convert the k-th frame image to the YCbCr color space;

[0052] Calculate the pixel average value of the C b channel, the pixel average value of the C r channel, the pixel mean square deviation of the C b channel and the pixel mean square deviation of the C r channel, and determine the candidate reference white point according to the calculation results;

[0053] Calculate the brightness value of each candidate reference white point, and select the reference white point from the candidate reference white points according to the brightness value;

[0054] Determine the gain values of the C r channel, the C g channel and the C b channel according to the reference white point;

[0055] Perform color restoration processing on the k-th frame image according to the gain values.

[0056] For example, the pixel points of each channel in the k-th frame image can be multiplied by the gain values of the corresponding channels respectively to complete the color restoration processing.

[0057] Optionally, in some possible implementation manners, selecting the reference white point from the candidate reference white points according to the brightness value specifically includes:

[0058] Sort all the candidate reference white points in descending order of brightness value, and use the candidate reference white points with the top preset proportion of brightness value rankings as the reference white points.

[0059] For example, the candidate reference white points with the top 10% brightness value rankings can be selected as the reference white points.

[0060] Optionally, in some possible implementation manners, determining the candidate reference white point according to the calculation results specifically includes:

[0061] Determine the pixel point (i, j) that satisfies the following formula as the candidate reference white point:

[0062] |C b (i, j)-(A b +T b ×sign(A b ))|<1.5×T b

[0063] |C r (i, j)-(1.5×A r +T r ×sign(A r ))|<1.5×T r

[0064] Among them, C b (i, j) is the pixel value of the pixel point (i, j) in C b channel, and C r (i, j) is the pixel value of the pixel point (i, j) in C r channel, A b is the average pixel value of the C b channel, and A r is the average pixel value of the C r channel, T b is the mean square error of the pixels of the C b channel, and T r is the mean square error of the pixels of the C r channel, and sign represents a mathematical operator.

[0065] Optionally, in some possible implementation manners, the gain value of each channel is determined according to the following formula:

[0066] Gain r = Y max / Ave r

[0067] Gain g = Y max / Ave g

[0068] Gain b = Y max / Ave b

[0069] Among them, Gain r is the gain value of the C r channel, Gain g is the gain value of the C g channel, Gain b is the gain value of the C b channel, Y max is the maximum brightness value of the k-th frame image, and Ave r is the average pixel value of the reference white point in the C r channel, and Ave g is the average pixel value of the reference white point in the C g channel, and Ave b is the average pixel value of the reference white point in the C b channel.

[0070] Optionally, in some possible implementation manners, the k-th frame image is subjected to color restoration processing according to the following formula:

[0071]

[0072] Among them, is the input image, is the output image after color restoration processing.

[0073] Optionally, in some possible implementation manners, perform moving target segmentation processing on the k-th frame image after color restoration processing to segment the moving target in the k-th frame image, specifically including:

[0074] Read the first N frame images in the video to construct a background model;

[0075] Compare the pixel value of each pixel point in the k-th frame image with the pixel value of the corresponding pixel point in the background model;

[0076] Mark the pixel points within the acceptable range as the background, and mark the pixel points outside the acceptable range as the foreground to obtain the moving target.

[0077] Optionally, the background model can be established by the single Gaussian method. This method believes that the probability of a certain pixel gray value appearing in the background image satisfies the Gaussian distribution. Then, the attributes of a specific pixel in the constructed background model should include two parameters: the mean value μ and the variance σ.

[0078] It should be noted that when constructing the background model using the above embodiments, k should be greater than N.

[0079] Optionally, in some possible implementation manners, the pixel points that satisfy the following formula are within the acceptable range:

[0080]

[0081] Among them, B is the background image, I is the k-th frame image, (x, y) is the pixel point, σ is the variance of the background model, and T is the preset threshold.

[0082] It should be noted that since σ changes very little after the background is updated, σ is usually not updated after the background is updated.

[0083] Optionally, in some possible implementation manners, update the background model according to the following formula:

[0084] B t (x, y) = a · B t-1 (x, y) + (1 - a) · I t (x, y)

[0085] Among them, a is the update parameter, and t is the number of times of moving target segmentation processing.

[0086] Optionally, in some possible implementation manners, the shape of each separated moving object in the k-th frame image is detected through a shape detection algorithm, and candidate signal flags are determined according to the shape detection result, specifically including:

[0087] Detect the contour of each separated moving object in the k-th frame image, take the two points with the largest distance on the contour as separation points, and separate the contour into two curves, namely a first end and a second end;

[0088] Use the Douglas-Peucker algorithm to approximate the shapes of the curves at the first end and the second end respectively to obtain contour approximation curves;

[0089] Judge whether the geometric parameter information of the contour approximation curve meets the geometric characteristics of the signal flag, and determine the moving object that meets the geometric characteristics of the signal flag as a candidate signal flag.

[0090] It should be noted that the implementation of the Douglas-Peucker algorithm is only based on the number of boundaries of the object, has rotation and translation invariance, and by setting a distance threshold, the problem of slight corner folding during the fluttering of the signal flag can be overcome.

[0091] The schematic diagram of using the Douglas-Peucker algorithm to approximate the shape of an original curve is as Figure 2 shown. First, mark the starting point and the ending point on the curve, find the point farthest from the line segment between the two points as the ending point, judge the distance between the ending point and the line segment. If the distance is greater than the set threshold, mark the ending point, and continue the recursive process between the starting point and the ending point, and between the ending point and the ending point; if the distance is less than the threshold, the recursion ends, and all unmarked points are discarded.

[0092] Since the contour of the target candidate region segmented in the preprocessing stage is a closed curve and the Douglas-Peucker algorithm cannot be directly applied, it is necessary to find the two points with the largest distance on the contour as separation points, and separate the entire contour into two curves, namely the upper end and the lower end, for shape approximation respectively. As Figure 3 shown in the schematic closed contour, after using the Douglas-Peucker algorithm to complete the recursive operation, the approximated figure is a quadrilateral.

[0093] The flowchart of using the Douglas-Peucker algorithm to perform shape analysis on ship signal flags is as Figure 4 shown, where both the maximum distance and the initial value of the maximum value are set to 0.

[0094] First, randomly select two points from the set of edge points of the contour, and determine whether the distance between the two points is greater than the maximum distance between two points on the currently known contour. If not, reselect the points; if so, update the maximum distance. After all the edge points are traversed, store the two points A and B with the maximum distance, and obtain the line segment between A and B.

[0095] Then, perform shape approximation on the upper curve and the lower curve respectively. Taking the upper curve as an example for illustration, the lower curve is the same and will not be elaborated. Calculate the distance between the points on the upper curve and the line segment AB, and determine whether it is greater than the maximum distance between the points on the known upper curve and the line segment AB. If not, reselect the points; otherwise, update the maximum distance. After all the points are traversed, obtain the point D that is farthest from the line segment AB. Determine whether the maximum distance is less than the preset distance threshold. If it is not less than, continue to find points on the line segments AD and DB to judge the distance; if it is less than, output the line segment between the two points as the final curve.

[0096] The polygon detection method based on the Douglas - Peucker algorithm outputs the contour approximation curve of the target candidate region, from which information such as the number of sides, length, and angle of the curve can be obtained. Considering the possible deformation problems of the signal flag during the ship's driving process, distinguishing between trapezoids and rectangles may cause misjudgment of the target, and there is a subsequent color operator to continuously screen the target. Therefore, in this method, during the shape detection stage, only the candidate targets that may be rectangles, swallowtails, triangles, and trapezoids are extracted according to the number of boundaries and input to the subsequent color detection stage for further analysis of the target.

[0097] Improve the Douglas - Peucker algorithm in the above way so that it can be applied to closed contours, which can improve the accuracy of shape recognition.

[0098] Optionally, in some possible implementation manners, perform RGB color separation on the k - th frame image after shape detection, perform enhancement processing on the preset color, calculate the area ratio of the preset color after enhancement processing, and identify the signal flag from the candidate signal flags according to the area ratio result, specifically including:

[0099] Determine the component colors of the signal flag, use each component color as the preset color, and determine the channel ratio information of each preset color;

[0100] Perform RGB color separation on the k - th frame image after shape detection according to the channel ratio information of each preset color to obtain the color separation image corresponding to each preset color;

[0101] Respectively judge whether the pixel value of each pixel point in each color separation image is greater than the pixel value of the corresponding preset color. If so, set the pixel value in the color separation image to 1; otherwise, set it to 0.

[0102] Judge the ratio of the area where the pixel value is set to 1 in each color-separated image to the area of the corresponding candidate signal flag area respectively;

[0103] Identify the signal flag from the candidate signal flags according to the ratio.

[0104] It should be noted that the main task in the color analysis stage is to detect the signal flag with characteristic colors in the candidate targets with specific shapes. The ship signal flag is composed of one or a combination of five colors: red, yellow, blue, white, and black. Analyzing the composition colors of the current 40 international signal flags, white will not appear alone. Therefore, red, yellow, blue, and black can be selected as the characteristic colors in the color detection stage.

[0105] Since the original image frames obtained by the perception terminal are usually in RGB format and the non-linear conversion operation amount of the color space is large, to ensure the real-time performance of the system, preferably, the image is analyzed in the RGB space.

[0106] For example, for the four characteristic colors of the ship signal flag, the characteristic color information is enhanced by the following methods:

[0107] 1. For red, the ratio value of the R channel and the G channel is used to highlight the red feature.

[0108] 2. For yellow, the ratio value of the G channel and the B channel is used to highlight the yellow feature.

[0109] 3. For blue, the ratio value of the B channel and the G channel is used to highlight the blue feature.

[0110] 4. For black, the R, G, and B channel values of the black part in the image are all 0. Therefore, the input image frame is first converted into a grayscale image, and then processed by the threshold segmentation method.

[0111] After completing the color segmentation, the candidate regions without characteristic colors can be excluded. At the same time, analyze the color pattern of the signal flag to obtain the recognition result.

[0112] For example, the proportion of a single characteristic color on the entire flag surface is generally greater than 1 / 5. Therefore, after completing the color segmentation, further compare the area of the characteristic color with the area of the candidate target. If the proportion of any one of the characteristic colors exceeds 1 / 5, it can be considered as the ship signal flag target.

[0113] For example, as Figure 5 shown, an exemplary schematic diagram of the ship signal flag detection process based on color information is provided, where Tr, Ty, Tb, and Th are the corresponding color thresholds set according to empirical values.

[0114] The method of color segmentation for candidate targets based on the color ratio between different channels has higher robustness compared to the method of performing threshold segmentation on a single color channel alone.

[0115] Optionally, in some possible implementation manners, it may include all or part of the above various implementation manners.

[0116] The present invention also provides a positioning detection system for ship signal flags, including:

[0117] A color restoration unit for obtaining an image of a target ship and performing color restoration processing on the k-th frame image in the image based on an automatic white balance algorithm with a dynamic threshold;

[0118] A moving target segmentation unit for performing moving target segmentation processing on the k-th frame image after color restoration processing to segment the moving targets in the k-th frame image;

[0119] A moving target separation unit for performing morphological opening operation on the k-th frame image after moving target segmentation to separate each moving target from each other;

[0120] A shape detection unit for detecting the shape of each separated moving target in the k-th frame image through a shape detection algorithm and determining candidate signal flags according to the shape detection results;

[0121] A color detection unit for performing RGB color separation on the k-th frame image after shape detection, enhancing a preset color, calculating the area ratio of the enhanced preset color, and identifying the signal flag from the candidate signal flags according to the area ratio result.

[0122] The positioning and detection system provided in this embodiment is applicable to the detection and recognition of ship signal flags. Aiming at the technical difficulty that the color characteristics of ship signal flags are easily affected by light, an image preprocessing method is first proposed. By introducing an automatic white balance algorithm based on dynamic thresholds, the image contrast is improved, the original color of the image is restored, the influence of light is weakened. Then, through the moving target segmentation technology and the morphological opening operation method, the target candidate areas where the ship signal flags may exist are obtained. Then, the shape features of the target candidate areas are extracted by a positioning and detection method that combines shape and color information. After obtaining the shape candidate areas, the RGB color ratio method is used to highlight the characteristic colors of the signal flags. By fusing shape and color segmentation, the interfering objects with only the same color or only the same shape in the image are filtered out, thus realizing the extraction of ship signal flag targets. Compared with the existing image recognition and detection schemes with poor adaptability to light changes, this method improves the robustness of the entire system to changes in light conditions by introducing the idea of color invariance in the image preprocessing stage, uses the moving target segmentation method to solve the problem that the gray distribution of the signal flags is uneven and difficult to segment, uses the morphological processing method according to the connection characteristics between the flag and the hull to obtain the candidate target areas, and the detection and positioning method that fuses the characteristic shape and characteristic color information of the signal flags can achieve higher accuracy than a single method, greatly reducing the amount of manual participation in the process of using signal flags for communication between ships, avoiding signal missed detection caused by personnel fatigue or absence problems, and improving the reliability and stability of ship signal flag detection.

[0123] Optionally, in some possible implementation manners, the color restoration unit is specifically configured to convert the k-th frame image to the YCbCr color space; calculate the pixel average value of the C b channel, the pixel average value of the C r channel, the pixel mean square deviation of the C b channel, and the pixel mean square deviation of the C r channel, determine the candidate reference white points according to the calculation results; calculate the brightness values of each candidate reference white point, and select the reference white point from the candidate reference white points according to the brightness values; determine the gain values of the C r channel, the C g channel, and the C b channel according to the reference white point; perform color restoration processing on the k-th frame image according to the gain values.

[0124] Optionally, in some possible implementation manners, the color restoration unit is specifically configured to sort all the candidate reference white points in descending order of brightness values, and use the candidate reference white points with the top preset proportion of brightness value rankings as the reference white points.

[0125] Optionally, in some possible implementation manners, the color restoration unit is specifically configured to determine the pixel points (i, j) that satisfy the following formula as the candidate reference white points:

[0126] |C b (i, j) - (A b + T b × sign(A b ))| < 1.5 × T b

[0127] |C r (i, j) - (1.5 × A r + T r × sign(A r ))| < 1.5 × T r

[0128] Wherein, C b (i, j) is the pixel value of the pixel point (i, j) in C b channel, C r (i, j) is the pixel value of the pixel point (i, j) in C r channel, A b is the average pixel value of C b channel, A r is the average pixel value of C r channel, T b is the mean square error of pixels in C b channel, T r is the mean square error of pixels in C r channel, sign represents a mathematical operator.

[0129] Optionally, in some possible implementation manners, the color restoration unit is specifically configured to determine the gain value of each channel according to the following formula:

[0130] Gain r = Y max / Ave r

[0131] Gain g = Y max / Ave g

[0132] Gain b = Y max / Ave b

[0133] Wherein, Gain r is the gain value of C r channel, Gain g is the gain value of C g channel, Gain b is the gain value of C b channel, Y max is the maximum brightness value of the k-th frame image, Aver is the average pixel value of the reference white point in the C r channel, Ave g is the average pixel value of the reference white point in the C g channel, Ave b is the average pixel value of the reference white point in the C b channel.

[0134] Optionally, in some possible embodiments, the color restoration unit is specifically configured to perform color restoration processing on the k-th frame image according to the following formula:

[0135]

[0136] where, is the input image, is the output image after color restoration processing.

[0137] Optionally, in some possible embodiments, the moving target segmentation unit is specifically configured to read the first N frame images in the video to construct a background model; compare the pixel value of each pixel point in the k-th frame image with the pixel value of the corresponding pixel point in the background model; mark the pixel points within the acceptable range as the background, and mark the pixel points outside the acceptable range as the foreground to obtain the moving target.

[0138] Optionally, in some possible embodiments, the moving target segmentation unit is specifically configured to determine whether a pixel point is within the acceptable range according to the following formula:

[0139]

[0140] where, B is the background image, I is the k-th frame image, (x, y) is the pixel point, σ is the variance of the background model, T is a preset threshold, and the pixel points satisfying the above formula are within the acceptable range.

[0141] Optionally, in some possible embodiments, the moving target segmentation unit is further configured to update the background model according to the following formula:

[0142] B t (x, y) = a · B t-1 (x, y) + (1 - a) · I t (x, y)

[0143] where, a is the update parameter, and t is the number of moving target segmentation processing times.

[0144] Optionally, in some possible embodiments, the shape detection unit is specifically configured to detect the contour of each separated moving object in the k-th frame image, take the two points with the largest distance on the contour as separation points, and separate the contour into two curves, i.e., a first end and a second end; use the Douglas-Peucker algorithm to approximate the shapes of the curves at the first end and the second end respectively to obtain contour approximation curves; determine whether the geometric parameter information of the contour approximation curves meets the geometric characteristics of a signal flag, and determine the moving object that meets the geometric characteristics of the signal flag as a candidate signal flag.

[0145] Optionally, in some possible embodiments, the color detection unit is specifically configured to determine the component colors of the signal flag, use each component color as a preset color, and determine the channel ratio information of each preset color; perform RGB color separation on the k-th frame image after shape detection according to the channel ratio information of each preset color to obtain a color separation image corresponding to each preset color; respectively determine whether the pixel value of each pixel point in each color separation image is greater than the pixel value of the corresponding preset color. If so, set the pixel value in the color separation image to 1, otherwise set it to 0; respectively determine the ratio of the area of the region where the pixel value is set to 1 in each color separation image to the area of the candidate signal flag region where it is located; identify the signal flag from the candidate signal flags according to the ratio.

[0146] Optionally, in some possible embodiments, it may include all or part of the above embodiments.

[0147] It can be understood that the above embodiments are product embodiments corresponding to the prior method embodiments. Therefore, the description of the product embodiments can refer to the prior method embodiments and will not be elaborated here.

[0148] The present invention also provides a readable storage medium, in which at least one program is stored, and when the at least one program is executed, it is used to implement the positioning and detection method of ship signal flags disclosed in any of the above embodiments and their combinations.

[0149] The present invention also provides a computer device, which includes: a processor and a memory. The memory is used to store at least one program, and the processor is used to read the at least one program to implement the positioning and detection method of ship signal flags disclosed in any of the above embodiments and their combinations.

[0150] A computer refers to a device that at least has a processor and a memory and can perform data operations, including not only a traditional computer but also any other form of device with the above functions and structures.

[0151] It should be understood that in the description of this specification, the reference to terms such as "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the partial features of different embodiments or examples.

[0152] Of course, without departing from the spirit and essence of the present invention, those skilled in the art can make various corresponding changes and deformations according to the present invention, but these changes and deformations should all fall within the protection scope of the claims of the present invention.

Claims

1. A method for positioning and detecting ship signal flags, characterized in that, it includes: Obtain an image of the target ship, and perform color restoration processing on the k-th frame image in the image based on an automatic white balance algorithm with a dynamic threshold; Perform moving target segmentation processing on the k-th frame image after color restoration processing to segment the moving targets in the k-th frame image; Perform morphological opening operation on the k-th frame image after moving target segmentation to separate each of the moving targets from each other; Detect the shape of each separated moving target in the k-th frame image through a shape detection algorithm, and determine candidate signal flags according to the shape detection results; Perform RGB color separation on the k-th frame image after shape detection, perform enhancement processing on a preset color, calculate the area ratio of the preset color after enhancement processing, and identify the signal flag from the candidate signal flags according to the area ratio result; Performing color restoration processing on the k-th frame image in the image based on an automatic white balance algorithm with a dynamic threshold specifically includes: Convert the k-th frame image to the YCbCr color space; Calculate C separately b the average pixel value of the C r the average pixel value of the C b the mean square error of pixels of the C r the mean square error of pixels of the C, and determine the candidate reference white point according to the calculation results; Calculate the brightness values of each candidate reference white point, and select a reference white point from the candidate reference white points according to the brightness values; Determine C based on the reference white point r channel, C g channel and C b the gain value of the channel; Perform color restoration processing on the k-th frame image according to the gain value; Selecting a reference white point from the candidate reference white points according to the brightness value specifically includes: Sort all the candidate reference white points in descending order according to the brightness value, and use the candidate reference white points with the top preset proportion of brightness values as the reference white points; Determine candidate reference white points according to the calculation results, specifically including: Determine the pixel points (i, j) that satisfy the following formula as candidate reference white points: |C b (i, j)-(A b +T b × sign(A b ))| < 1.5 × T b |C r (i, j) - (1.5 × A r + T r × sign(A r )) | < 1.5 × T r Among them, C b (i, j) is the pixel value of the pixel point (i, j) in C b channel, and C r (i, j) is the pixel value of the pixel point (i, j) in C r channel. A b is the average pixel value of the C b channel, and A r is the average pixel value of the C r channel. T b is the mean square error of the pixels of the C b channel, and T r is the mean square error of the pixels of the C r channel. sign represents a mathematical operator.

2. The method for positioning and detecting ship signal flags according to claim 1, characterized in that, Determine the gain value of each channel according to the following formula: Gain r = Y max / Ave r Gain g = Y max / Ave g Gain b = Y max / Ave b Among them, Gain r is the gain value of the C r channel, Gain g is the gain value of the C g channel, Gain b is the gain value of the C b channel, Y max is the maximum brightness value of the k-th frame image, Ave r is the pixel average value of the reference white point in the C r channel, Ave g is the pixel average value of the reference white point in the C g channel, Ave b is the pixel average value of the reference white point in the C b channel.

3. The method for positioning and detecting ship signal flags according to claim 2, characterized in that, Perform color restoration processing on the k-th frame image according to the following formula: Among them, is the input image, is the output image after color restoration processing.

4. The method for positioning and detecting ship signal flags according to claim 1, characterized in that, Performing moving target segmentation processing on the k-th frame image after color restoration processing to segment the moving targets in the k-th frame image specifically includes: Read the first N frame images in the image to construct a background model; Compare the pixel value of each pixel point in the k-th frame image with the pixel value of the corresponding pixel point in the background model; Mark the pixel points within the acceptable range as the background, and mark the pixel points outside the acceptable range as the foreground to obtain the moving target.

5. The method for positioning and detecting ship signal flags according to claim 4, characterized in that, The pixel points that satisfy the following formula are within the acceptable range: where B is the background image, I is the k-th frame image, (x, y) is the pixel point, σ is the variance of the background model, and T is a preset threshold.

6. The method for positioning and detecting ship signal flags according to claim 5, characterized in that, Update the background model according to the following formula: B t (x, y) = a·B t-1 (x, y) + (1 - a)·I t (x, y) where a is the update parameter and t is the number of times of moving target segmentation processing.

7. The method for positioning and detecting ship signal flags according to any one of claims 1 to 6, characterized in that, the shape of each separated moving object in the k-th frame image is detected by a shape detection algorithm, and candidate signal flags are determined according to the shape detection results, specifically including: detect the contour of each separated moving object in the k-th frame image, take the two points with the largest distance on the contour as separation points, and separate the contour into two curves, namely a first end and a second end; use the Douglas-Peucker algorithm to respectively approximate the shapes of the curves at the first end and the second end to obtain contour approximation curves; judge whether the geometric parameter information of the contour approximation curve meets the geometric characteristics of the signal flag, and determine the moving object that meets the geometric characteristics of the signal flag as a candidate signal flag.

8. The method for positioning and detecting ship signal flags according to any one of claims 1 to 6, characterized in that, perform RGB color separation on the k-th frame image after shape detection, perform enhancement processing on a preset color, calculate the area ratio of the preset color after enhancement processing, and identify the signal flag from the candidate signal flags according to the area ratio result, specifically including: determine the component colors of the signal flag, use each of the component colors as the preset color, and determine the channel ratio information of each preset color; perform RGB color separation on the k-th frame image after shape detection according to the channel ratio information of each preset color to obtain a color separation image corresponding to each preset color; respectively judge whether the pixel value of each pixel point in each color separation image is greater than the pixel value of the corresponding preset color. If so, set the pixel value in the color separation image to 1, otherwise set it to 0; respectively judge the ratio of the area of the region where the pixel value is set to 1 in each color separation image to the area of the corresponding candidate signal flag region; identify the signal flag from the candidate signal flags according to the ratio.

9. A positioning and detecting system for ship signal flags, characterized in that, comprising: a color restoration unit, configured to obtain an image of a target ship and perform color restoration processing on the k-th frame image in the image based on an automatic white balance algorithm with a dynamic threshold; a moving object segmentation unit, configured to perform moving object segmentation processing on the k-th frame image after color restoration processing to segment the moving objects in the k-th frame image; a moving object separation unit, configured to perform morphological opening operation on the k-th frame image after moving object segmentation to separate each moving object from each other; a shape detection unit, configured to detect the shape of each separated moving object in the k-th frame image by a shape detection algorithm and determine candidate signal flags according to the shape detection results; a color detection unit, configured to perform RGB color separation on the k-th frame image after shape detection, perform enhancement processing on a preset color, calculate the area ratio of the preset color after enhancement processing, and identify the signal flag from the candidate signal flags according to the area ratio result; The color restoration unit is specifically configured to convert the k-th frame image to the YCbCr color space; calculate the pixel average value of the C b channel, the pixel average value of the C r channel, the pixel mean square error of the C b channel, and the pixel mean square error of the C r channel, and determine a candidate reference white point according to the calculation results; calculate the brightness value of each candidate reference white point, and select a reference white point from the candidate reference white points according to the brightness value; Determine C according to the reference white point r channel, C g channel and C b the gain value of the channel; perform color restoration processing on the k-th frame image according to the gain value; The color restoration unit is specifically configured to sort all the candidate reference white points in descending order of the brightness value, and use the candidate reference white points with the top preset proportion of the brightness value ranking as the reference white points; The color restoration unit is specifically configured to determine the pixel points (i, j) that satisfy the following formula as candidate reference white points: |C b (i, j)-(A b +T b × sign(A b ))| < 1.5 × T b |C r (i, j) - (1.5 × A r + T r × sign(A r )) | < 1.5 × T r Among them, C b (i, j) is the pixel value of the pixel point (i, j) in C b channel, and C r (i, j) is the pixel value of the pixel point (i, j) in C r channel. A b is the average pixel value of the C b channel, and A r is the average pixel value of the C r channel. T b is the mean square error of the pixels of the C b channel, and T r is the mean square error of the pixels of the C r channel. sign represents a mathematical operator.

10. The positioning and detection system for ship signal flags according to claim 9, wherein, The color restoration unit is specifically configured to determine the gain value of each channel according to the following formula: Gain r = Y max / Ave r Gain g = Y max / Ave g Gain b = Y max / Ave b Among them, Gain r is the gain value of the C r channel, Gain g is the gain value of the C g channel, Gain b is the gain value of the C b channel, Y max is the maximum brightness value of the k-th frame image, Ave r is the pixel average value of the reference white point in C r channel, Ave g is the pixel average value of the reference white point in C g channel, Ave b is the pixel average value of the reference white point in C b channel.

11. The positioning and detection system for ship signal flags according to claim 10, wherein, The color restoration unit is specifically configured to perform color restoration processing on the k-th frame image according to the following formula: Among them, is the input image, is the output image after color restoration processing.

12. The positioning and detection system for ship signal flags according to claim 9, wherein, The moving target segmentation unit is specifically configured to read the first N frame images in the video to construct a background model; compare the pixel value of each pixel point in the k-th frame image with the pixel value of the corresponding pixel point in the background model; mark the pixel points within the acceptable range as the background, and mark the pixel points outside the acceptable range as the foreground to obtain the moving target.

13. The positioning and detection system for ship signal flags according to claim 12, wherein, The moving target segmentation unit is specifically configured to judge whether the pixel point is within the acceptable range according to the following formula: where B is the background image, I is the k-th frame image, (x, y) is the pixel point, σ is the variance of the background model, T is the preset threshold, and the pixel points that satisfy the above formula are within the acceptable range.

14. The positioning and detection system for ship signal flags according to claim 13, wherein, The moving target segmentation unit is further configured to update the background model according to the following formula: B t (x, y) = a·B t-1 (x, y) + (1 - a)·I t (x, y) where a is the update parameter and t is the number of moving target segmentation processing times.

15. The positioning and detection system for ship signal flags according to any one of claims 9 to 14, wherein, The shape detection unit is specifically configured to detect the contour of each separated moving target in the k-th frame image, use the two points with the largest distance on the contour as the separation points, and separate the contour into two curves at the first end and the second end; use the Douglas-Peucker algorithm to perform shape approximation on the curves at the first end and the second end respectively to obtain the contour approximation curve; judge whether the geometric feature of the signal flag is satisfied according to the geometric parameter information of the contour approximation curve, and determine the moving target that satisfies the geometric feature of the signal flag as the candidate signal flag.

16. The positioning and detection system for ship signal flags according to any one of claims 9 to 14, wherein, The color detection unit is specifically configured to determine the constituent colors of the signal flag, use each of the constituent colors as a preset color, and determine the channel ratio information of each preset color; perform RGB color separation on the k-th frame image after shape detection according to the channel ratio information of each preset color to obtain a color separation image corresponding to each preset color; respectively determine whether the pixel value of each pixel point in each color separation image is greater than the pixel value of the corresponding preset color, and if so, set the pixel value in the color separation image to 1, otherwise set it to 0; respectively determine the ratio of the area of the region where the pixel value is set to 1 in each color separation image to the area of the candidate signal flag region where it is located; identify the signal flag from the candidate signal flags according to the ratio.

17. A readable storage medium, characterized in that, at least one program is stored in the readable storage medium, and when the at least one program is executed, it is used to implement the positioning and detection method of the ship signal flag according to any one of claims 1 to 8.

18. A computer device, characterized in that, the computer device includes: a processor and a memory, the memory is used to store at least one program, and the processor is used to read the at least one program to implement the positioning and detection method of the ship signal flag according to any one of claims 1 to 8.

Images