Unmanned ship state identification method and system based on signal lamp

Abstract

An unmanned ship state identification method based on a signal lamp comprises the following steps: acquiring a video picture shot by a camera; whether a preset color exists in the video picture or not is judged, if not, the video picture is deleted, and if yes, the video picture is reserved; transmitting the video picture to a first model, and adjusting the color of the video picture by the first model to obtain a processed picture; and the processed picture is transmitted to a second model, the second model identifies the unmanned ship and the unmanned ship signal lamp in the processed picture, and the state of the unmanned ship in the video picture is determined through the number and color of the signal lamp. According to the method, the first model is arranged, the video picture is processed in the first model, and the color of the video picture is adjusted, so that the quality of the picture in a dark environment is optimized, and the situation that the second model cannot recognize the color subsequently, and state recognition errors of other unmanned ships are caused is avoided. And in severe cases, collision of the unmanned ship may occur and equipment of the unmanned ship is damaged.

Description

Technical Field

[0001] The present invention relates to the field of ship technology, and in particular to a signal light-based unmanned ship state recognition method and system. Background Art

[0002] With the continuous development of unmanned ship technology, its application in the fields of marine exploration, environmental monitoring, underwater detection and port freight is becoming more and more extensive. In order to ensure the safe navigation of unmanned ships in complex environments, it is particularly important to improve their signal recognition accuracy and adaptability.

[0003] However, at night, due to the dark environment, it is difficult to see the light emitted by the signal light, so it is impossible to effectively identify the color of a certain light. In addition, since some unmanned boats are in motion, the acquired images will have residual images, and different colors of lights will overlap in the residual image, affecting the judgment of the color of the signal light. This makes it impossible for unmanned boats to effectively identify their status, which ultimately affects the work of unmanned boats. For this reason, a method that can improve the recognition of unmanned boat signal lights at night is urgently needed. Summary of the invention

[0004] In view of the above-mentioned defects, the purpose of the present invention is to propose a method and system for identifying the status of an unmanned ship based on signal lights, so as to solve the problem that the signal lights of the unmanned ship cannot be identified due to image quality problems in a dark environment.

[0005] To achieve this purpose, the present invention adopts the following technical solution: a method for identifying the state of an unmanned ship based on a signal light, comprising the following steps:

[0006] Get the video footage taken by the camera;

[0007] Determine whether there is a preset color in the video image, if not, delete the video image, if yes, keep the video image;

[0008] The video image is transmitted to the first model, and the first model adjusts the color of the video image to obtain a processed image;

[0009] The processed image is transmitted to the second model, and the second model identifies the unmanned ship and the signal light of the unmanned ship in the processed image, and determines the state of the unmanned ship in the video image by the number and color of the signal light.

[0010] Preferably, the first model performs the following operations:

[0011] In one second of the video, two frames are extracted and marked as the first frame and the second frame respectively;

[0012] Taking the first screen as a prototype, a first processing sub-screen and a second processing sub-screen are respectively copied;

[0013] Respectively obtain each pixel in the first processing sub-picture and the second processing sub-picture, and store them in the first sequence and the second sequence in sequence respectively;

[0014] In the first sequence, each pixel is gray-scaled by a first threshold value;

[0015] In the second sequence, each pixel is gray-scaled by a second threshold value;

[0016] Merging the first sequence with the second sequence to obtain a first replacement picture;

[0017] Taking the second screen as a prototype, a third processing sub-screen and a fourth processing sub-screen are respectively copied;

[0018] Respectively obtain each pixel in the third processing sub-picture and the fourth processing sub-picture, and store them in the third sequence and the fourth sequence in sequence respectively;

[0019] In the third sequence, each pixel has its brightness adjusted by a third threshold value;

[0020] In the fourth sequence, each pixel has its brightness adjusted by a fourth threshold value;

[0021] The third sequence is merged with the fourth sequence to obtain a second replacement picture;

[0022] The second replacement picture is combined with the first replacement picture for high frequency images to obtain the processed picture.

[0023] Preferably, the sum of the first threshold and the second threshold is 1, and the first threshold is not equal to the second threshold; the sum of the third threshold and the fourth threshold is 1, and the third threshold is not equal to the fourth threshold.

[0024] Preferably, the method further comprises the following steps: based on the recognition result of the second model, confirming the average brightness of the non-unmanned ship area in the processing picture;

[0025] According to the result that the average brightness falls into the preset brightness range, the flashing frequency of the signal light in the unmanned boat is adjusted.

[0026] Preferably, before the video images are transmitted to the first model, the video images need to be preprocessed, wherein the preprocessing includes one or a combination of denoising, deinterlacing, channel scaling and channel sharpening.

[0027] An unmanned ship state recognition system based on signal lights, characterized in that the unmanned ship state recognition method based on signal lights is used, comprising an input module, a judgment module, a first processing module and a second processing module;

[0028] The input module is used to obtain the video images taken by the camera;

[0029] The judging module is used to judge whether there is a preset color in the video image, and if not, delete the video image; if so, keep the video image;

[0030] The first processing module is used to transmit the video picture to the first model, and the first model adjusts the color of the video picture to obtain a processed picture;

[0031] The second processing module is used to transmit the processed image to the second model, and the second model identifies the unmanned ship and the signal light of the unmanned ship in the processed image, and determines the state of the unmanned ship in the video image by the number and color of the signal lights.

[0032] Preferably, the first processing module includes a segmentation submodule, a first grayscale processing submodule, a second grayscale processing submodule and a synthesis submodule;

[0033] The segmentation submodule is used to extract two frames from one second of the video image, and mark them as the first frame and the second frame respectively;

[0034] The first grayscale processing submodule is used to use the first picture as a prototype to copy a first processing sub-picture and a second processing sub-picture respectively;

[0035] Respectively obtain each pixel in the first processing sub-picture and the second processing sub-picture, and store them in the first sequence and the second sequence in sequence respectively;

[0036] In the first sequence, each pixel is gray-scaled by a first threshold value;

[0037] In the second sequence, each pixel is gray-scaled by a second threshold value;

[0038] Merging the first sequence with the second sequence to obtain a first replacement picture;

[0039] The second grayscale processing submodule is used to copy the second picture as a prototype to obtain a third processing sub-picture and a fourth processing sub-picture respectively;

[0040] Respectively obtain each pixel in the third processing sub-picture and the fourth processing sub-picture, and store them in the third sequence and the fourth sequence in sequence respectively;

[0041] In the third sequence, each pixel has its brightness adjusted by a third threshold value;

[0042] In the fourth sequence, each pixel has its brightness adjusted by a fourth threshold value;

[0043] The third sequence is merged with the fourth sequence to obtain a second replacement picture;

[0044] The synthesis submodule is used to perform high-frequency image combination of the second replacement picture and the first replacement picture to obtain the processed picture.

[0045] Preferably, it also includes a regulating module;

[0046] The adjustment module is used to determine the average brightness of the non-unmanned ship area in the processing picture based on the recognition result of the second model;

[0047] According to the result that the average brightness falls into the preset brightness range, the flashing frequency of the signal light in the unmanned boat is adjusted.

[0048] One of the above technical solutions has the following advantages or beneficial effects: In the present invention, a first model is provided, in which the video image is processed and the color of the video image is adjusted, thereby optimizing the image quality in a dark environment, and avoiding the subsequent second model being unable to recognize the color, resulting in the state recognition error of other unmanned ships. In severe cases, the unmanned ships may collide and damage the unmanned ship equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is a flow chart of an embodiment of the method of the present invention.

[0050] Figure 2 It is a schematic diagram of the structure of an embodiment of the system of the present invention. DETAILED DESCRIPTION

[0051] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0052] In the description of the embodiments of the present invention, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0053] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, the meaning of "plurality" is two or more. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0054] like Figures 1-2 As shown, a method for identifying the state of an unmanned ship based on a signal light comprises the following steps:

[0055] Get the video footage taken by the camera;

[0056] Determine whether there is a preset color in the video image, if not, delete the video image, if yes, keep the video image;

[0057] The video image is transmitted to the first model, and the first model adjusts the color of the video image to obtain a processed image;

[0058] The processed image is transmitted to the second model, and the second model identifies the unmanned ship and the signal light of the unmanned ship in the processed image, and determines the state of the unmanned ship in the video image by the number and color of the signal light.

[0059] In the present invention, a camera is provided in the unmanned boat, and the camera can record the direction in which the unmanned boat is traveling. At this time, the video screen can be judged to determine whether a preset color appears in the video screen. If a preset color appears, it means that other unmanned boats may appear in the current video screen. For example, a red light, a green light, and a yellow light are provided in the signal light of the unmanned boat. At this time, red, green, and yellow are used as preset colors. Of course, in order to improve the recognition accuracy, colors similar to red, green, and yellow can also be set as preset colors, such as pink, dark green, etc.

[0060] If the preset color is not recognized, the video will be deleted. Because the unmanned boat is relatively small, its structure is relatively compact and cannot install a large storage device. Therefore, if the preset color is not recognized, the video will not be saved. When the preset color appears in the video, a video will be saved until the preset color disappears from the video.

[0061] Since the unmanned boat may travel at night when in operation, some colors are difficult to be recognized in the machine model in a dark environment. For this reason, a first model is provided in the present invention, in which the video image is processed and the color of the video image is adjusted, thereby optimizing the image quality in a dark environment and avoiding the subsequent second model being unable to recognize the color, resulting in the state recognition error of other unmanned boats. In serious cases, the unmanned boats may collide and damage the unmanned boat equipment.

[0062] The second model is equipped with a trained recognition model, which is used to identify unmanned boats, because the place where the preset color appears may not be on the unmanned boat. If the unmanned boat status is distinguished only by color, it is easy to cause misjudgment. For this reason, the unmanned boat and the unmanned boat signal light in the processing picture are identified, and then the status of other unmanned boats is determined according to the combination of signal lights of different colors.

[0063] Preferably, the first model performs the following operations:

[0064] In one second of the video, two frames are extracted and marked as the first frame and the second frame respectively;

[0065] Taking the first screen as a prototype, a first processing sub-screen and a second processing sub-screen are respectively copied;

[0066] Respectively obtain each pixel in the first processing sub-picture and the second processing sub-picture, and store them in the first sequence and the second sequence in sequence respectively;

[0067] In the first sequence, each pixel is gray-scaled by a first threshold value;

[0068] In the second sequence, each pixel is gray-scaled by a second threshold value;

[0069] Merging the first sequence with the second sequence to obtain a first replacement picture;

[0070] Taking the second screen as a prototype, a third processing sub-screen and a fourth processing sub-screen are respectively copied;

[0071] Respectively obtain each pixel in the third processing sub-picture and the fourth processing sub-picture, and store them in the third sequence and the fourth sequence in sequence respectively;

[0072] In the third sequence, each pixel has its brightness adjusted by a third threshold value;

[0073] In the fourth sequence, each pixel has its brightness adjusted by a fourth threshold value;

[0074] The third sequence is merged with the fourth sequence to obtain a second replacement picture;

[0075] The second replacement picture is combined with the first replacement picture for high frequency images to obtain the processed picture.

[0076] In order to improve the clarity and visibility of the video image in a dark environment, the present invention extracts two frames from one second of the video image. The first frame and the second frame are extracted in the same second and have continuity in time. When the subsequent second replacement frame is synthesized with the first replacement frame, there will be no image fragmentation, so that the processed frame is close to the video image.

[0077] The first processing sub-picture and the second processing sub-picture will store the pixel information in the first sequence and the second sequence in order respectively, and then adjust the grayscale;

[0078] Specifically, when making adjustments, a first threshold with a higher grayscale threshold can be used to adjust the first sequence, and then a second threshold with a lower grayscale threshold can be used to adjust the second sequence. After the adjustments, a pixel image with a higher grayscale and a pixel image with a lower grayscale will be obtained. After obtaining the first replacement image, the first replacement image will contain high grayscale features and low grayscale features. Features with different grayscales can remove unnecessary noise or interference. This helps to improve the clarity and recognizability of the image and improve the clarity of the image in a dark environment.

[0079] Similarly, the second replacement picture will also include high-brightness features and low-brightness features. Brightness adjustment can improve the overall brightness distribution of the image, making the image brighter or softer, thereby enhancing the visual experience and solving the problem of afterimages when the unmanned boat is moving.

[0080] Preferably, the sum of the first threshold and the second threshold is 1, and the first threshold is not equal to the second threshold; the sum of the third threshold and the fourth threshold is 1, and the third threshold is not equal to the fourth threshold.

[0081] Since the first threshold is not equal to the second threshold (the same applies to the third and fourth thresholds), different parts or features of the image can be processed differently. This differential processing helps to enhance the contrast and layering of the image and make the details more prominent.

[0082] In addition, by setting the sum of the thresholds to 1, we can ensure that the overall balance of pixel values ​​is maintained during the adjustment process. This weighting method allows fine grayscale or brightness adjustments to the image without causing the overall image to be too dark or too bright.

[0083] Preferably, the method further comprises the following steps: based on the recognition result of the second model, confirming the average brightness of the non-unmanned ship area in the processing picture;

[0084] According to the result that the average brightness falls into the preset brightness range, the flashing frequency of the signal light in the unmanned boat is adjusted.

[0085] The average brightness in the processed image can represent the brightness of the current environment. When the environment is too dark, the flashing frequency of the unmanned boat signal light needs to be increased to increase the probability of other ships recognizing the unmanned boat, thereby improving the safety of the unmanned boat when driving.

[0086] Preferably, before the video images are transmitted to the first model, the video images need to be preprocessed, wherein the preprocessing includes one or a combination of denoising, deinterlacing, channel scaling and channel sharpening.

[0087] Preprocessing can remove noise, redundant information and unnecessary interference in the video, such as blur, distortion, noise, etc., thereby improving the image quality input to the first model. At the same time, it can reduce the size and complexity of the image, reduce the amount of calculation for model processing, thereby speeding up the processing speed and improving the overall efficiency. It meets the requirements of the unmanned boat for recognition speed when driving.

[0088] An unmanned ship state recognition system based on signal lights, characterized in that the unmanned ship state recognition method based on signal lights is used, comprising an input module, a judgment module, a first processing module and a second processing module;

[0089] The input module is used to obtain the video images taken by the camera;

[0090] The judging module is used to judge whether there is a preset color in the video image, and if not, delete the video image; if so, keep the video image;

[0091] The first processing module is used to transmit the video picture to the first model, and the first model adjusts the color of the video picture to obtain a processed picture;

[0092] The second processing module is used to transmit the processed image to the second model, and the second model identifies the unmanned ship and the signal light of the unmanned ship in the processed image, and determines the state of the unmanned ship in the video image by the number and color of the signal lights.

[0093] Preferably, the first processing module includes a segmentation submodule, a first grayscale processing submodule, a second grayscale processing submodule and a synthesis submodule;

[0094] The segmentation submodule is used to extract two frames from one second of the video image, and mark them as the first frame and the second frame respectively;

[0095] The first grayscale processing submodule is used to use the first picture as a prototype to copy a first processing sub-picture and a second processing sub-picture respectively;

[0096] Respectively obtain each pixel in the first processing sub-picture and the second processing sub-picture, and store them in the first sequence and the second sequence in sequence respectively;

[0097] In the first sequence, each pixel is gray-scaled by a first threshold value;

[0098] In the second sequence, each pixel is gray-scaled by a second threshold value;

[0099] Merging the first sequence with the second sequence to obtain a first replacement picture;

[0100] The second grayscale processing submodule is used to copy the second picture as a prototype to obtain a third processing sub-picture and a fourth processing sub-picture respectively;

[0101] Respectively obtain each pixel in the third processing sub-picture and the fourth processing sub-picture, and store them in the third sequence and the fourth sequence in sequence respectively;

[0102] In the third sequence, each pixel has its brightness adjusted by a third threshold value;

[0103] In the fourth sequence, each pixel has its brightness adjusted by a fourth threshold value;

[0104] The third sequence is merged with the fourth sequence to obtain a second replacement picture;

[0105] The synthesis submodule is used to perform high-frequency image combination of the second replacement picture and the first replacement picture to obtain the processed picture.

[0106] Preferably, it also includes a regulating module;

[0107] The adjustment module is used to determine the average brightness of the non-unmanned ship area in the processing picture based on the recognition result of the second model;

[0108] According to the result that the average brightness falls into the preset brightness range, the flashing frequency of the signal light in the unmanned boat is adjusted.

[0109] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0110] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A method for identifying the state of an unmanned ship based on a signal light, characterized in that: The steps include: Get the video footage taken by the camera; Determine whether there is a preset color in the video image, if not, delete the video image, if yes, keep the video image; The video image is transmitted to the first model, and the first model adjusts the color of the video image to obtain a processed image; The processed image is transmitted to the second model, and the second model identifies the unmanned ship and the signal light of the unmanned ship in the processed image, and determines the state of the unmanned ship in the video image by the number and color of the signal light.

2. The method for identifying the state of an unmanned ship based on a signal light according to claim 1, characterized in that: The first model performs the following operations: In one second of the video, two frames are extracted and marked as the first frame and the second frame respectively; Taking the first screen as a prototype, a first processing sub-screen and a second processing sub-screen are respectively copied; Respectively obtain each pixel in the first processing sub-picture and the second processing sub-picture, and store them in the first sequence and the second sequence in sequence respectively; In the first sequence, each pixel is gray-scaled by a first threshold value; In the second sequence, each pixel is gray-scaled by a second threshold value; Merging the first sequence with the second sequence to obtain a first replacement picture; Taking the second screen as a prototype, a third processing sub-screen and a fourth processing sub-screen are respectively copied; Respectively obtain each pixel in the third processing sub-picture and the fourth processing sub-picture, and store them in the third sequence and the fourth sequence in sequence respectively; In the third sequence, each pixel has its brightness adjusted by a third threshold value; In the fourth sequence, each pixel has its brightness adjusted by a fourth threshold value; The third sequence is merged with the fourth sequence to obtain a second replacement picture; The second replacement picture is combined with the first replacement picture for high frequency images to obtain the processed picture.

3. The method for identifying the state of an unmanned ship based on a signal light according to claim 2, characterized in that: The sum of the first threshold and the second threshold is 1, and the first threshold is not equal to the second threshold; the sum of the third threshold and the fourth threshold is 1, and the third threshold is not equal to the fourth threshold.

4. The method for identifying the state of an unmanned ship based on a signal light according to claim 1, characterized in that: The method further includes the following steps: based on the recognition result of the second model, determining the average brightness of the non-unmanned ship area in the processing picture; According to the result that the average brightness falls into the preset brightness range, the flashing frequency of the signal light in the unmanned boat is adjusted.

5. The method for identifying the state of an unmanned ship based on a signal light according to claim 1, characterized in that: Before the video images are transmitted to the first model, the video images need to be preprocessed, wherein the preprocessing includes one or a combination of denoising, deinterlacing, channel scaling and channel sharpening.

6. An unmanned ship status recognition system based on signal lights, characterized in that: The unmanned ship state recognition method based on signal lights according to any one of claims 1 to 5 comprises an input module, a judgment module, a first processing module and a second processing module; The input module is used to obtain the video images taken by the camera; The judging module is used to judge whether there is a preset color in the video image, and if not, delete the video image; if so, keep the video image; The first processing module is used to transmit the video picture to the first model, and the first model adjusts the color of the video picture to obtain a processed picture; The second processing module is used to transmit the processed image to the second model, and the second model identifies the unmanned ship and the signal light of the unmanned ship in the processed image, and determines the state of the unmanned ship in the video image by the number and color of the signal lights.

7. The unmanned ship status recognition system based on signal lights according to claim 6 is characterized in that: The first processing module includes a segmentation submodule, a first grayscale processing submodule, a second grayscale processing submodule and a synthesis submodule; The segmentation submodule is used to extract two frames from one second of the video image, and mark them as the first frame and the second frame respectively; The first grayscale processing submodule is used to use the first picture as a prototype to copy a first processing sub-picture and a second processing sub-picture respectively; Respectively obtain each pixel in the first processing sub-picture and the second processing sub-picture, and store them in the first sequence and the second sequence in sequence respectively; In the first sequence, each pixel is gray-scaled by a first threshold value; In the second sequence, each pixel is gray-scaled by a second threshold value; Merging the first sequence with the second sequence to obtain a first replacement picture; The second grayscale processing submodule is used to copy the second picture as a prototype to obtain a third processing sub-picture and a fourth processing sub-picture respectively; Respectively obtain each pixel in the third processing sub-picture and the fourth processing sub-picture, and store them in the third sequence and the fourth sequence in sequence respectively; In the third sequence, each pixel has its brightness adjusted by a third threshold value; In the fourth sequence, each pixel has its brightness adjusted by a fourth threshold value; The third sequence is merged with the fourth sequence to obtain a second replacement picture; The synthesis submodule is used to perform high-frequency image combination of the second replacement picture and the first replacement picture to obtain the processed picture.

8. The unmanned ship status recognition system based on signal lights according to claim 6 is characterized in that: Also includes a regulation module; The adjustment module is used to determine the average brightness of the non-unmanned ship area in the processing picture based on the recognition result of the second model; According to the result that the average brightness falls into the preset brightness range, the flashing frequency of the signal light in the unmanned boat is adjusted.

Images