Method and device for sea target detection and storage medium

Abstract

The invention relates to the technical field of computer vision, and discloses a method and device for sea target detection and a storage medium. The method comprises the following steps: determining a subset as a current test set from an acquired marine target image data total set, and processing the current test set based on a current marine target pre-training model to obtain a corresponding training target detection result, the current marine target pre-training model is obtained by training based on a set target detection deep neural network; obtaining an experiment target detection result obtained by performing a human eye alignment experiment according to the current test set, and adjusting the current marine target pre-training model according to the training target detection result and the experiment target detection result to obtain an adjusted marine target pre-training model; and performing sea target detection through the adjusted sea target pre-training model. Therefore, automatic sea target detection is realized, and the target detection capability is equivalent to the human eye capability.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, for example, to a method, device and storage medium for marine target detection. Background Art

[0002] There are some maritime target recognition tasks in the fields of ship visibility design and maritime target visibility assessment. These tasks include evaluating whether a maritime target can be found by the naked eye through massive videos or images, or evaluating the detectability of a certain marine camouflage pattern in simulation design tools. These tasks can be completed by humans, but the efficiency is very low and long-term work is harmful to human eye health.

[0003] In the related technology, the target detection algorithm can classify and locate the targets of uncertain types and quantities in images or videos. Therefore, the target detection algorithm can be used to perform these marine target recognition tasks. However, due to the particularity of marine scenes, many factors will cause the performance of general target detection algorithms to decline. Therefore, the accuracy of marine target detection needs to be improved.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present application, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] In order to provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. The summary is not an extensive review, nor is it intended to identify key / critical components or delineate the scope of protection of these embodiments, but rather serves as a prelude to the detailed description that follows.

[0006] The embodiments of the present disclosure provide a method, an apparatus, a device and a storage medium for marine target detection to solve the technical problem that the accuracy of marine target detection needs to be improved.

[0007] In some embodiments, the method comprises:

[0008] Determine a subset from the total set of acquired marine target image data as a current test set, and process the current test set based on a current marine target pre-training model to obtain a corresponding training target detection result, wherein the current marine target pre-training model is obtained by training based on a set target detection deep neural network;

[0009] Acquire the experimental target detection result obtained by performing the human eye alignment experiment according to the current test set, and adjust the current marine target pre-training model according to the training target detection result and the experimental target detection result to obtain the adjusted marine target pre-training model;

[0010] The maritime target detection is performed by adjusting the maritime target pre-training model.

[0011] In some embodiments, the process of acquiring the total set of marine target image data includes:

[0012] Acquire an open source first marine target image dataset, acquire a second marine target image dataset through infrared imaging or visible light imaging, and acquire a third marine target image dataset through simulation modeling;

[0013] A total set of marine target image data is formed according to the first marine target image data set, the second marine target image data set and the third marine target image data set, and original target information is annotated for each image.

[0014] In some embodiments, acquiring the second marine target image dataset comprises:

[0015] Obtain the first real-shot image of a specific marine target through infrared imaging or visible light imaging;

[0016] Performing image data enhancement processing on the images in the first marine target image data set to obtain a corresponding first enhanced image;

[0017] The fidelity of the first enhanced image is verified based on the first real-shot image. If the verification passes, the first real-shot image and the first enhanced image are added to the second marine target image data set.

[0018] In some embodiments, acquiring the third marine target image dataset comprises:

[0019] Acquire a second real-shot image of a specific marine target;

[0020] Performing three-dimensional modeling according to a specific marine target, constructing a corresponding marine simulation environment, and generating a second simulation image that matches the target type, imaging conditions, and environmental conditions in the marine simulation environment;

[0021] The second simulation image is verified for fidelity based on the second real-shot image. If the verification is passed, the second real-shot image and the second simulation image are added to the third marine target image data set.

[0022] In some embodiments, the process of obtaining the current marine target pre-training model includes:

[0023] dividing the total marine target image data set into two or more marine target image data subsets;

[0024] A subset of maritime target image data is determined as a training set, and a set target detection deep neural network is trained according to the training set to obtain a current maritime target pre-training model, wherein the training set is different from the current test set.

[0025] In some embodiments, obtaining the experimental target detection result obtained by performing the human eye alignment experiment according to the current test set includes:

[0026] Obtaining first target information annotated by the subject on a test image in the current test set, wherein the subject performs image feature learning of marine targets based on the test image carrying the original target information;

[0027] Verification is performed based on the original target information, the first target information, and second target information annotated by the subject after the verification reinforcement learning on the detection image in the current test set is obtained, wherein the detection image is different from the test image.

[0028] In some embodiments, obtaining the adjusted maritime target pre-training model includes:

[0029] According to the training target detection results and the experimental target detection results, the original target information of the detection image is adjusted, and the number of samples of the detection image is adjusted to obtain an adjusted current test set;

[0030] According to the adjusted current test set, the current maritime target pre-training model is adjusted and trained to obtain an adjusted maritime target pre-training model.

[0031] In some embodiments, after obtaining the adjusted pre-trained model of the maritime target, the method further includes:

[0032] The adjusted maritime target pre-training model is replaced with the current maritime target pre-training model.

[0033] In some embodiments, the apparatus for marine target detection includes a processor and a memory storing program instructions, and the processor is configured to execute the above-mentioned method for marine target detection when executing the program instructions.

[0034] In some embodiments, the storage medium stores program instructions, and when the program instructions are run, the above-mentioned method for marine target detection is executed.

[0035] The method, device, and storage medium for marine target detection provided by the embodiments of the present disclosure can achieve the following technical effects:

[0036] After training the current maritime target pre-training model based on the set target detection deep neural network, the current maritime target pre-training model can be fine-tuned according to the human eye alignment experiment to obtain the adjusted maritime target pre-training model, and maritime target detection can be performed through the adjusted maritime target pre-training model. In this way, not only automated maritime target detection is achieved, but also the detection capability is equivalent to that of the human eye, which can ensure that the experimental results obtained by completing the maritime target recognition task are consistent with those of humans, thereby improving the accuracy of maritime target detection.

[0037] The above general description and the following description are exemplary and explanatory only and are not intended to limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] One or more embodiments are exemplarily described by corresponding drawings, which do not limit the embodiments. Elements with the same reference numerals in the drawings are shown as similar elements, and the drawings do not constitute a scale limitation, and wherein:

[0039] Figure 1 is a flow chart of a method for detecting targets at sea provided by an embodiment of the present disclosure;

[0040] Figure 2 It is a schematic diagram of a scenario for training-verification-detection in a human eye experiment provided by an embodiment of the present disclosure;

[0041] Figure 3 is a flow chart of a method for detecting targets at sea provided by an embodiment of the present disclosure;

[0042] Figure 4 is a structural schematic diagram of a device for detecting targets at sea provided by an embodiment of the present disclosure;

[0043] Figure 5 is a structural schematic diagram of a device for detecting targets at sea provided by an embodiment of the present disclosure;

[0044] Figure 6 is a structural schematic diagram of a device for detecting targets at sea provided by an embodiment of the present disclosure;

[0045] Figure 7 It is a schematic diagram of a detection device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0046] In order to be able to understand the features and technical contents of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the convenience of explanation, a full understanding of the disclosed embodiments is provided through multiple details. However, one or more embodiments can still be implemented without these details. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.

[0047] The terms "first", "second", etc. in the specification and claims of the embodiments of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so that the embodiments of the embodiments of the present disclosure described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions.

[0048] Unless otherwise stated, the term "plurality" means two or more.

[0049] In the embodiment of the present disclosure, the character " / " indicates that the preceding and following objects are in an "or" relationship. For example, A / B indicates: A or B.

[0050] The term "and / or" is a description of the association relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or, A and B.

[0051] In the fields of ship visibility design and maritime target visibility assessment, there are some maritime target recognition tasks. In order to complete these tasks automatically, it is necessary to accurately model the perception ability of the human eye and obtain a deep neural network that can replace the human eye to complete the maritime target detection task. Devices and equipment with this deep neural network as the core functional module can be used to replace humans to complete some repetitive, inefficient or human eye health-affecting maritime target recognition tasks. In the disclosed embodiment, after the current maritime target pre-training model is obtained by training based on the set target detection deep neural network, the current maritime target pre-training model can be fine-tuned according to the human eye alignment experiment to obtain the adjusted maritime target pre-training model, and the maritime target detection is performed through the adjusted maritime target pre-training model. In this way, not only the automated maritime target detection is realized, but also the detection ability is equivalent to that of the human eye, which can ensure that the experimental results obtained by completing the maritime target recognition task are consistent with humans, and improve the accuracy of maritime target detection.

[0052] Figure 1 FIG. 1 is a flow chart of a method for detecting targets at sea provided by an embodiment of the present disclosure. Figure 1 As shown in the figure, the process of marine target detection includes:

[0053] Step 101: Determine a subset from the total set of acquired marine target image data as the current test set, and process the current test set based on the current marine target pre-training model to obtain the corresponding training target detection result, wherein the current marine target pre-training model is trained based on the set target detection deep neural network.

[0054] The target detection algorithm can classify and locate targets of uncertain types and numbers in images or videos. The development of deep learning technology has pushed the target detection algorithm to a new level. The target detection algorithm based on deep neural network has surpassed the traditional target detection algorithm in terms of algorithm recognition accuracy and speed, and even surpassed the target detection ability of the human eye in some aspects. Therefore, in the embodiment of the present disclosure, a pre-trained model of marine targets can be trained based on the set target detection deep neural network, and then marine target detection can be performed based on the trained pre-trained model of marine targets.

[0055] However, target detection based on deep learning depends on the training image data set. Therefore, the disclosed embodiment needs to construct a total set of marine target image data, which is required to include marine target images under different target types, imaging angles, imaging distances, lighting conditions, weather conditions, sea conditions, etc., and is required to include target image samples that exceed the target detection limit of the human eye under these different conditions. For example, if a marine target can be detected by the human eye at an imaging distance of 1 kilometer under a specific condition, the total set of image data needs to include images of the target at an imaging distance greater than 1 kilometer under the same conditions. In some embodiments, the process of acquiring the total set of marine target image data includes: acquiring an open source first marine target image data set, acquiring a second marine target image data set through infrared imaging or visible light imaging, and acquiring a third marine target image data set through simulation modeling; forming a total set of marine target image data according to the first marine target image data set, the second marine target image data set, and the third marine target image data set, and annotating each image with original target information.

[0056] Among them, through data communication, web crawlers, etc., the detection equipment can collect open source maritime target image data sets, including real-shot images of specific maritime targets that require human eye target detection capability modeling, that is, including: maritime target images under different target types, imaging angles, imaging distances, lighting conditions, weather conditions, sea conditions, etc., and of course, target images that exceed the human eye target detection limit under these different conditions.

[0057] In some embodiments, obtaining a second marine target image data set includes: obtaining a first real-shot image of a specific marine target by infrared imaging or visible light imaging; performing image data enhancement processing on the image in the first marine target image data set to obtain a corresponding first enhanced image; and verifying the fidelity of the first enhanced image according to the first real-shot image, and adding the first real-shot image and the first enhanced image to the second marine target image data set if the verification passes. That is, the first real-shot image of a specific marine target can be obtained by infrared imaging or visible light imaging, and then, including but not limited to using an image data enhancement method, the first marine target image data set obtained by open source can be expanded, so that marine target image data under imaging conditions such as imaging angles, imaging distances, lighting conditions, weather conditions, and sea conditions different from those of the open source images can be obtained, that is, the first enhanced image is obtained, and compared with the first real-shot image to verify the fidelity of the first enhanced image; if the verification passes, the first real-shot image and the first enhanced image are added to the second marine target image data set.

[0058] In some embodiments, obtaining the third marine target image data set includes: obtaining a second real-shot image of a specific marine target; performing three-dimensional modeling according to the specific marine target, constructing a corresponding marine simulation environment, and generating a second simulation image that matches the target type, imaging conditions, and environmental conditions in the marine simulation environment; verifying the fidelity of the second simulation image according to the second real-shot image, and adding the second real-shot image and the second simulation image to the third marine target image data set if the verification passes. Specifically, it may include: performing three-dimensional modeling on a specific marine target that requires human eye target detection capability modeling, constructing a high-fidelity marine environment simulation environment, and then batch-generating marine target image data under different target types, imaging angles, imaging distances, lighting conditions, weather conditions, sea conditions, and other imaging conditions, remembering the second simulation image, and comparing it with the second real-shot data to verify the fidelity of the second simulation image; if the verification passes, adding the second real-shot image and the second simulation image to the third marine target image data set.

[0059] After the detection device obtains the first marine target image data set, the second marine target image data set, and the third marine target image data set, all the image data can be combined into a marine target image data set, and all the image data can be uniformly labeled to provide the type and location data of the marine target in each image, that is, each image in the marine target image data set carries the original target information, and the target information includes: the type and location data of the target.

[0060] Of course, in some embodiments, the total set of maritime target image data may include only the first maritime target image data set, or include: the first maritime target image data set and the second maritime target image data set, or include: the first maritime target image data set and the third maritime target image data set, and so on.

[0061] It can be seen that the total set of marine target image data includes multiple images with original target information. Therefore, the images in the total set of marine target image data can be used as training samples. These training samples are used to train the set target detection deep neural network to obtain the corresponding marine target pre-training model.

[0062] Among them, the set target detection deep neural network includes: target detection algorithm based on deep neural network, and the target detection algorithm may include: single-stage target detection algorithms such as YOLO series algorithms and SSD algorithms, as well as two-stage target detection algorithms such as R-CNN algorithm and Faster R-CNN algorithm, etc.

[0063] In some embodiments, the set target detection deep neural network can be trained directly according to the total set of marine target image data to obtain the corresponding marine target pre-trained model. Alternatively, only one or more subsets of the total set of marine target image data are needed to train the set target detection deep neural network to obtain the corresponding marine target pre-trained model. Therefore, in some embodiments, the process of obtaining the current marine target pre-trained model includes: dividing the total set of marine target image data into two or more subsets of marine target image data; determining a subset of marine target image data as a training set, and training the set target detection deep neural network according to the training set to obtain the current marine target pre-trained model, wherein the training set is different from the current test set.

[0064] The detection device can randomly divide the acquired total set of marine target image data into two or more marine target image data subsets, and determine one marine target image data subset as a training set. Therefore, the total set of marine target image data can include: a training set, a test set 1, a test set 2... and other different subsets. The training set is used to train the selected target detection deep neural network, and the corresponding current marine target pre-training model is obtained through training.

[0065] The total set of marine target image data includes not only the training set, but also multiple different subsets such as test set 1, test set 2, etc. Therefore, one of the multiple different subsets such as test set 1, test set 2, etc. can be determined as the current test set. Of course, if the total set of marine target image data only includes the two subsets of training set and test set, then this test set can be determined as the current test subset. If the total set of marine target image data includes multiple different subsets such as test set 1, test set 2, etc., then a test set can be determined as the current test set in a set order or randomly.

[0066] Therefore, the current test set can be processed according to the current pre-trained model of maritime targets to obtain the corresponding training target detection results.

[0067] Step 102: Obtain the experimental target detection result obtained by performing the human eye alignment experiment according to the current test set, and adjust the current marine target pre-training model according to the training target detection result and the experimental target detection result to obtain the adjusted marine target pre-training model.

[0068] In some embodiments, obtaining the experimental target detection result obtained by performing a human eye alignment experiment based on the current test set includes: obtaining first target information marked by the subject on the test image in the current test set, wherein the subject performs image feature learning of the marine target based on the test image carrying the original target information; performing verification based on the original target information and the first target information, and obtaining second target information marked by the subject on the detection image in the current test set after the verification reinforcement learning, wherein the detection image is different from the test image.

[0069] The current test set includes multiple images, and each image carries the original target information. In this way, the multiple images in the current test set can be divided into test images and detection images. Then, the subjects are recruited to conduct the human eye alignment experiment based on the test images. The experimental process uses the same learning process as the deep neural network, that is, the "training-validation-detection" process. The following can describe the "training-validation-detection" process with the current test set as test set 1.

[0070] In the training phase, the subjects will undergo a learning process for a small number of image samples in the test set 1. The subjects will be presented with image samples with original target information to learn the image features of different marine targets. That is, the image samples are test images. In this way, the subjects learn the image features of marine targets based on the test images carrying the original target information. Figure 2 As described, the detection device can display a test image carrying the target type on the screen.

[0071] In the verification phase, for the test images selected from the test set 1, the subjects will feedback the corresponding target recognition results, that is, the detection device obtains the first target information marked by the subject on the test image, so that the subject can be judged whether it is correct or wrong based on the original target information and the first target information; the verification phase is used to help the subjects enhance the learning effect and become familiar with the operation process of the detection phase, avoiding the possibility of generating invalid data due to incorrect operation in the detection phase due to misunderstanding of the operation process. Figure 2 As shown, after the detection device displays a test image on the screen, it can present a choice of target type. In this way, after the subject makes a choice, the detection device can receive the selection information input by the subject, that is, obtain the corresponding first target information, and then judge whether the subject is correct or wrong based on the original target information corresponding to the test image, the first target information.

[0072] In the detection phase, the subject who has verified the enhanced learning marks the location and type of the target on the remaining images in the test set 1; in this way, the detection device can obtain the second target information marked by the subject who has verified the enhanced learning on the detection image in the current test set. Figure 2 As shown, the detection image that appears on the detection device is different from the test images that appear in the training phase and the verification phase, and the target type selected by the subject can be obtained, that is, the second target information is obtained.

[0073] In the detection phase, images that have appeared in the practice phase and the verification phase will not appear, that is, the detection image is different from the test image. Of course, it is also required that each image is detected by at least two subjects. In this way, the target detection capability of the marine target pre-training model can be further improved to a degree of alignment with the human eye.

[0074] In this way, the experimental target detection result obtained by performing the human eye alignment experiment according to the current test set can be obtained according to the second target information. Then, according to the training target detection result and the experimental target detection result, the current marine target pre-training model is adjusted to obtain the adjusted marine target pre-training model.

[0075] In some embodiments, obtaining the adjusted maritime target pre-training model includes: adjusting the original target information of the detection image and the number of samples of the detection image according to the training target detection results and the experimental target detection results to obtain an adjusted current test set; adjusting and training the current maritime target pre-training model according to the adjusted current test set to obtain an adjusted maritime target pre-training model.

[0076] The detection equipment can compare the target detection results obtained by the current marine target pre-training and the human eye alignment experiment on the same image data set, determine under which imaging conditions the training target detection ability of the pre-trained model is stronger than the experimental target detection results of the human eye alignment experiment, and under which imaging conditions the training target detection ability of the pre-trained model is weaker than the experimental target detection results of the human eye alignment experiment, and adjust the number of labeled data and image samples of the current test set according to different situations to obtain the adjusted current test set.

[0077] The current marine target pre-training model is fine-tuned using the adjusted current test set to make the model closer to the target detection capability of the human eye.

[0078] The current test set may be test set 1, test set 2, test set 3, or test set 4, etc., or, in some embodiments, after obtaining the adjusted pre-trained model for the maritime target, the method further includes: replacing the adjusted pre-trained model for the maritime target with the current pre-trained model for the maritime target. In this way, if the current pre-trained model for the maritime target is adjusted according to test set 1, the pre-trained model for the maritime target may be further adjusted according to test set 2, test set 3, and test set 4, etc., based on the above process, until the target detection capability of the obtained pre-trained model for the maritime target is fully aligned with the human eye.

[0079] Step 103: Detect marine targets by adjusting the marine target pre-training model.

[0080] After the detection equipment obtains the real-life image of the maritime target, it can input it into the adjusted maritime target pre-training model to obtain the target information, thus realizing maritime target detection.

[0081] In this way, the detection equipment can collect the pre-trained model of marine targets trained based on the set target detection deep neural network to detect marine targets. In addition, the pre-trained model of marine targets has also been adjusted through human eye alignment experiments, and its target detection capability is highly aligned with the human eye.

[0082] It can be seen that in the disclosed embodiment, after the current maritime target pre-training model is obtained by training based on the set target detection deep neural network, the current maritime target pre-training model can be fine-tuned and trained according to the human eye alignment experiment to obtain the adjusted maritime target pre-training model, and maritime target detection is performed through the adjusted maritime target pre-training model. In this way, not only automated maritime target detection is achieved, but also the detection capability is equivalent to that of the human eye, which can ensure that the experimental results obtained by completing the maritime target recognition task are consistent with those of humans, thereby improving the accuracy of maritime target detection.

[0083] The operation flow is summarized into a specific embodiment below to illustrate the marine target detection process provided by the embodiment of the present invention.

[0084] In one embodiment of the present disclosure, the target detection algorithm based on the set target detection deep neural network can be the Faster R-CNN algorithm. Figure 2 , Figure 3 ,The process for maritime target detection includes:

[0085] Step 301: The detection device obtains an open source first marine target image data set, obtains a second marine target image data set through infrared imaging or visible light imaging, and obtains a third marine target image data set through simulation modeling.

[0086] Step 302: The detection device forms a total set of marine target image data based on the first marine target image data set, the second marine target image data set, and the third marine target image data set, and annotates each image with original target information.

[0087] Step 303: The detection device divides the total set of marine target image data into a plurality of different subsets such as a training set, a test set 1, a test set 2, etc., and trains the Faster R-CNN algorithm according to the training set to obtain a current marine target pre-training model.

[0088] Step 304: The detection device determines the current test set from a test set in ascending order of number, and processes the current test set based on the current marine target pre-training model to obtain the corresponding training target detection result.

[0089] Step 305: The detection device obtains the first target information marked by the subject on the test image in the current test set, wherein the subject performs image feature learning of the marine target based on the test image carrying the original target information.

[0090] Among them, the subjects can Figure 2 As shown, image feature learning of marine targets was performed, i.e., the subjects were trained.

[0091] Step 306: The detection device performs verification based on the original target information and the first target information, and obtains the second target information marked by the subject on the detection image in the current test set after the verification reinforcement learning, and obtains the corresponding experimental target detection result based on the second target information.

[0092] Likewise, if Figure 2 As shown, after verification and testing, the detection device obtains the second target information, and obtains the corresponding experimental target detection results based on the second target information of multiple subjects.

[0093] Step 307: The detection device adjusts the original target information of the detection image and the number of samples of the detection image according to the training target detection result and the experimental target detection result to obtain an adjusted current test set.

[0094] Step 308: The detection device adjusts and trains the current marine target pre-training model according to the adjusted current test set to obtain an adjusted marine target pre-training model.

[0095] Step 309: Determine whether all test sets have been subjected to the eye alignment experiment. If so, execute step 311; otherwise, execute step 310.

[0096] Step 310: The detection device replaces the adjusted marine target pre-training model with the current marine target pre-training model and returns to step 304.

[0097] Step 311: The detection device performs marine target detection by adjusting the marine target pre-training model.

[0098] It can be seen that in this embodiment, after the current maritime target pre-training model is obtained by training based on the Faster R-CNN algorithm, the current maritime target pre-training model can be fine-tuned and trained multiple times according to the human eye alignment experiment to obtain the adjusted maritime target pre-training model, and the maritime target detection is performed through the adjusted maritime target pre-training model. In this way, not only the automated maritime target detection is realized, but also the detection capability is fully aligned with the human eye, which can ensure that the experimental results obtained by completing the maritime target recognition task are consistent with humans, thereby improving the accuracy of maritime target detection.

[0099] According to the above process for marine target detection, a device for marine target detection can be constructed.

[0100] Figure 4 Schematic diagram of a structure of a device for detecting marine targets provided by an embodiment of the present disclosure. Figure 4 As shown, the marine target detection device 400 includes: a determination training module 410 , an experiment adjustment module 420 and a target detection module 430 .

[0101] The determination training module 410 is configured to determine a subset from the total set of acquired marine target image data as a current test set, and process the current test set based on the current marine target pre-training model to obtain a corresponding training target detection result, wherein the current marine target pre-training model is obtained by training based on a set target detection deep neural network.

[0102] The experimental adjustment module 420 is configured to obtain the experimental target detection result obtained by performing the human eye alignment experiment according to the current test set, and adjust the current marine target pre-training model according to the training target detection result and the experimental target detection result to obtain the adjusted marine target pre-training model.

[0103] The target detection module 430 is configured to perform marine target detection by using the adjusted marine target pre-training model.

[0104] In some embodiments, determining the training module 410 includes:

[0105] The acquisition unit is configured to acquire an open source first marine target image data set, acquire a second marine target image data set through infrared imaging or visible light imaging, and acquire a third marine target image data set through simulation modeling.

[0106] The summary annotation unit is configured to form a summary set of marine target image data based on the first marine target image data set, the second marine target image data set, and the third marine target image data set, and annotate each image with original target information.

[0107] In some embodiments, the acquisition unit is specifically configured to acquire a first real-shot image of a specific maritime target through infrared imaging or visible light imaging; perform image data enhancement processing on the image in the first maritime target image data set to obtain a corresponding first enhanced image; perform realism verification on the first enhanced image based on the first real-shot image, and if the verification passes, add the first real-shot image and the first enhanced image to the second maritime target image data set.

[0108] In some embodiments, the acquisition unit is specifically configured to acquire a second real-shot image of a specific marine target; perform three-dimensional modeling based on the specific marine target, construct a corresponding marine simulation environment, and generate a second simulated image that matches the target type, imaging conditions, and environmental conditions in the marine simulation environment; perform realism verification on the second simulated image based on the second real-shot image, and if the verification passes, add the second real-shot image and the second simulated image to a third marine target image data set.

[0109] In some embodiments, it also includes: a training module, which is configured to divide the total set of marine target image data into two or more marine target image data subsets; determine a marine target image data subset as a training set, and train the set target detection deep neural network according to the training set to obtain the current marine target pre-training model, wherein the training set is different from the current test set.

[0110] In some embodiments, the experiment adjustment module 420 includes:

[0111] The learning detection unit is configured to obtain first target information annotated by the subject on a test image in a current test set, wherein the subject performs image feature learning of marine targets based on the test image carrying the original target information.

[0112] The verification unit is configured to perform verification based on the original target information and the first target information, and obtain the second target information annotated by the subject after the verification reinforcement learning on the detection image in the current test set, wherein the detection image is different from the test image.

[0113] In some embodiments, the experiment adjustment module 420 further includes:

[0114] The adjustment unit is configured to adjust the original target information of the detection image and the number of samples of the detection image according to the training target detection results and the experimental target detection results to obtain an adjusted current test set; according to the adjusted current test set, the current maritime target pre-training model is adjusted and trained to obtain an adjusted maritime target pre-training model.

[0115] In some embodiments, it also includes:

[0116] The replacement update module is configured to replace the adjusted maritime target pre-trained model with the current maritime target pre-trained model.

[0117] The marine target detection process used in the marine target detection device is further described below in conjunction with the embodiments.

[0118] In one embodiment of the present disclosure, the target detection algorithm based on the set target detection deep neural network may be an SSD algorithm.

[0119] Figure 5 Schematic diagram of a structure of a device for detecting marine targets provided by an embodiment of the present disclosure. Figure 4 As shown, the marine target detection device 400 includes: a determination training module 410, an experiment adjustment module 420, a target detection module 430, a training acquisition module 440 and a replacement update module 450. Among them, the determination training module 410 includes: a collection acquisition unit 411 and a summary labeling unit 412, and the experiment adjustment module 420 includes: a learning detection unit 421, a verification acquisition unit 422 and an adjustment acquisition unit 423.

[0120] The acquisition unit 411 in the training module 410 is determined to acquire an open source first marine target image data set, acquire a second marine target image data set through infrared imaging or visible light imaging, and acquire a third marine target image data set through simulation modeling; the summary annotation unit 412 can form a summary set of marine target image data based on the first marine target image data set, the second marine target image data set, and the third marine target image data set, and annotate each image with original target information.

[0121] In this way, the training module 440 can divide the total set of marine target image data into a plurality of different subsets such as a training set, a test set 1, a test set 2, etc., and train the SSD algorithm according to the training set to obtain the current marine target pre-training model. Thus, the training module 410 can determine the current test set from a test set in ascending order of numbers, and process the current test set based on the current marine target pre-training model to obtain the corresponding training target detection result.

[0122] The learning detection unit 421 in the experimental adjustment module 420 obtains the first target information marked by the subject on the test image in the current test set, wherein the subject performs image feature learning of the marine target based on the test image carrying the original target information; the verification unit 422 performs verification based on the original target information and the first target information, and obtains the second target information marked by the subject on the detection image in the current test set after the verification reinforcement learning, and obtains the corresponding experimental target detection result based on the second target information; the adjustment unit 423 adjusts the original target information of the detection image based on the training target detection result and the experimental target detection result, and adjusts the number of samples of the detection image to obtain the adjusted current test set; and, based on the adjusted current test set, adjusts and trains the current marine target pre-training model to obtain the adjusted marine target pre-training model.

[0123] If all test sets have been subjected to the human eye alignment experiment, the target detection module 430 performs marine target detection by adjusting the marine target pre-training model. If not all test sets have been subjected to the human eye alignment experiment, the replacement update module 450 replaces the adjusted marine target pre-training model with the current marine target pre-training model, and then the determination training module 410 and the experiment adjustment module 420 can continue to adjust the marine target pre-training model so that the target detection capability of the marine target pre-training model is fully aligned with the human eye.

[0124] It can be seen that in this embodiment, after the device for marine target detection is trained based on the set target detection deep neural network to obtain the current marine target pre-training model, the current marine target pre-training model can be fine-tuned and trained multiple times according to the human eye alignment experiment to obtain the adjusted marine target pre-training model, and the marine target detection is performed through the adjusted marine target pre-training model. In this way, not only the automated marine target detection is realized, but also the detection capability is fully aligned with the human eye, which can ensure that the experimental results obtained by completing the marine target recognition task are consistent with humans, thereby improving the accuracy of marine target detection.

[0125] Combination Figure 6 The embodiment of the present disclosure provides a device 600 for detecting targets at sea, comprising:

[0126] The processor 1000 and the memory 1001 may also include a communication interface 1002 and a bus 1003. The processor 1000, the communication interface 1002, and the memory 1001 may communicate with each other through the bus 1003. The communication interface 1002 may be used for information transmission. The processor 1000 may call the logic instructions in the memory 1001 to execute the method for marine target detection of the above embodiment.

[0127] In addition, the logic instructions in the above-mentioned memory 1001 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0128] The memory 1001 is a computer-readable storage medium that can be used to store software programs and computer executable programs, such as program instructions / modules corresponding to the method in the embodiment of the present disclosure. The processor 1000 executes the function application and data processing by running the program instructions / modules stored in the memory 1001, that is, the method for marine target detection in the above method embodiment is implemented.

[0129] The memory 1001 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 1001 may include a high-speed random access memory and may also include a non-volatile memory.

[0130] An embodiment of the present disclosure provides a device for detecting a target at sea, comprising: a processor and a memory storing program instructions, wherein the processor is configured to execute a method for detecting a target at sea when executing the program instructions.

[0131] Combination Figure 7 , the embodiment of the present disclosure provides a detection device 700, including: a device body, and the above-mentioned device for detecting targets at sea 400 (600). The device for detecting targets at sea 400 (600) is installed on the device body. The installation relationship described here is not limited to placement inside the product, but also includes installation connections with other components of the product, including but not limited to physical connections, electrical connections or signal transmission connections, etc. Those skilled in the art can understand that the device for detecting targets at sea 400 (600) can be adapted to a feasible device body, thereby realizing other feasible embodiments.

[0132] An embodiment of the present disclosure provides a storage medium storing program instructions, which, when run, execute the above-mentioned method for marine target detection.

[0133] An embodiment of the present disclosure provides a computer program product, which includes a computer program stored on a storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the above-mentioned method for detecting marine targets.

[0134] The above-mentioned storage medium may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0135] The technical solution of the embodiment of the present disclosure can be embodied in the form of a software product, which is stored in a storage medium and includes one or more instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in the embodiment of the present disclosure. The aforementioned storage medium may be a non-transient storage medium, including: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes, or a transient storage medium.

[0136] The above description and the accompanying drawings fully illustrate the embodiments of the present disclosure so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent only possible changes. Unless explicitly required, individual components and functions are optional, and the order of operation may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. The scope of the embodiments of the present disclosure includes the entire scope of the claims, and all available equivalents of the claims. When used in this application, although the terms "first", "second", etc. may be used in this application to describe each element, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, without changing the meaning of the description, the first element can be called the second element, and similarly, the second element can be called the first element, as long as all occurrences of the "first element" are renamed consistently and all occurrences of the "second element" are renamed consistently. The first element and the second element are both elements, but may not be the same element. Moreover, the words used in this application are only used to describe the embodiments and are not used to limit the claims. As used in the description of the embodiments and claims, unless the context clearly indicates, the singular forms "a", "an" and "the" are intended to include the plural forms as well. Similarly, the term "and / or" as used in this application refers to any and all possible combinations of one or more associated listings. In addition, when used in this application, the term "comprise" and its variants "comprises" and / or comprising refer to the presence of stated features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups of these. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the presence of other identical elements in the process, method or device including the elements. In this article, each embodiment may focus on the differences from other embodiments, and the same and similar parts between the embodiments may refer to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can refer to the description of the method part.

[0137] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. The technicians may use different methods for each specific application to implement the described functions, but such implementations should not be considered to exceed the scope of the embodiments of the present disclosure. The technicians may clearly understand that, for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above may refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0138] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units can be only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to implement this embodiment. In addition, each functional unit in the embodiment of the present disclosure may be integrated in a processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit.

[0139] The flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to the embodiment of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. In the description corresponding to the flowchart and the block diagram in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in a different order from the order disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified functions or actions, or may be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. A method for detecting marine targets, characterized in that: include: Determine a subset from the total set of acquired marine target image data as a current test set, and process the current test set based on a current marine target pre-training model to obtain a corresponding training target detection result, wherein the current marine target pre-training model is obtained by training based on a set target detection deep neural network; Acquire the experimental target detection result obtained by performing the human eye alignment experiment according to the current test set, and adjust the current marine target pre-training model according to the training target detection result and the experimental target detection result to obtain the adjusted marine target pre-training model; The maritime target detection is performed by adjusting the maritime target pre-training model.

2. The method according to claim 1, characterized in that The acquisition process of the total set of marine target image data includes: Acquire an open source first marine target image dataset, acquire a second marine target image dataset through infrared imaging or visible light imaging, and acquire a third marine target image dataset through simulation modeling; A total set of marine target image data is formed according to the first marine target image data set, the second marine target image data set and the third marine target image data set, and original target information is annotated for each image.

3. The method according to claim 2, characterized in that The acquiring of the second marine target image data set comprises: Obtain the first real-shot image of a specific marine target through infrared imaging or visible light imaging; Performing image data enhancement processing on the images in the first marine target image data set to obtain a corresponding first enhanced image; The fidelity of the first enhanced image is verified based on the first real-shot image. If the verification passes, the first real-shot image and the first enhanced image are added to the second marine target image data set.

4. The method according to claim 2, characterized in that: The acquiring of the third marine target image data set comprises: Acquire a second real-shot image of a specific marine target; Performing three-dimensional modeling according to a specific marine target, constructing a corresponding marine simulation environment, and generating a second simulation image that matches the target type, imaging conditions, and environmental conditions in the marine simulation environment; The second simulation image is verified for fidelity based on the second real-shot image. If the verification is passed, the second real-shot image and the second simulation image are added to the third marine target image data set.

5. The method according to claim 1, characterized in that The process of obtaining the current marine target pre-training model includes: Dividing the total marine target image data set into two or more marine target image data subsets; A subset of maritime target image data is determined as a training set, and a set target detection deep neural network is trained according to the training set to obtain a current maritime target pre-training model, wherein the training set is different from the current test set.

6. The method according to claim 1, characterized in that The obtaining of the experimental target detection result obtained by performing the human eye alignment experiment according to the current test set includes: Obtaining first target information annotated by the subject on a test image in the current test set, wherein the subject performs image feature learning of marine targets based on the test image carrying the original target information; Verification is performed based on the original target information, the first target information, and second target information annotated by the subject after the verification reinforcement learning on the detection image in the current test set is obtained, wherein the detection image is different from the test image.

7. The method according to claim 6, characterized in that The adjusted maritime target pre-training model comprises: According to the training target detection results and the experimental target detection results, the original target information of the detection image is adjusted, and the number of samples of the detection image is adjusted to obtain an adjusted current test set; According to the adjusted current test set, the current maritime target pre-training model is adjusted and trained to obtain an adjusted maritime target pre-training model.

8. The method according to any one of claims 1 to 7, characterized in that: After obtaining the adjusted pre-trained model of the maritime target, the method further includes: The adjusted maritime target pre-training model is replaced with the current maritime target pre-training model.

9. A device for detecting marine targets, the device comprising a processor and a memory storing program instructions, characterized in that: The processor is configured to perform the method for marine target detection according to any one of claims 1 to 8 when executing the program instructions.

10. A storage medium storing program instructions, characterized in that: When the program instructions are executed, the method for detecting marine targets as described in any one of claims 1 to 8 is executed.

Images