A method and device for infrared ship target recognition

Abstract

The invention discloses an infrared ship target recognition method and device, the method comprising: obtaining image information to be recognized; preprocessing the image information to be recognized to obtain initial recognition image information; using a target image recognition model to recognize the initial recognition image information to obtain target infrared ship recognition result information.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an infrared ship target recognition method and device. Background Art

[0002] In recent years, with the continuous development of infrared detection technology, the recognition of infrared ship targets has attracted much attention in the fields of military reconnaissance and civilian detection. Compared with visible light images, infrared images have the advantages of strong anti-interference ability and are not affected by light. However, they also have the problems of few features such as color texture and low background contrast. Especially for marine ship targets, due to their long shooting distance, they are easily submerged in sea clutter. In the background of near-shore ports, there are many ships parked, they are closely arranged and the targets are small. The above situations will lead to imbalanced samples of infrared ship recognition. Realizing infrared ship target recognition under imbalanced samples is a very challenging task. Traditional ship target recognition usually adopts the method of manually designed features for extraction. The method is single, inefficient and poor in generalization ability. The currently widely used deep learning method relies heavily on a large number of labeled samples to train the network to improve performance. However, slicing and labeling sample images is very labor-intensive and resource-intensive. Therefore, an infrared ship target recognition method and device are provided to improve the recognition accuracy and efficiency of imbalanced infrared ships in complex scenes. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide an infrared ship target recognition method and device which are beneficial to improving the recognition accuracy and efficiency of unbalanced infrared ships in complex scenes.

[0004] In order to solve the above technical problems, the first aspect of the embodiment of the present invention discloses an infrared ship target recognition method, the method comprising:

[0005] Obtaining image information to be identified;

[0006] Preprocessing the image information to be recognized to obtain initial recognition image information;

[0007] The target image recognition model is used to perform recognition processing on the initial recognition image information to obtain target infrared ship recognition result information.

[0008] A second aspect of an embodiment of the present invention discloses an infrared ship target recognition device, the device comprising:

[0009] An acquisition module, used for acquiring image information to be identified;

[0010] A first processing module, used for preprocessing the image information to be recognized to obtain initial recognition image information;

[0011] The second processing module is used to use the target image recognition model to perform recognition processing on the initial recognition image information to obtain target infrared ship recognition result information.

[0012] The third aspect of the present invention discloses another infrared ship target recognition device, the device comprising:

[0013] A memory storing executable program code;

[0014] a processor coupled to the memory;

[0015] The processor calls the executable program code stored in the memory to execute part or all of the steps in the infrared ship target recognition method disclosed in the first aspect of the embodiment of the present invention.

[0016] The fourth aspect of the present invention discloses a computer-readable storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute some or all of the steps in the infrared ship target recognition method disclosed in the first aspect of an embodiment of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 It is a flow chart of an infrared ship target recognition method disclosed in an embodiment of the present invention;

[0019] Figure 2 It is a structural schematic diagram of an infrared ship target recognition device disclosed in an embodiment of the present invention;

[0020] Figure 3 It is a structural schematic diagram of another infrared ship target recognition device disclosed in an embodiment of the present invention;

[0021] Figure 4 It is a structural schematic diagram of a sample amplification model disclosed in an embodiment of the present invention;

[0022] Figure 5 It is a structural schematic diagram of a target image recognition model disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or equipment that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or equipment.

[0025] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0026] The present invention discloses an infrared ship target recognition method and device, which are beneficial to improving the recognition accuracy and efficiency of unbalanced infrared ships in complex scenes. Detailed descriptions are given below.

[0027] Embodiment 1

[0028] See also Figure 1 , Figure 1 1 is a flow chart of an infrared ship target recognition method disclosed in an embodiment of the present invention. Figure 1 The infrared ship target recognition method described is applied to a management system, such as a local server or a cloud server for management, and the embodiments of the present invention do not limit this. Figure 1 As shown, the infrared ship target recognition method may include the following operations:

[0029] 101. Obtain information of an image to be identified.

[0030] 102. Preprocess the image information to be recognized to obtain initial recognition image information.

[0031] 103. The target image recognition model is used to process the initial recognition image information to obtain the target infrared ship recognition result information.

[0032] It should be noted that the above-mentioned image information to be identified may be acquired by an infrared camera or other infrared devices, and the embodiment of the present invention does not limit this.

[0033] In this optional embodiment, as an optional implementation manner, the above-mentioned preprocessing of the image information to be recognized to obtain the initial recognition image information includes:

[0034] Perform detection processing on the image information to be identified to obtain the detected ship image information;

[0035] Correcting the detected ship image information to obtain corrected ship image information;

[0036] The corrected ship image information is processed by subject contour segmentation to obtain initial recognition image information.

[0037] It should be noted that the above detection of the image information to be identified may be based on the detection and identification of the contour features of the ship by a neural network, which is not limited in the embodiment of the present invention.

[0038] It should be noted that the above correction processing may be to remove unclear abnormal pixels in the detected ship image information, which is not limited in the embodiment of the present invention.

[0039] It should be noted that the above-mentioned main body contour segmentation processing of the corrected ship image information is to identify and segment the main body contour of the ship for further segmentation processing, which can be implemented based on a neural network or a deep learning algorithm, and the embodiment of the present invention is not limited thereto.

[0040] It should be noted that the average recognition accuracy of the infrared ship target recognition method of the present application can reach 92.28%.

[0041] It can be seen that implementing the infrared ship target recognition method described in the embodiment of the present invention is conducive to improving the recognition accuracy and efficiency of unbalanced infrared ships in complex scenes.

[0042] In an optional embodiment, if Figure 5 As shown, the target image recognition model includes a seventh convolution module, a first feature extraction module, a second feature extraction module, a third feature extraction module, a first fusion module, a multi-scale topology module, a first overlay mapping module, a second overlay mapping module, a third overlay mapping module, a fourth overlay mapping module, a second connection module, and a second activation module; wherein,

[0043] The input end of the seventh convolution module is connected to the second model input of the target image recognition model, and the output end of the seventh convolution module is respectively connected to the input end of the first feature extraction module, the input end of the second feature extraction module, the input end of the third feature extraction module and the input end of the first fusion module; the output end of the first feature extraction module, the output end of the second feature extraction module and the output end of the third feature extraction module are all connected to the input end of the first fusion module; the output end of the first fusion module is connected to the input end of the multi-scale topology module; the output end of the multi-scale topology module is connected to the input end of the first overlay mapping module; the output end of the first overlay mapping module is connected to the input end of the second overlay mapping module; the output end of the second overlay mapping module is connected to the input end of the third overlay mapping module; the output end of the third overlay mapping module is connected to the input end of the fourth overlay mapping module; the output end of the fourth overlay mapping module is connected to the input end of the second connection module; the output end of the second connection module is connected to the input end of the second activation module; the output end of the second activation module is connected to the second model output of the target image recognition model.

[0044] It should be noted that the number of channels of the seventh convolution module is 256, and the convolution kernel size is 1*3, which is not limited in the embodiment of the present invention.

[0045] It should be noted that the above-mentioned first fusion module is constructed based on matrix summation, which is not limited in the embodiment of the present invention.

[0046] It should be noted that the above-mentioned multi-scale topology module is constructed based on the multi-scale channel attention mechanism, which is not limited in the embodiment of the present invention.

[0047] It should be noted that the above-mentioned second connection module is constructed based on the fully connected layer, which is not limited in the embodiment of the present invention.

[0048] It should be noted that the second activation module is constructed based on the weighted sum of the sigmoid activation function and the softmax activation function, which is not limited in the embodiment of the present invention.

[0049] It should be noted that the model architectures of the first feature extraction module, the second feature extraction module, and the third feature extraction module are consistent, and the embodiment of the present invention does not limit this.

[0050] It should be noted that the model architectures of the first overlay mapping module, the second overlay mapping module, the third overlay mapping module, and the fourth overlay mapping module are consistent, and the embodiment of the present invention does not limit this. Further, it can prevent gradient disappearance and improve the utilization rate of features.

[0051] It can be seen that implementing the infrared ship target recognition method described in the embodiment of the present invention is conducive to improving the recognition accuracy and efficiency of unbalanced infrared ships in complex scenes.

[0052] In another optional embodiment, the first feature extraction module includes a first convolution unit, a second convolution unit, and a third convolution unit; wherein,

[0053] The input end of the first convolution unit is connected to the input end of the seventh convolution module, the output end of the first convolution unit is connected to the input end of the second convolution unit; the output end of the second convolution unit is connected to the input end of the third convolution unit; the output end of the third convolution unit is connected to the input end of the first fusion module;

[0054] The first overlay mapping module includes a fourth convolution unit, a fifth convolution unit, a first normalization unit, a second normalization unit, a first activation unit, a second activation unit, and a first fusion unit; wherein,

[0055] The input end of the fourth convolution unit is connected to the input end of the multi-scale topology module, and the output end of the fourth convolution unit is connected to the input end of the first normalization unit; the output end of the first normalization unit is connected to the input end of the first activation unit; the output end of the first activation unit is connected to the input end of the fifth convolution unit; the output end of the fifth convolution unit is connected to the input end of the second normalization unit; the input end of the first fusion unit is respectively connected to the output end of the second normalization unit and the input end of the multi-scale topology module, and the output end of the first fusion unit is connected to the input end of the second activation unit; the output end of the second activation unit is connected to the input end of the second overlay mapping module.

[0056] It should be noted that the number of channels of the first convolution unit, the second convolution unit, and the third convolution unit are all 256, and the convolution kernel sizes are 1*7, 1*5, and 1*3, respectively, which is not limited in the embodiment of the present invention.

[0057] It should be noted that the number of channels of the fourth convolution unit and the fifth convolution unit are both 256, and the size of the convolution kernel is 1*3.

[0058] It should be noted that the above-mentioned first normalization unit and second normalization unit are constructed based on the batch normalization layer, which is not limited in the embodiment of the present invention.

[0059] It should be noted that the first activation unit and the second activation unit are constructed based on sigmoid activation, which is not limited in the embodiment of the present invention.

[0060] It should be noted that the above-mentioned first fusion unit is constructed based on matrix summation, which is not limited in the embodiment of the present invention.

[0061] It can be seen that implementing the infrared ship target recognition method described in the embodiment of the present invention is conducive to improving the recognition accuracy and efficiency of unbalanced infrared ships in complex scenes.

[0062] In yet another optional embodiment, the target image recognition model is obtained based on the following steps:

[0063] Obtain initial image training information;

[0064] Performing an amplification process on the initial image training information to obtain target image training information; the target image training information includes P target training image information;

[0065] The initial image recognition model is trained using the target image training information to obtain a target image recognition model.

[0066] It should be noted that the above-mentioned initial image training information includes no more than 9,000 pieces of initial training image information. Further, the initial training image information includes infrared images of warships, sailboats, cruise ships, bulk carriers, fishing boats, yachts, container ships and other types of ships, which are not limited in the embodiment of the present invention. Further, its pixel size is 384*288, which is not limited in the embodiment of the present invention.

[0067] It should be noted that the above P is a positive integer not less than 10,000.

[0068] It can be seen that implementing the infrared ship target recognition method described in the embodiment of the present invention is conducive to improving the recognition accuracy and efficiency of unbalanced infrared ships in complex scenes.

[0069] In yet another optional embodiment, the initial image training information is augmented to obtain the target image training information, including:

[0070] Slicing the initial image training information to obtain first image training information; the first image training information includes M first training image information;

[0071] Classify the first image training information to obtain second image training information and third image training information; the second image training information includes N second training image information; the third image training information includes L third training image information;

[0072] Performing labeling processing on the second image training information to obtain labeled image training information; the labeled image training information includes N labeled training image information;

[0073] Using the sample augmentation model to generate samples for the labeled image training information and the third image training information to obtain augmented image training information; the augmented image training information includes M augmented training image information;

[0074] The labeled image training information, the third image training information and the augmented image training information are fused to obtain the target image training information.

[0075] It should be noted that the above slicing processing of the initial image training information is to segment the initial training image information according to the ship type, and to separate the image block of each ship as an independent first training image information. Further, the above M is a positive integer not less than 9000.

[0076] It should be noted that the classification process of the first image training information is to randomly divide the first training image information in the first image training information into two sets, namely, the second image training information and the third image training information. Preferably, the above N is 5000. Furthermore, the above L=MN.

[0077] It should be noted that the above-mentioned annotation processing of the second image training information is labeling processing of the second training image information, which can be completed by labeling tools such as LabelImg and RectLabel, or can be completed by user annotation, and the embodiment of the present invention is not limited thereto.

[0078] It should be noted that the above-mentioned use of the sample augmentation model to generate samples of the labeled image training information and the third image training information is to learn data characteristics from real samples through the sample augmentation model, and map the learned data characteristics to a two-dimensional space, thereby obtaining false samples, that is, the sample augmentation model generates false samples in an attempt to deceive the initial image recognition model. The initial image recognition model continuously enhances its learning ability to identify false samples, thereby achieving Nash equilibrium, that is, obtaining the target image recognition model.

[0079] It should be noted that the above fusion processing of the labeled image training information, the third image training information and the augmented image training information is to place the above three data information into the same data document, thereby forming one data information, namely, the target image training information.

[0080] It can be seen that implementing the infrared ship target recognition method described in the embodiment of the present invention is conducive to improving the recognition accuracy and efficiency of unbalanced infrared ships in complex scenes.

[0081] In an optional embodiment, if Figure 4 As shown, the sample amplification model includes a first convolution module, a second convolution module, a third convolution module, a fourth convolution module, a fifth convolution module, a sixth convolution module, a first normalization module, a first activation module, a first sampling module, a first flattening module, and a first connection module; wherein,

[0082] The input end of the first convolution module is connected to the first model input of the sample augmentation model, and the output end of the first convolution module is connected to the input end of the first normalization module; the output end of the first normalization module is connected to the input end of the first activation module; the output end of the first activation module is connected to the input end of the first sampling module; the output end of the first sampling module is connected to the input end of the second convolution module; the output end of the second convolution module is connected to the input end of the third convolution module; the output end of the third convolution module is connected to the input end of the fourth convolution module; the output end of the fourth convolution module is connected to the input end of the fifth convolution module; the output end of the fifth convolution module is connected to the input end of the sixth convolution module; the output end of the sixth convolution module is connected to the input end of the first flat module; the output end of the first flat module is connected to the input end of the first connection module; the output end of the first connection module is connected to the first model output of the sample augmentation model.

[0083] It should be noted that the number of channels of the first convolution module, the second convolution module, the third convolution module, the fourth convolution module, the fifth convolution module, and the sixth convolution module are 256, 128, 64, 32, 16, and 3, respectively, and the convolution kernel size is 1*3.

[0084] It should be noted that the above-mentioned first normalization module is constructed based on the batch normalization layer, which is not limited in the embodiment of the present invention.

[0085] It should be noted that the above-mentioned first activation module is constructed based on the relu activation function, which is not limited in the embodiment of the present invention.

[0086] It should be noted that the above-mentioned first sampling module is an up-sampling operation module, which is not limited in the embodiment of the present invention.

[0087] It should be noted that the first flat module is a flatten operation layer, which is not limited in the embodiment of the present invention.

[0088] It should be noted that the above-mentioned first connection module is constructed based on the fully connected layer, which is not limited in the embodiment of the present invention.

[0089] It should be noted that the image dimensions of the first model input and the first model output are 256*64 and 1*1000 respectively, which is not limited in the embodiment of the present invention.

[0090] It can be seen that implementing the infrared ship target recognition method described in the embodiment of the present invention is conducive to improving the recognition accuracy and efficiency of unbalanced infrared ships in complex scenes.

[0091] In another optional embodiment, the initial image recognition model is trained using the target image training information to obtain the target image recognition model, including:

[0092] Selecting a number of target training image information from the target image training information to obtain target training sample information; the target training sample information includes Q target sample image information; the Q target sample image information includes X augmented training image information, Y labeled training image information and Z third training image information;

[0093] Using the target training sample information to train the initial image recognition model, obtaining a training image recognition model and training result information; the training result information includes X first training result values, Y second training result values, and Z third training result values;

[0094] The loss function is used to calculate and process the training result information to obtain the loss function value;

[0095] Among them, the loss function is:

[0096]

[0097] Where L is the loss function value; x i is the confidence probability corresponding to the i-th augmented training image information input to the initial image recognition model; y j is the confidence probability corresponding to the jth labeled training image information input to the initial image recognition model; h xp is the confidence probability corresponding to the third training image information input to the initial image recognition model; i yp is the first training result value corresponding to the i-th augmented training image information input to the initial image recognition model; j is the second training result value corresponding to the jth labeled training image information input to the initial image recognition model; zp h is the third training result value corresponding to the third training image information input to the initial image recognition model for the hth time; a, b, c, d and e are the first constant value, the second constant value, the third constant value, the fourth constant value and the fifth constant value respectively;

[0098] Determine whether the loss function value meets the training termination condition and obtain the conditional judgment result;

[0099] When the result of the conditional judgment is no, the training image recognition model is determined as a new initial image recognition model, and a trigger is executed to select a number of target training image information from the target image training information to obtain target training sample information;

[0100] When the conditional judgment result is yes, the training image recognition model is determined to be the target image recognition model.

[0101] Label image training information, third image training information and augmented image training information

[0102] It should be noted that the above-mentioned selecting a number of target training image information from the target image training information to identify randomly selecting X augmented training image information, Y annotated training image information and Z third training image information from the augmented image training information, the annotated image training information and the third image training information to obtain Q target sample image information. Further, the above-mentioned X, Y and Z are all positive integers greater than 1000.

[0103] It should be noted that the above-mentioned training process of the initial image recognition model using the target training sample information is to sequentially input X augmented training image information, Y labeled training image information and Z third training image information into the initial image recognition model for training process.

[0104] It should be noted that when the initial image recognition model uses the second activation module to classify the augmented training image information and the third training image information, the weight of the sigmoid activation function is 1, and the weight of the softmax activation function is 0, so that the final first training result value and the third training result value can be obtained. Further, when the initial image recognition model uses the second activation module to classify the labeled training image information, the weight of the sigmoid activation function is 0, and the weight of the softmax activation function is 1, so that the second training result value can be obtained.

[0105] It should be noted that the above confidence probability is set by the user when forming the training image information, and its value is between 0-1. It is specifically a random value given by the system and adjusted by the user, and the embodiment of the present invention does not limit it.

[0106] It should be noted that the above-mentioned first constant value, second constant value, third constant value, fourth constant value and fifth constant value are all constants, which can be set by the user or adopt the default values ​​given by the system, and the embodiment of the present invention does not limit them.

[0107] It should be noted that the above training termination condition is the data convergence consisting of the loss function value and the historical loss function value.

[0108] It can be seen that implementing the infrared ship target recognition method described in the embodiment of the present invention is conducive to improving the recognition accuracy and efficiency of unbalanced infrared ships in complex scenes.

[0109] Embodiment 2

[0110] See also Figure 2 , Figure 2 : is a schematic diagram of the structure of an infrared ship target recognition device disclosed in an embodiment of the present invention. Figure 2 The described device can be applied to a warehouse management system, such as a local server or cloud server for infrared ship target recognition management of warehouse logistics, etc., and the embodiments of the present invention are not limited thereto. Figure 2 As shown, the device may include:

[0111] An acquisition module 201 is used to acquire image information to be identified;

[0112] The first processing module 202 is used to pre-process the image information to be recognized to obtain initial recognition image information;

[0113] The second processing module 203 is used to perform recognition processing on the initial recognition image information using the target image recognition model to obtain target infrared ship recognition result information.

[0114] It can be seen that implementation Figure 2 The described infrared ship target recognition device is beneficial to improving the recognition accuracy and efficiency of unbalanced infrared ships in complex scenes.

[0115] In another optional embodiment, Figure 2 As shown, the target image recognition model includes a seventh convolution module, a first feature extraction module, a second feature extraction module, a third feature extraction module, a first fusion module, a multi-scale topology module, a first overlay mapping module, a second overlay mapping module, a third overlay mapping module, a fourth overlay mapping module, a second connection module, and a second activation module; wherein,

[0116] The input end of the seventh convolution module is connected to the second model input of the target image recognition model, and the output end of the seventh convolution module is respectively connected to the input end of the first feature extraction module, the input end of the second feature extraction module, the input end of the third feature extraction module and the input end of the first fusion module; the output end of the first feature extraction module, the output end of the second feature extraction module and the output end of the third feature extraction module are all connected to the input end of the first fusion module; the output end of the first fusion module is connected to the input end of the multi-scale topology module; the output end of the multi-scale topology module is connected to the input end of the first overlay mapping module; the output end of the first overlay mapping module is connected to the input end of the second overlay mapping module; the output end of the second overlay mapping module is connected to the input end of the third overlay mapping module; the output end of the third overlay mapping module is connected to the input end of the fourth overlay mapping module; the output end of the fourth overlay mapping module is connected to the input end of the second connection module; the output end of the second connection module is connected to the input end of the second activation module; the output end of the second activation module is connected to the second model output of the target image recognition model.

[0117] It can be seen that implementation Figure 2 The described infrared ship target recognition device is beneficial to improving the recognition accuracy and efficiency of unbalanced infrared ships in complex scenes.

[0118] In yet another optional embodiment, Figure 2As shown, the first feature extraction module includes a first convolution unit, a second convolution unit, and a third convolution unit; wherein,

[0119] The input end of the first convolution unit is connected to the input end of the seventh convolution module, the output end of the first convolution unit is connected to the input end of the second convolution unit; the output end of the second convolution unit is connected to the input end of the third convolution unit; the output end of the third convolution unit is connected to the input end of the first fusion module;

[0120] The first overlay mapping module includes a fourth convolution unit, a fifth convolution unit, a first normalization unit, a second normalization unit, a first activation unit, a second activation unit, and a first fusion unit; wherein,

[0121] The input end of the fourth convolution unit is connected to the input end of the multi-scale topology module, and the output end of the fourth convolution unit is connected to the input end of the first normalization unit; the output end of the first normalization unit is connected to the input end of the first activation unit; the output end of the first activation unit is connected to the input end of the fifth convolution unit; the output end of the fifth convolution unit is connected to the input end of the second normalization unit; the input end of the first fusion unit is respectively connected to the output end of the second normalization unit and the input end of the multi-scale topology module, and the output end of the first fusion unit is connected to the input end of the second activation unit; the output end of the second activation unit is connected to the input end of the second overlay mapping module.

[0122] It can be seen that implementation Figure 2 The described infrared ship target recognition device is beneficial to improving the recognition accuracy and efficiency of unbalanced infrared ships in complex scenes.

[0123] In yet another optional embodiment, Figure 2 As shown, the target image recognition model is obtained by the second processing module 203 executing the following steps:

[0124] Obtain initial image training information;

[0125] Performing an amplification process on the initial image training information to obtain target image training information; the target image training information includes P target training image information;

[0126] The initial image recognition model is trained using the target image training information to obtain a target image recognition model.

[0127] It can be seen that implementation Figure 2 The described infrared ship target recognition device is beneficial to improving the recognition accuracy and efficiency of unbalanced infrared ships in complex scenes.

[0128] In yet another optional embodiment, Figure 2 As shown, the second processing module 203 performs amplification processing on the initial image training information to obtain target image training information, including:

[0129] Slicing the initial image training information to obtain first image training information; the first image training information includes M first training image information;

[0130] Classify the first image training information to obtain second image training information and third image training information; the second image training information includes N second training image information; the third image training information includes L third training image information;

[0131] Performing labeling processing on the second image training information to obtain labeled image training information; the labeled image training information includes N labeled training image information;

[0132] Using the sample augmentation model to generate samples for the labeled image training information and the third image training information to obtain augmented image training information; the augmented image training information includes M augmented training image information;

[0133] The labeled image training information, the third image training information and the augmented image training information are fused to obtain the target image training information.

[0134] It can be seen that implementation Figure 2 The described infrared ship target recognition device is beneficial to improving the recognition accuracy and efficiency of unbalanced infrared ships in complex scenes.

[0135] In yet another optional embodiment, Figure 2 As shown, the sample amplification model includes a first convolution module, a second convolution module, a third convolution module, a fourth convolution module, a fifth convolution module, a sixth convolution module, a first normalization module, a first activation module, a first sampling module, a first flattening module, and a first connection module; wherein,

[0136] The input end of the first convolution module is connected to the first model input of the sample augmentation model, and the output end of the first convolution module is connected to the input end of the first normalization module; the output end of the first normalization module is connected to the input end of the first activation module; the output end of the first activation module is connected to the input end of the first sampling module; the output end of the first sampling module is connected to the input end of the second convolution module; the output end of the second convolution module is connected to the input end of the third convolution module; the output end of the third convolution module is connected to the input end of the fourth convolution module; the output end of the fourth convolution module is connected to the input end of the fifth convolution module; the output end of the fifth convolution module is connected to the input end of the sixth convolution module; the output end of the sixth convolution module is connected to the input end of the first flat module; the output end of the first flat module is connected to the input end of the first connection module; the output end of the first connection module is connected to the first model output of the sample augmentation model.

[0137] It can be seen that implementation Figure 2The described infrared ship target recognition device is beneficial to improving the recognition accuracy and efficiency of unbalanced infrared ships in complex scenes.

[0138] In yet another optional embodiment, Figure 2 As shown, the second processing module 203 uses the target image training information to train the initial image recognition model to obtain the target image recognition model, including:

[0139] Selecting a number of target training image information from the target image training information to obtain target training sample information; the target training sample information includes Q target sample image information; the Q target sample image information includes X augmented training image information, Y labeled training image information and Z third training image information;

[0140] Using the target training sample information to train the initial image recognition model, obtaining a training image recognition model and training result information; the training result information includes X first training result values, Y second training result values, and Z third training result values;

[0141] The loss function is used to calculate and process the training result information to obtain the loss function value;

[0142] Among them, the loss function is:

[0143]

[0144] Where L is the loss function value; x i is the confidence probability corresponding to the i-th augmented training image information input to the initial image recognition model; y j is the confidence probability corresponding to the jth labeled training image information input to the initial image recognition model; h xp is the confidence probability corresponding to the third training image information input to the initial image recognition model; i yp is the first training result value corresponding to the i-th augmented training image information input to the initial image recognition model; j is the second training result value corresponding to the jth labeled training image information input to the initial image recognition model; zp h is the third training result value corresponding to the third training image information input to the initial image recognition model for the hth time; a, b, c, d and e are the first constant value, the second constant value, the third constant value, the fourth constant value and the fifth constant value respectively;

[0145] Determine whether the loss function value meets the training termination condition and obtain the conditional judgment result;

[0146] When the result of the conditional judgment is no, the training image recognition model is determined as a new initial image recognition model, and a trigger is executed to select a number of target training image information from the target image training information to obtain target training sample information;

[0147] When the conditional judgment result is yes, the training image recognition model is determined to be the target image recognition model.

[0148] It can be seen that implementation Figure 2 The described infrared ship target recognition device is beneficial to improving the recognition accuracy and efficiency of unbalanced infrared ships in complex scenes.

[0149] Embodiment 3

[0150] See also Figure 3 , Figure 3 : is a schematic diagram of the structure of another infrared ship target recognition device disclosed in an embodiment of the present invention. Figure 3 The described device can be applied to a warehouse management system, such as a local server or cloud server for infrared ship target recognition management of warehouse logistics, etc., and the embodiments of the present invention are not limited thereto. Figure 3 As shown, the device may include:

[0151] A memory 301 storing executable program codes;

[0152] a processor 302 coupled to the memory 301;

[0153] The processor 302 calls the executable program code stored in the memory 301 to execute the steps in the infrared ship target recognition method described in the first embodiment.

[0154] Embodiment 4

[0155] An embodiment of the present invention discloses a computer-readable storage medium storing a computer program for electronic data exchange, wherein the computer program enables a computer to execute the steps of the infrared ship target recognition method described in the first embodiment.

[0156] Embodiment 5

[0157] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute the steps in the infrared ship target recognition method described in the first embodiment.

[0158] The device embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, i.e., they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0159] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or partly contributed to the prior art in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0160] Finally, it should be noted that the infrared ship target recognition method and device disclosed in the embodiments of the present invention disclose only the preferred embodiments of the present invention, which are only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An infrared ship target recognition method, characterized in that: The method comprises: Obtaining image information to be identified; Preprocessing the image information to be recognized to obtain initial recognition image information; The preprocessing of the image information to be recognized to obtain the initial recognized image information includes: Performing detection processing on the image information to be identified to obtain detected ship image information; Correcting the detected ship image information to obtain corrected ship image information; Performing subject contour segmentation processing on the corrected ship image information to obtain initial recognition image information; Using a target image recognition model to perform recognition processing on the initial recognition image information to obtain target infrared ship recognition result information; Among them, the target image recognition model includes a seventh convolution module, a first feature extraction module, a second feature extraction module, a third feature extraction module, a first fusion module, a multi-scale topology module, a first overlay mapping module, a second overlay mapping module, a third overlay mapping module, a fourth overlay mapping module, a second connection module, and a second activation module; wherein the number of channels of the seventh convolution module is 256, and the convolution kernel size is 1*3; the first fusion module is constructed based on matrix summation; the multi-scale topology module is constructed based on a multi-scale channel attention mechanism; the second connection module is constructed based on a fully connected layer; the second activation module is constructed based on the weighted sum of a sigmoid activation function and a softmax activation function; The input end of the seventh convolution module is connected to the second model input of the target image recognition model, and the output end of the seventh convolution module is respectively connected to the input end of the first feature extraction module, the input end of the second feature extraction module, the input end of the third feature extraction module and the input end of the first fusion module; the output end of the first feature extraction module, the output end of the second feature extraction module and the output end of the third feature extraction module are all connected to the input end of the first fusion module; the output end of the first fusion module is connected to the input end of the multi-scale topology module; the output end of the multi-scale topology module is connected to the input end of the first overlay mapping module; the output end of the first overlay mapping module is connected to the input end of the second overlay mapping module; the output end of the second overlay mapping module is connected to the input end of the third overlay mapping module; the output end of the third overlay mapping module is connected to the input end of the fourth overlay mapping module; the output end of the fourth overlay mapping module is connected to the input end of the second connection module; the output end of the second connection module is connected to the input end of the second activation module; the output end of the second activation module is connected to the second model output of the target image recognition model; The first feature extraction module includes a first convolution unit, a second convolution unit, and a third convolution unit; the number of channels of the first convolution unit, the second convolution unit, and the third convolution unit are all 256, and the convolution kernel sizes are 1*7, 1*5, and 1*3, respectively; The input end of the first convolution unit is connected to the input end of the seventh convolution module, the output end of the first convolution unit is connected to the input end of the second convolution unit; the output end of the second convolution unit is connected to the input end of the third convolution unit; the output end of the third convolution unit is connected to the input end of the first fusion module; The first overlay mapping module includes a fourth convolution unit, a fifth convolution unit, a first normalization unit, a second normalization unit, a first activation unit, a second activation unit, and a first fusion unit; wherein the number of channels of the fourth convolution unit and the fifth convolution unit are both 256, and the convolution kernel size is 1*3; the first normalization unit and the second normalization unit are constructed based on a batch normalization layer; the first activation unit and the second activation unit are constructed based on sigmoid activation; the first fusion unit is constructed based on matrix summation; The input end of the fourth convolution unit is connected to the input end of the multi-scale topology module, and the output end of the fourth convolution unit is connected to the input end of the first normalization unit; the output end of the first normalization unit is connected to the input end of the first activation unit; the output end of the first activation unit is connected to the input end of the fifth convolution unit; the output end of the fifth convolution unit is connected to the input end of the second normalization unit; the input end of the first fusion unit is respectively connected to the output end of the second normalization unit and the input end of the multi-scale topology module, and the output end of the first fusion unit is connected to the input end of the second activation unit; the output end of the second activation unit is connected to the input end of the second overlay mapping module.

2. The infrared ship target recognition method according to claim 1, characterized in that: The target image recognition model is obtained based on the following steps: Obtain initial image training information; Performing an amplification process on the initial image training information to obtain target image training information; the target image training information includes P target training image information; The target image training information is used to train the initial image recognition model to obtain the target image recognition model.

3. The infrared ship target recognition method according to claim 2, characterized in that: The amplifying the initial image training information to obtain the target image training information includes: Slicing the initial image training information to obtain first image training information; the first image training information includes M first training image information; Classify the first image training information to obtain second image training information and third image training information; the second image training information includes N second training image information; the third image training information includes L third training image information; Performing labeling processing on the second image training information to obtain labeled image training information; the labeled image training information includes the N labeled training image information; Using a sample augmentation model to generate samples for the labeled image training information and the third image training information to obtain augmented image training information; the augmented image training information includes the M augmented training image information; The labeled image training information, the third image training information and the augmented image training information are fused to obtain target image training information.

4. The infrared ship target recognition method according to claim 3 is characterized in that: The sample amplification model includes a first convolution module, a second convolution module, a third convolution module, a fourth convolution module, a fifth convolution module, a sixth convolution module, a first normalization module, a first activation module, a first sampling module, a first flattening module, and a first connection module; wherein, The input end of the first convolution module is connected to the first model input of the sample amplification model, and the output end of the first convolution module is connected to the input end of the first normalization module; the output end of the first normalization module is connected to the input end of the first activation module; the output end of the first activation module is connected to the input end of the first sampling module; the output end of the first sampling module is connected to the input end of the second convolution module; the output end of the second convolution module is connected to the input end of the third convolution module; the output end of the third convolution module is connected to the input end of the fourth convolution module; the output end of the fourth convolution module is connected to the input end of the fifth convolution module; the output end of the fifth convolution module is connected to the input end of the sixth convolution module; the output end of the sixth convolution module is connected to the input end of the first flat module; the output end of the first flat module is connected to the input end of the first connection module; the output end of the first connection module is connected to the first model output of the sample amplification model.

5. The infrared ship target recognition method according to claim 2, characterized in that: The step of training the initial image recognition model using the target image training information to obtain the target image recognition model includes: Selecting a number of target training image information from the target image training information to obtain target training sample information; the target training sample information includes Q target sample image information; the Q target sample image information includes X augmented training image information, Y labeled training image information and Z third training image information; Using the target training sample information to train the initial image recognition model, obtaining a training image recognition model and training result information; the training result information includes the X first training result values, the Y second training result values, and the Z third training result values; Using a loss function to calculate the training result information to obtain a loss function value; Wherein, the loss function is: Wherein, L is the loss function value; x i is the confidence probability corresponding to the i-th augmented training image information input to the initial image recognition model; y j is the confidence probability corresponding to the jth labeled training image information input to the initial image recognition model; h xp is the confidence probability corresponding to the hth third training image information input to the initial image recognition model; i yp is the first training result value corresponding to the i-th augmented training image information input to the initial image recognition model; j is the second training result value corresponding to the j-th labeled training image information input to the initial image recognition model; zp h is the third training result value corresponding to the third training image information input to the initial image recognition model for the hth time; a, b, c, d and e are the first constant value, the second constant value, the third constant value, the fourth constant value and the fifth constant value respectively; Determine whether the loss function value satisfies the training termination condition, and obtain a conditional judgment result; When the result of the conditional judgment is no, the training image recognition model is determined as a new initial image recognition model, and the step of selecting a plurality of target training image information from the target image training information is triggered to obtain target training sample information; When the conditional judgment result is yes, the training image recognition model is determined to be the target image recognition model.

6. An infrared ship target recognition device, characterized in that: The device comprises: An acquisition module, used for acquiring image information to be identified; A first processing module, used for preprocessing the image information to be recognized to obtain initial recognition image information; The preprocessing of the image information to be recognized to obtain the initial recognized image information includes: Performing detection processing on the image information to be identified to obtain detected ship image information; Correcting the detected ship image information to obtain corrected ship image information; Performing subject contour segmentation processing on the corrected ship image information to obtain initial recognition image information; The second processing module is used to perform recognition processing on the initial recognition image information using a target image recognition model to obtain target infrared ship recognition result information; Among them, the target image recognition model includes a seventh convolution module, a first feature extraction module, a second feature extraction module, a third feature extraction module, a first fusion module, a multi-scale topology module, a first overlay mapping module, a second overlay mapping module, a third overlay mapping module, a fourth overlay mapping module, a second connection module, and a second activation module; wherein the number of channels of the seventh convolution module is 256, and the convolution kernel size is 1*3; the first fusion module is constructed based on matrix summation; the multi-scale topology module is constructed based on a multi-scale channel attention mechanism; the second connection module is constructed based on a fully connected layer; the second activation module is constructed based on the weighted sum of a sigmoid activation function and a softmax activation function; The input end of the seventh convolution module is connected to the second model input of the target image recognition model, and the output end of the seventh convolution module is respectively connected to the input end of the first feature extraction module, the input end of the second feature extraction module, the input end of the third feature extraction module and the input end of the first fusion module; the output end of the first feature extraction module, the output end of the second feature extraction module and the output end of the third feature extraction module are all connected to the input end of the first fusion module; the output end of the first fusion module is connected to the input end of the multi-scale topology module; the output end of the multi-scale topology module is connected to the input end of the first overlay mapping module; the output end of the first overlay mapping module is connected to the input end of the second overlay mapping module; the output end of the second overlay mapping module is connected to the input end of the third overlay mapping module; the output end of the third overlay mapping module is connected to the input end of the fourth overlay mapping module; the output end of the fourth overlay mapping module is connected to the input end of the second connection module; the output end of the second connection module is connected to the input end of the second activation module; the output end of the second activation module is connected to the second model output of the target image recognition model; The first feature extraction module includes a first convolution unit, a second convolution unit, and a third convolution unit; the number of channels of the first convolution unit, the second convolution unit, and the third convolution unit are all 256, and the convolution kernel sizes are 1*7, 1*5, and 1*3, respectively; The input end of the first convolution unit is connected to the input end of the seventh convolution module, the output end of the first convolution unit is connected to the input end of the second convolution unit; the output end of the second convolution unit is connected to the input end of the third convolution unit; the output end of the third convolution unit is connected to the input end of the first fusion module; The first overlay mapping module includes a fourth convolution unit, a fifth convolution unit, a first normalization unit, a second normalization unit, a first activation unit, a second activation unit, and a first fusion unit; wherein the number of channels of the fourth convolution unit and the fifth convolution unit are both 256, and the convolution kernel size is 1*3; the first normalization unit and the second normalization unit are constructed based on a batch normalization layer; the first activation unit and the second activation unit are constructed based on sigmoid activation; the first fusion unit is constructed based on matrix summation; The input end of the fourth convolution unit is connected to the input end of the multi-scale topology module, and the output end of the fourth convolution unit is connected to the input end of the first normalization unit; the output end of the first normalization unit is connected to the input end of the first activation unit; the output end of the first activation unit is connected to the input end of the fifth convolution unit; the output end of the fifth convolution unit is connected to the input end of the second normalization unit; the input end of the first fusion unit is respectively connected to the output end of the second normalization unit and the input end of the multi-scale topology module, and the output end of the first fusion unit is connected to the input end of the second activation unit; the output end of the second activation unit is connected to the input end of the second overlay mapping module.

7. An infrared ship target recognition device, characterized in that: The device comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the infrared ship target recognition method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are called, they are used to execute the infrared ship target recognition method as described in any one of claims 1-5.

Images