Pixel-level labeling method and device for infrared target

Abstract

The invention relates to the technical field of image processing, and provides a pixel-level labeling method and device for an infrared target. The method comprises the following steps: acquiring an infrared image; inputting the infrared image into an embedded platform running an infrared target segmentation algorithm for pixel-level segmentation to obtain an initial labeling target; if the initial labeling target does not meet the precision requirement of the operator, obtaining an adjusted infrared target segmentation algorithm in response to a segmentation threshold adjustment instruction of the operator for the infrared target segmentation algorithm, the segmentation threshold adjustment instruction being used for indicating that the segmentation threshold of the infrared target segmentation algorithm is adjusted according to the precision requirement; and segmenting the initial labeling target again according to the adjusted infrared target segmentation algorithm until a final labeling target meeting the precision requirement is obtained. According to the invention, the problem of difficulty in labeling the infrared target is solved.

Description

Technical Field

[0001] This application relates to the field of image processing technologies, and in particular, to a pixel-level annotation method and device for infrared targets. Background Art

[0002] Infrared target detection belongs to passive detection and has characteristics such as good concealment and strong anti-interference ability. Therefore, it has been widely used in the field of target detection. However, in practical applications, in order to better analyze the radiation characteristics of the target and the detection accuracy of the algorithm, it is necessary to accurately segment the target area from the image. However, due to the complex and changeable application scenarios of the infrared detection system and the wide distribution range of the scale and contrast of the target, this makes the annotation work of the target challenging. Summary of the Invention

[0003] In view of this, this application provides a pixel-level annotation method and device for infrared targets to solve the problem of difficult annotation work for infrared targets in the prior art.

[0004] In the first aspect of this application, a pixel-level annotation method for infrared targets is provided, including: preparing an infrared image; inputting the infrared image into an embedded platform running an infrared target segmentation algorithm for pixel-level segmentation to obtain an initial annotation target, where the infrared target segmentation algorithm is used for pixel-level segmentation of the infrared image; if the initial annotation target does not meet the accuracy requirements of the operator, then in response to the threshold adjustment instruction of the operator for the infrared target segmentation algorithm, obtain the infrared target segmentation algorithm after adjusting the threshold, and then use the infrared target segmentation algorithm after adjusting the threshold to perform fine segmentation on the initial annotation target to obtain a segmentation result, and iteratively update the segmentation threshold according to the segmentation result until a final annotation target that meets the accuracy requirements is obtained, and the threshold adjustment instruction is used to indicate that the segmentation threshold of the infrared target segmentation algorithm is adjusted according to the accuracy requirements; if there is a target area in the infrared image that is not detected by the embedded platform, then in response to the annotation instruction of the operator, annotate the target area that is not detected by the embedded platform to obtain all annotation targets corresponding to the infrared image, where the target area represents the area to be annotated in the infrared image.

[0005] In a second aspect of the present application, a pixel-level annotation device for infrared targets is provided, including: an acquisition module configured to acquire an infrared image, display the infrared image and the annotation target, and complete the interaction with the operator; a first annotation module configured to input the infrared image into an embedded platform running an infrared target segmentation algorithm for pixel-level segmentation to obtain an initial annotation target, where the infrared target segmentation algorithm is used to perform pixel-level segmentation on the infrared image; a threshold adjustment module configured to, if the initial annotation target does not meet the accuracy requirements of the operator, in response to the operator's threshold adjustment instruction for the infrared target segmentation algorithm, obtain the infrared target segmentation algorithm after adjusting the threshold, then use the infrared target segmentation algorithm after adjusting the threshold to perform fine segmentation on the initial annotation target to obtain a segmentation result, and iteratively update the threshold based on the segmentation result until a final annotation target that meets the accuracy requirements is obtained, where the segmentation threshold adjustment instruction is used to indicate that the segmentation threshold of the infrared target segmentation algorithm is adjusted according to the accuracy requirements; a second annotation module configured to, if there is a target area in the infrared image that has not been detected by the embedded platform, in response to the operator's annotation instruction, annotate the target area that has not been detected by the embedded platform to obtain all the annotation targets corresponding to the infrared image, where the target area represents the area to be annotated in the infrared image.

[0006] The above at least one technical solution adopted by the present application can achieve the following beneficial effects:

[0007] By acquiring an infrared image, inputting the infrared image into an embedded platform running an infrared target segmentation algorithm for pixel-level segmentation to obtain an initial annotation target, returning the obtained initial annotation target to the operator, and having the operator determine whether the initial annotation target meets the accuracy requirements. If it does not meet the accuracy requirements, then receive the threshold adjustment instruction issued by the operator based on the segmentation error of the initial annotation target, adjust the segmentation threshold of the infrared target segmentation algorithm according to the threshold adjustment instruction, and use the infrared target segmentation algorithm after adjusting the threshold to perform segmentation on the initial annotation target again, repeating this step until a final annotation target that meets the accuracy requirements is obtained. In this way, the infrared target segmentation algorithm can be used to perform rough segmentation on the infrared image to obtain an initial annotation target, and then the operator can judge the segmentation accuracy of the initial annotation target. If the segmentation accuracy does not meet the requirements of the operator, the segmentation threshold can be adjusted manually, and the initial annotation target can be re-segmented based on the adjusted infrared target segmentation algorithm to achieve accurate target segmentation. By manually annotating the target areas not detected by the embedded platform, the annotation work of all target areas can be completed. In this way, by combining deep learning and manual adjustment methods, accurate target annotation can be achieved, the difficulty of pixel-level annotation of infrared targets can be effectively reduced, and the annotation efficiency of the target and the accuracy of the annotation result can be greatly improved. Description of the Drawings

[0008] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the accompanying drawings required in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0009] Figure 1 It is a schematic flowchart of a pixel-level annotation method for infrared targets provided by an embodiment of the present application;

[0010] Figure 2 It is a schematic flowchart of an infrared target segmentation algorithm provided by an embodiment of the present application;

[0011] Figure 3 It is an interaction schematic diagram of a pixel-level annotation method for infrared targets provided by an embodiment of the present application;

[0012] Figure 4 It is a schematic structural diagram of a pixel-level annotation device for infrared targets provided by an embodiment of the present application. Detailed implementation manners

[0013] In the following description, for the purpose of illustration rather than limitation, specific details such as specific system structures and technologies are proposed to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0014] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims indicates at least one of the connected objects, and the character " / " generally represents an "or" relationship between the associated objects before and after.

[0015] In addition, it should be noted that the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0016] Infrared detection technology refers to the technology that can utilize infrared radiation to detect and identify objects. It can detect, track, image and analyze the target by detecting the infrared radiation emitted by the target object or reflecting infrared light. For example, in industrial inspection, infrared detection technology can be used to detect temperature anomalies of equipment and discover potential faults in advance. In order to better analyze the radiation characteristics of the target and the detection accuracy of the algorithm, it is necessary to segment the target from the infrared image. However, due to the complex and changeable application scenarios of the infrared detection system, the annotation work of infrared targets is full of challenges and it is difficult to carry out the annotation work efficiently. Therefore, in order to reduce the difficulty of pixel-level annotation, reduce the workload of annotators, and improve the annotation efficiency, it is particularly important to develop an algorithm and device that can preprocess infrared images and accurately segment the target area.

[0017] This application provides a pixel-level annotation method for infrared targets. The infrared image is preprocessed through an embedded platform, and then the operator adjusts the segmentation threshold to re-segment the initial annotation result output by the embedded platform to obtain the final annotation result, so as to efficiently extract targets of different scales in different scenarios from the infrared image, achieve precise separation of the target area and the background area, and at the same time reduce the resource requirements of the target annotation for the hardware platform.

[0018] Next, a pixel-level annotation method and device for infrared targets according to an embodiment of the present application will be described in detail with reference to the accompanying drawings.

[0019] Figure 1 It is a schematic flowchart of a pixel-level annotation method for infrared targets provided by an embodiment of the present application. As Figure 1 shown, the pixel-level annotation method for infrared targets includes:

[0020] S101, Prepare an infrared image.

[0021] Specifically, an infrared image can be annotated through a labeling software to obtain the final annotation target, that is, the labeling software can obtain the infrared image containing the target to be labeled from the pre-prepared dataset to be labeled, so as to prepare for further inputting the infrared image into the embedded platform for target area segmentation processing.

[0022] Among them, the pre-prepared dataset to be labeled can be an image dataset containing infrared targets collected and prepared by an operator, and this dataset to be labeled can be sourced from infrared sensors, thermal imaging devices, etc. The labeling software can be specialized infrared target labeling software, which can be used to mark target areas, perform refined adjustments on labeling results, etc. In this solution, the labeling software running on the computer has the functions of selecting images to be labeled, selecting target areas to be re-segmented, and adjusting the segmentation threshold for each target area, which is not limited here.

[0023] S102: Input the infrared image into an embedded platform running an infrared target segmentation algorithm for pixel-level segmentation to obtain an initial labeled target. The infrared target segmentation algorithm is used to perform pixel-level segmentation on the infrared image.

[0024] Specifically, inputting the infrared image into the embedded platform for pixel-level segmentation to obtain an initial labeled target can be understood as performing preliminary target detection and target segmentation on the infrared image through the infrared target segmentation algorithm, so as to quickly obtain the initially generated initial labeling result and reduce the workload of manual labeling.

[0025] Furthermore, the embedded platform can be an embedded Artificial Intelligence (AI) platform. This embedded AI platform can execute target detection or segmentation tasks based on the infrared target segmentation algorithm, automatically identify and extract the target areas in the infrared image, that is, obtain the initial labeled target and the final labeled target.

[0026] S103: If the initial labeled target does not meet the accuracy requirements of the operator, then in response to the threshold adjustment instruction of the operator for the infrared target segmentation algorithm, obtain the infrared target segmentation algorithm with the adjusted threshold, and then use the infrared target segmentation algorithm with the adjusted threshold to perform fine segmentation on the initial labeled target to obtain a segmentation result. Based on the segmentation result, iteratively update the segmentation threshold until the final labeled target that meets the accuracy requirements is obtained. The threshold adjustment instruction is used to indicate that the segmentation threshold of the infrared target segmentation algorithm is adjusted according to the accuracy requirements.

[0027] Specifically, the results of automatic algorithms may often have errors. After the embedded AI platform obtains the initial labeled target, it can return the initial labeled target to the computer for the operator to judge the accuracy. If the accuracy requirements do not meet the requirements of the operator's operation, the threshold of the infrared target segmentation algorithm can be corrected by manually adjusting the threshold, so that the boundary of the target labeling result is more accurate or more in line with the accuracy requirements of the operator.

[0028] It should be noted that manual adjustment of the segmentation threshold can be achieved by executing the segmentation threshold adjustment instruction issued by the operator. That is, after the operator analyzes the error of the initial labeled target, the segmentation threshold of the infrared target labeling algorithm is adjusted according to the error between the initial labeled target and the accuracy requirement, and target segmentation is performed according to different criteria. The error can be obtained by the operator comparing the initial labeled target with the accuracy requirement. As an example, setting a higher threshold can make the target area labeling more strict, while setting a lower threshold can make more areas be classified as target areas. The content of the segmentation threshold adjustment instruction can be determined according to the initial labeled target and the accuracy requirement to accurately divide the target area and the background, so as to obtain a more accurate labeling result.

[0029] Furthermore, the initial labeled target is segmented again according to the adjusted infrared target segmentation algorithm to obtain a segmentation result, and the segmentation threshold is iteratively updated based on the segmentation result. That is, the adjusted segmentation result can be judged. If the accuracy requirement is not yet met, the operator issues a new segmentation threshold adjustment instruction based on the actual situation, and repeats the threshold adjustment step until the final labeled target that meets the accuracy requirement is output.

[0030] S104. If there is a target area in the infrared image that has not been detected by the embedded platform, in response to the labeling instruction of the operator, the target area that has not been detected by the embedded platform is labeled to obtain all the labeled targets corresponding to the infrared image. The target area represents the area to be labeled in the infrared image.

[0031] Specifically, after the embedded platform processes the infrared image, there may still be undetected target areas. At this time, the operator can manually label the target areas that have not been detected by the embedded platform until all the target areas in the infrared image are labeled to obtain all the labeled targets.

[0032] It should be noted that in an infrared image, there may be multiple target areas that need to be labeled. Therefore, when this solution performs segmentation processing on infrared targets, each target area can be labeled separately. The task is considered completed only when the final labeling target of each target area is obtained. After all target areas are labeled, the labeling task of the next infrared image is entered. That is, multiple initial labeling targets may be obtained after processing by the embedded platform. The operator judges each initial labeling target in turn and issues a segmentation threshold adjustment instruction specifically for each initial labeling target that does not meet the accuracy requirements until all target areas meet the accuracy requirements. After the embedded platform preliminarily segments the infrared image, all initial labeling targets can be returned to the computer. The operator can arbitrarily select an initial labeling target for judgment and re-segmentation. After obtaining the corresponding final labeling target, another initial labeling target in the current infrared image returned by the embedded platform is processed. The above steps are repeated until all target areas are adjusted. Then, the operator manually labels the target areas not detected by the embedded platform. After all target areas in the infrared image are labeled, switch to the next frame of infrared image and repeat the steps of this solution. No further explanation is made here.

[0033] According to the technical solution provided by the embodiment of the present application, the infrared image is preprocessed by the infrared target segmentation algorithm running in the embedded platform, and then accurate target segmentation is achieved through manual adjustment or manual labeling, which can effectively reduce the difficulty of pixel-level labeling of infrared targets, greatly improve the labeling efficiency, and reduce the R & D cost.

[0034] In some embodiments, the infrared image is input into an embedded platform running an infrared target segmentation algorithm for pixel-level segmentation to obtain an initial labeling target, including: preprocessing the infrared image using an input layer to obtain a feature matrix, where the preprocessing includes histogram equalization and normalization; performing multi-scale feature extraction on the feature matrix using a nested network layer to obtain multi-scale features; performing weighted summation on the multi-scale features using an attention mechanism layer to obtain an attention feature map; processing the attention feature map using a dropout layer to obtain a heat map; and performing segmentation and clustering processing on the heat map through a domain clustering layer to obtain an initial labeling target. The infrared target segmentation algorithm includes an input layer, a nested network layer, an attention mechanism layer, a dropout layer, and a domain clustering layer.

[0035] Specifically, the embedded AI platform runs an infrared target segmentation algorithm to perform rough segmentation on the target area. In order to achieve high-precision segmentation of different-scale targets in different scenarios, an infrared target segmentation method based on a nested network and an attention mechanism can be used to segment the target area of the infrared image to obtain an initial labeling target.

[0036] Figure 2It is a schematic flowchart of an infrared target segmentation algorithm provided by an embodiment of the present application. The following will be combined with Figure 2 to illustrate this embodiment.

[0037] The infrared image can be preprocessed through the input layer (such as histogram equalization and normalization) to obtain a high-contrast feature matrix, namely x 1 , x 2 , x 3 , …, x n , where n can represent the number of feature channels or the number of dimensions. The feature matrix is input into the nested network layer, and the feature map splicing method is used to progressively fuse features of different scales, extract multi-scale features containing rich semantic information, then further fuse the multi-scale features by adding an attention mechanism, and optimize the network model (i.e., the infrared target segmentation algorithm) in combination with the dropout algorithm to accelerate the convergence speed of the network model iteration, improve the recognition accuracy and generalization ability of the network model, and finally extract the infrared target area through the adaptive threshold segmentation algorithm and the domain clustering algorithm.

[0038] It should be noted that during the training process, the dropout layer can be used to randomly adjust the structure of the network model. This layer can be used to accelerate the convergence speed of the iteration during the training model process, that is, during the training process, hidden neurons in the network are obtained in a random probability manner, and their activation functions are set to 0 to stop their work. Through dropout, overfitting of the model to the training set can be effectively avoided, and the generalization ability of the model can be improved.

[0039] In some other embodiments, the nested network layer includes a series of nodes for extracting multi-scale features. The nested network layer feature matrix is used for multi-scale feature extraction to obtain multi-scale features, including: processing the input features through the nodes to obtain the output features of each node. Different nodes correspond to different input features, and the input features represent the feature matrix or the output features from adjacent nodes; multi-scale features are obtained by layer-by-layer passing of the output features of each node.

[0040] Specifically, the nested network layer (NN network layer) can process the feature matrix obtained from the preprocessed infrared image to obtain a feature map with strong discriminative ability (i.e., multi-scale features). This nested network layer can be understood as a multi-scale feature extraction module for infrared images, that is, the input of each node is aggregated by adopting a feature map splicing mechanism for the output features of adjacent nodes, so that features of different depths of the network can fully retain shallow detail information and deep semantic information, and multi-scale features are obtained to improve the representation ability of multi-scale targets.

[0041] Multiple nodes included in a nested network layer can each be labeled as A, and each node can perform a convolution operation or a feature extraction operation. The input of the first layer of nodes can be the feature matrix output by the previous layer. Feature extraction is performed on the feature matrix to obtain the output features of the first layer of nodes. For each subsequent node, the output features of adjacent nodes can be aggregated using a feature map concatenation mechanism as the input to this node, so that the output of this node contains information of different scales.

[0042] In some other embodiments, an attention mechanism layer is used to perform weighted summation on multi-scale features to obtain an attention feature map, including: processing the multi-scale features through a convolution layer with a preset shape to obtain the weights corresponding to each layer of features of the multi-scale features; using the weights corresponding to each layer of features to perform weighted summation on the multi-scale features to obtain the attention feature map.

[0043] Specifically, after obtaining the multi-scale features, the multi-scale features can be processed through an attention mechanism (i.e., the attention mechanism) to highlight key features, that is, the multi-scale features (i.e., h1, h2,..., hn) are processed through a convolution layer with a preset shape (such as 1, 1, 1), where h can represent the input of the attention mechanism layer. Then, the outputs (O(1)...O(t)...O(n)) of the convolution layer are scaled to the same scale for concatenation and normalized along the channel direction to obtain the weights corresponding to each layer of features in the multi-scale features, where O can represent the output result of the convolution layer, and t can represent the t-th layer of features. Finally, the obtained weights are used to perform weighted summation on the multi-scale features to obtain a feature map that can effectively enhance the saliency of small targets, that is, the attention feature map v.

[0044] In some other embodiments, the dropout layer includes a convolution layer. The dropout layer is used to process the attention feature map to obtain a heat map, including: processing the attention feature map through the convolution layer to obtain an intermediate feature map; performing weighted summation on the intermediate feature map to obtain the heat map, and the heat map is used to reflect the difference between the target area and the background.

[0045] Specifically, the attention feature map can be processed through the convolution layer in the dropout layer to obtain an intermediate feature map, and weighted summation is performed on the intermediate feature map to obtain a heat map that reflects the difference between the target area and the background.

[0046] It should be understood that during the model training process, the dropout layer randomly selects nodes that need to stop working with a certain probability, sets their activation values to 0, and keeps the activation values of other nodes unchanged. The nodes that stop working no longer affect subsequent processing and do not affect the update of model parameters during backpropagation, thereby achieving random adjustment of the network structure, accelerating the convergence speed, and improving the generalization ability of the model. By processing the attention feature map through the dropout layer and randomly adjusting the structure of the network model, the risk of overfitting of the model can be reduced.

[0047] It should also be understood that during the model training process, the dropout layer works in a random manner to accelerate the convergence speed of model iteration and improve the generalization ability of the model. During the inference process after the model training is completed, the dropout layer works in a deterministic manner to increase the certainty of the model inference result.

[0048] In some other embodiments, the heat map is segmented and clustered through the domain clustering layer to obtain the initial labeled target, including: using the adaptive threshold segmentation algorithm to process the heat map to obtain the target pixel points. The adaptive threshold segmentation algorithm is used to distinguish the target pixel points in the infrared image from the background; through the domain clustering algorithm, the target pixel points belonging to the same target area are clustered to obtain the initial labeled target. The domain clustering algorithm is used to cluster the target pixel points in the same target area.

[0049] Specifically, after obtaining the heat map, the heat map can be processed through the domain clustering layer, that is, the pixels in the heat map are divided into the target area and the background area through the adaptive threshold segmentation algorithm, and the pixel points belonging to the same target are clustered together through the domain clustering algorithm to obtain the initial labeled target. The adaptive threshold segmentation algorithm can dynamically adjust the threshold according to the input features, thereby effectively distinguishing the target pixel points from the background pixel points. The domain clustering algorithm can group the pixel points according to the similarity between pixels or other clustering methods, cluster the pixel points belonging to the same target area, obtain the initial labeled target, and number the initial labeled target, which is not limited here.

[0050] After that, the output layer can output the processing result of the dropout layer and the processing result of the domain clustering layer, that is, the initial labeled target Y.

[0051] According to the technical solution provided by the above embodiment, different-scale features are extracted through a nested network layer, improving the representation ability for targets of different scales. The attention mechanism is used to strengthen important features, which can effectively enhance the feature map of target saliency. The dropout layer is used to improve the generalization ability and enhance the adaptability to different scenarios. Finally, the domain clustering layer is used to integrate the target regions and output the initial labeled target. Selecting a high-precision target segmentation algorithm in this way can greatly reduce the labeling amount of operators and significantly shorten the labeling cycle.

[0052] Figure 3 is an interaction schematic diagram of a pixel-level labeling method for infrared targets provided by an embodiment of the present application. As Figure 3 shown, the interaction process of the pixel-level labeling method for infrared targets includes:

[0053] An operator can operate the labeling software running on the computer, select an infrared image for processing, input the infrared image into the embedded AI platform. The embedded AI platform performs target labeling preprocessing on the infrared image to obtain the initial labeled target, and returns the obtained labeling result, that is, the initial labeled target, to the operating software for the operator to judge whether it meets the accuracy requirements. In the case of not meeting the accuracy requirements, the operator operates the software to send a segmentation threshold adjustment instruction to the embedded AI platform to re-segment the initial labeled target until the final labeled target that meets the accuracy requirements is obtained. Then the operator identifies and labels the target regions not detected by the embedded platform. After completing the labeling of each target region in the infrared image, the labeling task of the next frame of infrared image can be sequentially executed until all infrared images are processed. The segmentation threshold adjustment instruction can be determined by the operator according to the initial labeled target and the accuracy requirements, and is an operation instruction for increasing or decreasing the segmentation threshold of the infrared target segmentation algorithm, which will not be elaborated here.

[0054] In this way, the infrared image can be preprocessed by using a deep learning-based method, and then accurate target segmentation can be achieved through manual adjustment, which can effectively reduce the difficulty of pixel-level labeling of infrared targets, greatly improve the labeling efficiency, reduce the R & D cost, and obtain accurate labeling results.

[0055] All the above optional technical solutions can be combined arbitrarily to form optional embodiments of the present application, which will not be elaborated one by one here.

[0056] The following is an embodiment of the device of the present application, which can be used to execute the method embodiment of the present application. For details not disclosed in the device embodiment of the present application, please refer to the method embodiment of the present application.

[0057] Figure 4 is a schematic diagram of a pixel-level labeling device for infrared targets provided by an embodiment of the present application. As Figure 4As shown in the figure, the pixel-level annotation device for the infrared target includes:

[0058] An acquisition module 401, configured to acquire an infrared image, display the infrared image and the annotation target, and complete the interaction with the operator;

[0059] A first annotation module 402, configured to input the infrared image into an embedded platform running an infrared target segmentation algorithm for pixel-level segmentation to obtain an initial annotation target, and the infrared target segmentation algorithm is used for pixel-level segmentation of the infrared image;

[0060] A threshold adjustment module 403, configured to, if the initial annotation target does not meet the accuracy requirements of the operator, in response to the threshold adjustment instruction of the operator for the infrared target segmentation algorithm, obtain the infrared target segmentation algorithm after adjusting the threshold, then use the infrared target segmentation algorithm after adjusting the threshold to perform fine segmentation on the initial annotation target to obtain a segmentation result, and iteratively update the threshold according to the segmentation result until a final annotation target that meets the accuracy requirements is obtained. The segmentation threshold adjustment instruction is used to indicate that the segmentation threshold of the infrared target segmentation algorithm is adjusted according to the accuracy requirements;

[0061] A second annotation module 404, configured to, if there is a target area in the infrared image that is not detected by the embedded platform, in response to the annotation instruction of the operator, annotate the target area that is not detected by the embedded platform to obtain all annotation targets corresponding to the infrared image, and the target area represents the area to be annotated in the infrared image.

[0062] In some embodiments, the first annotation module 402 is specifically configured to preprocess the infrared image using an input layer to obtain a feature matrix, and the preprocessing includes histogram equalization and normalization; perform multi-scale feature extraction on the feature matrix using a nested network layer to obtain multi-scale features; perform weighted summation on the multi-scale features using an attention mechanism layer to obtain an attention feature map; process the attention feature map using a dropout layer to obtain a heat map; perform segmentation and clustering processing on the heat map through a domain clustering layer to obtain an initial annotation target. The infrared target segmentation algorithm includes an input layer, a nested network layer, an attention mechanism layer, a dropout layer, and a domain clustering layer.

[0063] In some embodiments, the first annotation module 402 is specifically configured to process the input features through nodes to obtain the output features of each node, and different nodes correspond to different input features, and the input features represent the feature matrix or the output features from adjacent nodes; obtain multi-scale features by successively transmitting the output features of each node.

[0064] In some embodiments, the first annotation module 402 is specifically configured to process the multi-scale features through a convolutional layer with a preset shape to obtain the weights corresponding to each layer of features of the multi-scale features; and perform weighted summation on the multi-scale features by using the weights corresponding to each layer of features to obtain an attention feature map.

[0065] In some embodiments, the first annotation module 402 is specifically configured to process the attention feature map through a convolutional layer to obtain an intermediate feature map; and perform weighted summation on the intermediate feature map to obtain a heat map, where the heat map is used to reflect the difference between the target area and the background.

[0066] In some embodiments, the first annotation module 402 is specifically configured to use an adaptive threshold segmentation algorithm to process the heat map to obtain target pixel points, where the adaptive threshold segmentation algorithm is used to distinguish the target pixel points in the infrared image from the background; and perform clustering on the target pixel points belonging to the same target area through a domain clustering algorithm to obtain an initial annotation target, where the domain clustering algorithm is used to perform clustering processing on the target pixel points in the same target area.

[0067] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution is prior or posterior, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0068] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A pixel-level labeling method for infrared targets, characterized in that: include: Prepare infrared images; Inputting the infrared image into an embedded platform running an infrared target segmentation algorithm for pixel-level segmentation to obtain an initial labeled target, wherein the infrared target segmentation algorithm is used to perform pixel-level segmentation on the infrared image; If the initial labeled target does not meet the operator's accuracy requirement, then in response to the operator's threshold adjustment instruction for the infrared target segmentation algorithm, the infrared target segmentation algorithm after the threshold is adjusted is obtained, and then the infrared target segmentation algorithm after the threshold is adjusted is used to perform fine segmentation on the initial labeled target to obtain a segmentation result, and according to the segmentation result, the segmentation threshold is iteratively updated until a final labeled target that meets the accuracy requirement is obtained, and the threshold adjustment instruction is used to instruct the segmentation threshold of the infrared target segmentation algorithm to be adjusted according to the accuracy requirement; If there is a target area in the infrared image that is not detected by the embedded platform, the target area that is not detected by the embedded platform is marked in response to the operator's marking instruction to obtain all marked targets corresponding to the infrared image, and the target area represents the area to be marked in the infrared image.

2. The method according to claim 1, characterized in that The step of inputting the infrared image to an embedded platform running an infrared target segmentation algorithm for pixel-level segmentation to obtain an initial labeled target includes: Preprocessing the infrared image using an input layer to obtain a feature matrix, wherein the preprocessing includes histogram equalization and normalization; Performing multi-scale feature extraction on the feature matrix using a nested network layer to obtain multi-scale features; Using the attention mechanism layer to perform weighted summation on the multi-scale features to obtain an attention feature map; The attention feature map is processed using a dropout layer to obtain a heat map; The heat map is segmented and clustered through a domain clustering layer to obtain the initial labeled target. The infrared target segmentation algorithm includes the input layer, the nested network layer, the attention mechanism layer, the dropout layer and the domain clustering layer.

3. The method according to claim 2, characterized in that The nested network layer includes a series of nodes for extracting multi-scale features, and the multi-scale feature extraction of the feature matrix using the nested network layer to obtain the multi-scale features includes: Processing the input features through the nodes to obtain output features of each node, where different nodes correspond to different input features, and the input features represent the feature matrix or output features from adjacent nodes; The multi-scale features are obtained by transferring the output features of each node layer by layer.

4. The method according to claim 2, characterized in that: The method of using the attention mechanism layer to perform weighted summation on the multi-scale features to obtain an attention feature map includes: Processing the multi-scale features through a convolutional layer of a preset shape to obtain a weight corresponding to each layer of features of the multi-scale features; The multi-scale features are weightedly summed using the weights corresponding to the features of each layer to obtain the attention feature map.

5. The method according to claim 2, characterized in that: The dropout layer includes a convolution layer, and the attention feature map is processed by the dropout layer to obtain a heat map, including: Processing the attention feature map using the convolutional layer to obtain an intermediate feature map; A heat map is obtained by weighted summing of the intermediate feature maps, and the heat map is used to reflect the difference between the target area and the background.

6. The method according to claim 2, characterized in that The segmentation and clustering process of the heat map through the domain clustering layer to obtain the initial annotation target includes: The thermal image is processed using an adaptive threshold segmentation algorithm to obtain target pixels, wherein the adaptive threshold segmentation algorithm is used to distinguish the target pixels from the background in the infrared image; The target pixels belonging to the same target area are clustered by a domain clustering algorithm to obtain the initial labeled target. The domain clustering algorithm is used to cluster the target pixels in the same target area.

7. A pixel-level labeling device for infrared targets, characterized in that: include: An interaction module is configured to acquire an infrared image, display the infrared image and annotated targets, and complete interaction with an operator; A first annotation module is configured to input the infrared image into an embedded platform running an infrared target segmentation algorithm for pixel-level segmentation to obtain an initial annotation target, wherein the infrared target segmentation algorithm is used to perform pixel-level segmentation on the infrared image; The threshold adjustment module is configured to, if the initial labeled target does not meet the operator's accuracy requirement, respond to the operator's threshold adjustment instruction for the infrared target segmentation algorithm, obtain the infrared target segmentation algorithm after the threshold is adjusted, then use the infrared target segmentation algorithm after the threshold is adjusted to perform fine segmentation on the initial labeled target to obtain a segmentation result, and iteratively update the threshold according to the segmentation result until a final labeled target that meets the accuracy requirement is obtained, wherein the segmentation threshold adjustment instruction is used to instruct the segmentation threshold of the infrared target segmentation algorithm to be adjusted according to the accuracy requirement; The second labeling module is configured to label the target area not detected by the embedded platform in the infrared image in response to the labeling instruction of the operator, so as to obtain all the labeled targets corresponding to the infrared image, and the target area represents the area to be labeled in the infrared image.

Images