A method for semantic positioning of concealed objects based on a convolutional neural network

Abstract

This invention discloses a method for semantic localization of hidden objects based on convolutional neural networks, comprising the following steps: constructing a model, wherein the model adopts a fully convolutional neural network structure; training the constructed model; the training objective is to optimize the specific parameters of each convolutional operation in the fully convolutional neural network so that the loss function of the input training image satisfies the conditions; inputting the image to be localized into the trained model to obtain the segmentation result; detecting and localizing hidden objects in the image to be localized; and performing semantic localization on the hidden objects in the image to be localized. This invention can be quickly deployed and applied after the model training is completed, without iterative calculations. The computational complexity of human body part segmentation and object localization is low, and real-time processing and display are possible, making it highly practical.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and specifically to a method for semantic localization of hidden objects based on convolutional neural networks. Background Technology

[0002] With the development of millimeter-wave imaging technology, millimeter-wave body scanning imaging equipment has become an important option for security checks in airports, train stations, and other densely populated areas. In particular, in June 2018, the Civil Aviation Administration of China (CAAC) issued the "Internal Control Standards for the Identification of Civil Aviation Millimeter-Wave Body Scanning Security Inspection Equipment" and the "Test Procedures for the Detection Capability of Prohibited Items by Civil Aviation Millimeter-Wave Body Scanning Security Inspection Equipment" (hereinafter referred to as the "Internal Control Standards" and the "Test Procedures"), officially including millimeter-wave body scanning equipment in the list of security inspection equipment for Chinese civil aviation. These documents clearly define the indicators and functions of the security inspection equipment, indicating a broad development prospect for millimeter-wave body scanning in China.

[0003] Millimeter-wave imaging equipment has the advantages of high imaging resolution and the ability to penetrate clothing, which makes it advantageous for detecting items hidden in clothing. However, this feature also poses the problem of exposing passenger privacy. The Internal Control Standard stipulates that millimeter-wave human body security scanners must have certain privacy protection functions. For example, when the obtained human body security images are displayed in the background, the faces of the personnel must be blurred to protect privacy. On the security personnel interface displayed to the outside world, the original imaging results are not displayed. Instead, a mannequin image with the location of the items is displayed. As for how to mark the location of the items, the Internal Control Standard does not make clear constraints. At present, there are some published works on human body image privacy protection. For example, Ye Jinjing et al. published a millimeter-wave human body image privacy protection method based on graphics method (Ye Jinjing, Zhou Jian, Sun Qianchen et al. Active millimeter-wave imaging privacy protection algorithm [J]. Journal of Infrared and Millimeter Waves, 2017, 36(4): 505-512.). However, there are currently no published papers or patents discussing methods for projecting the location information of items onto mannequin images. Based on research on existing products in the market, the mainstream method is to project the location of the item onto the doll image using markers or boxes. However, due to the differences in body shape among individuals, while the doll image uses a fixed model, this marking method inevitably has some ambiguity, causing inconvenience to security inspectors. Summary of the Invention

[0004] To address the aforementioned problems, this invention proposes a method for semantic localization of hidden objects based on convolutional neural networks, comprising the following steps:

[0005] A model is constructed, which adopts a fully convolutional neural network structure;

[0006] The constructed model is trained; the goal of training is to optimize the specific parameters of each convolution operation in the fully convolutional neural network so that the loss function of the input training image satisfies the conditions.

[0007] Input the image to be located into the trained model to obtain the segmentation result;

[0008] Detect and locate hidden objects in the image to be located;

[0009] Perform semantic localization on hidden objects in the image to be located;

[0010] Based on the semantic localization results of the hidden object, the area of ​​the hidden object on the human body is obtained.

[0011] Furthermore, the input to the fully convolutional neural network is a human security inspection image. Through convolution, pooling, deconvolution, and feature map copying and stitching operations, a feature map is obtained. The size of the feature map is... Where H and W are the height and width of the input image, respectively, and C represents the number of channels in the feature map, i.e., the region to which the pixel belongs.

[0012] Furthermore, the loss function of the input training image is the average of the loss functions of each pixel in the image; the specific formula for the loss function of each pixel is as follows:

[0013]

[0014] in, Let represent the loss function for the pixel at coordinates (i,j). The pixel category is manually labeled, and its value is either 0 or 1;

[0015] The pixel value of each pixel in the normalized feature map is represented by the following formula:

[0016]

[0017] in, Let i, j, and k be the pixel values ​​of the feature map, where i, j, and k are the coordinates of the image height, width, and number of channels, respectively.

[0018] Furthermore, the specific process of training the constructed model is as follows:

[0019] Update the training images and input them into the constructed model;

[0020] Enhance the image;

[0021] The forward propagation network is used to compute the output feature map;

[0022] Combined with the manually labeled results, calculate the loss function. If the loss function of the input training image is less than the set standard, then training ends; otherwise, use the backpropagation algorithm to update the parameters and optimize the gradient. Repeat the above steps until the loss function of the input training image is less than the set standard, then training ends.

[0023] Furthermore, the enhancement operations performed on the image specifically include image cropping, radial transformation, contrast adjustment, brightness adjustment, and random combinations thereof.

[0024] Furthermore, the semantic positioning of the item specifically refers to:

[0025] The specific formula for the semantic location I of the concealed object on the human body is:

[0026]

[0027] in, The probability of a certain hidden item being in region U is given by the following formula:

[0028]

[0029] in, The operation calculates the number of pixels in the set, where D is the set of pixels occupied by a hidden object, obtained from the coordinates of the object detection box. These coordinates are obtained during the object detection and localization process. Further, the region to which the hidden object belongs is subdivided based on its semantic location I on the human body. Specifically, the center position of the object detection box is compared with the center position of its region. If the center position of the object detection box is smaller than the center position of its region, the object is located on the left side of the human body; otherwise, it is located on the right side.

[0030] Furthermore, the trained model is evaluated using an alternating ratio. The model evaluation specifically involves: [denoteing...] Let be the number of pixels in the u-th region that are inferred to be the v-th region. The number of pixels that accurately classify the u-th region. This represents the number of pixels in region u that were incorrectly classified as other regions. The number of pixels in other regions that were incorrectly classified into the u-th region; for the u-th region, the intersection-union ratio (IoU) between the model classification result and the manually labeled IoU region. for:

[0031]

[0032] Compared with the prior art, the significant advantages of this invention are:

[0033] (1) The method proposed in this invention can reduce the ambiguity of item location and improve the efficiency of security inspectors' manual inspection.

[0034] (2) The method proposed in this invention is based on convolutional neural network technology and has a strong feature extraction capability. Under the condition that the model is fully trained, the accuracy of human body part recognition is very high. It can adapt to millimeter wave human body images of different genders and body types. Moreover, it can reduce the requirements for the accuracy of the passenger's posture when the passenger is being inspected.

[0035] (3) The present invention can be quickly deployed and applied after the model training is completed. It does not require iterative calculation. The computational complexity of human body part division and object positioning is low. It can be processed and displayed in real time and has high practical value. Attached Figure Description

[0036] Figure 1 The flowchart of the method in this embodiment.

[0037] Figure 2a Human body millimeter-wave images in the human body security inspection image classification diagram.

[0038] Figure 2b The results of manual annotation of human body parts in the human body security inspection image segmentation diagram.

[0039] Figure 3 A schematic diagram of the semantic segmentation network structure for human body parts according to an embodiment of the present invention.

[0040] Figure 4 A flowchart of the training steps for the semantic segmentation model of human body parts in this embodiment of the invention.

[0041] Figure 5a The embodiment of the present invention provides the human body segmentation result of example a based on a convolutional neural network model.

[0042] Figure 5b The embodiment of the present invention provides the human body segmentation result of example b based on a convolutional neural network model.

[0043] Figure 5c The embodiment of the present invention provides the human body segmentation result of example c based on a convolutional neural network model.

[0044] Figure 5d The embodiment of the present invention provides the human body segmentation result of example d based on a convolutional neural network model.

[0045] Figure 6a The item detection and positioning results for example a in this embodiment of the invention.

[0046] Figure 6b The item detection and positioning results for example b in this embodiment of the invention.

[0047] Figure 6c The item detection and location results for example c in this embodiment of the invention.

[0048] Figure 6d The item detection and location results for example d in this embodiment of the invention.

[0049] Figure 7a1 In this embodiment of the invention, the item in example a is mapped onto the image after model segmentation.

[0050] Figure 7a2 The final semantic localization result of the item in Example a of this invention on the doll image.

[0051] Figure 7b1 The mapping of the item in Example b in this embodiment of the invention onto the image after model segmentation.

[0052] Figure 7b2 The final semantic localization result of the item in Example b on the doll image in this embodiment of the invention.

[0053] Figure 7c1 The mapping of the item in example c on the image after model segmentation in this embodiment of the invention.

[0054] Figure 7c2 The final semantic localization result of the item in Example c of this embodiment of the invention on the doll image.

[0055] Figure 7d1 The mapping of the item in example d on the image after model segmentation in this embodiment of the invention.

[0056] Figure 7d2 The final semantic localization result of the item in Example d of this invention on the doll image. Detailed Implementation

[0057] The following detailed description, in conjunction with the accompanying drawings, illustrates a specific implementation of the semantic localization method for hidden objects based on convolutional neural networks according to the present invention. The specific implementation flow of the method is as follows: Figure 1 As shown.

[0058] The method is divided into two parts: "model building" and "model inference and object localization". The "model building" part outputs a human body region segmentation model for "model inference and object localization".

[0059] Model building consists of the following steps:

[0060] 1. Human body region division design

[0061] In collaborative body searches, individuals are required to stand with their arms and torso separated and their feet apart, ensuring that no parts of the body obstruct each other. This provides convenient conditions for semantic localization. To achieve accuracy and rationality in semantic localization, the semantic division of human images should be sufficiently detailed and possess clear physiological meaning. Taking the frontal view of the human body as an example, the body can be divided into several categories: head and neck, chest, abdomen, upper arm, lower arm, thigh, and calf. Except for the head and neck, the background is also considered as a separate category, resulting in a total of eight regions. The remaining areas can be further divided into left and right sides, as illustrated in Figure 2. Figure 2a Human body millimeter-wave images in the human body security inspection image classification diagram. Figure 2b The results of manual annotation of human body parts in the human body security inspection image segmentation diagram.

[0062] 2. Model Design

[0063] a) Semantic partitioning neural network architecture design

[0064] Convolutional neural network structures used for semantic segmentation of human body parts, such as... Figure 3 As shown, the network input is the original human security inspection image. Through a series of convolutional calculations, pooling calculations, deconvolutional calculations, and feature map copying and stitching operations, a feature map is finally obtained. The size of the feature map is [size missing]. H and W represent the height and width of the input image, respectively, while C represents the number of channels in the feature map, indicating which region each channel represents. In this embodiment, C=8, meaning the feature map is divided into 8 regions.

[0065] b) Loss function design

[0066] Each pixel value of the feature map is denoted as Here, i, j, k are the coordinates of the image's height, width, and number of channels, respectively. After operations such as convolution, pooling, and upsampling, the pixel values ​​in the feature map range from real numbers. To obtain the category information of each pixel, the softmax function is first used to normalize the data of each channel, resulting in a normalized feature map where the value of each pixel is... Its calculation formula is

[0067]

[0068] For each pixel location, the difference between the feature map inferred by the model and the class of the ground truth label is calculated using the cross-entropy function, and this difference is used as the loss function. Its calculation formula is

[0069]

[0070] in The pixel categories are manually labeled, and their values ​​are either 0 or 1. In this model, the different regions of the human body do not overlap; therefore, for a fixed position i, j, only one channel has a value of 1, while the rest are 0. The loss function for the entire input training image is the average of the loss functions of each pixel in the image.

[0071] 3. Model Training

[0072] The basic process of model training is as follows: Figure 4 As shown, the model training process is iterative. At the beginning of each iteration, the images used for training are input into the convolutional neural network to obtain the output results, which are then combined with the manually labeled results to obtain the loss function for this calculation. The goal of model training is to optimize the specific parameters of each convolutional operation in the network to minimize the loss function. The optimization direction of the model parameters (i.e., the gradient of each parameter) is obtained through the backpropagation algorithm.

[0073] To avoid the model only being applicable to a small number of specific images, a large number of human security inspection images need to be prepared in advance for training. Furthermore, before inputting the images into the model, operations such as cropping, affine transformation, and contrast / brightness adjustment are required to further improve the model's generalization ability.

[0074] The training dataset in this embodiment consists of 300 millimeter-wave images from four experimenters of different body types, with an image size of [missing information]. The input training image size is fixed. The output feature map is obtained by randomly cropping from the original image, and its size is [size missing]. To improve the efficiency of model training, the cropped results of 4 images are input for synchronous training each time.

[0075] 4. Model Evaluation

[0076] Due to factors such as system noise, background noise, and system channel consistency, millimeter-wave images contain some background clutter, resulting in blurred edges of the target image. Therefore, completely accurate segmentation of the human body region is impossible. To evaluate the accuracy of the segmentation, the intersection-over-union ratio (IoU) is used to measure the difference between the model's segmentation results and the manually labeled results. Let be the number of pixels in the u-th region that are inferred to be the v-th region. The number of pixels that accurately classify the u-th region. This represents the number of pixels in region u that were incorrectly classified as other regions. The number of pixels in other regions that were incorrectly classified into the u-th region; for the u-th region, the intersection-union ratio (IoU) between the model classification result and the manually labeled IoU region. for:

[0077]

[0078] Test results show that when the Intersection over Union (IoU) reaches 0.8 or higher, the segmentation effect is better and can be used as input for subsequent processing. In this embodiment, 75 millimeter-wave human body images of various types and manually annotated region information were used to test the trained model. The inference results of each region obtained from the test are shown in Table 1. The IoU index of each region segmentation exceeded 0.8, indicating good segmentation effect. Figure 5 shows a partial segmentation example of the actual model for human security inspection images of different body types.

[0079] area background neck chest belly upper arm forearm thigh calves IoU 0.957 0.881 0.877 0.901 0.833 0.818 0.900 0.838

[0080] Table 1 Test results of the human body region segmentation model

[0081] "Model Inference and Item Location" includes the following steps:

[0082] 1. Inference of human body region division

[0083] The model employs a fully convolutional neural network structure, therefore no modifications are needed to the millimeter-wave image of the human body; it can be directly input into the trained model to obtain the segmentation results. The pixel set of each region is denoted as . The specific illustrations for the four examples (example a, example b, example c, and example d) in this embodiment can be seen in Figure 5. Figure 5a The present invention provides a model based on a convolutional neural network for segmenting the human body in example a. Figure 5b The present invention provides a model based on a convolutional neural network for segmenting the human body in example b. Figure 5c The embodiment of the present invention provides the human body segmentation result of example c based on a convolutional neural network model. Figure 5d The embodiment of the present invention provides the human body segmentation result of example d based on a convolutional neural network model.

[0084] 2. Item detection and positioning

[0085] The specific method for detecting and locating objects in the original image is not required in this embodiment. However, the format of the object information provided by the detection method must be clearly defined before applying the method in this embodiment. A typical object detection and localization algorithm provides the position and size information of the object in the original image through a bounding box. The specific output format is the coordinates of the top-left corner of the bounding box. and the coordinates of the bottom right corner This embodiment uses a certain object detection algorithm with a certain output rectangular bounding box as an example. Figure 6 shows an example of object detection and localization on a millimeter-wave image. Among them, Figure 6a The item detection and location results for example a in this embodiment of the invention are as follows: Figure 6b The item detection and location results for example b in this embodiment of the invention are as follows: Figure 6cThe item detection and location results for example c in this embodiment of the invention are as follows: Figure 6d The item detection and location results for example d in this embodiment of the invention.

[0086] 3. Semantic Location of Items

[0087] The set of pixels occupied by an item can be obtained as D based on the coordinates of the item detection bounding box. The human body region where the item is located can be determined by the intersection of the item's pixel set and the region set. Let the probability of the item being in the u-th class region be denoted as... Then there is

[0088]

[0089] in The operation represents the count of pixels in the set. The semantic location I of an object on the human body can be determined using the following formula.

[0090]

[0091] Based on the semantic location calculation results of the items, the areas of the items can be marked on the doll image. The human body structure has obvious left-right symmetry; therefore, the center positions of the item frames can be compared. The object's location is determined by the size of the center position (median horizontal coordinate of pixels) of its region I. If the center position of the object's bounding box is smaller than the center position of region I, the object is located on the left half; otherwise, it is on the right half. Figure 7 shows an example of the semantic localization results of the object on the doll image.

[0092] exist Figure 7a1 In the given example, there are three suspicious items, located on the right chest and both sides of the thighs of the human body, respectively. The probability distribution of the locations of the three items, calculated using the aforementioned method, is shown in Table 2. The mapping results of the suspicious items on the doll are as follows: Figure 7a2 As shown, the mapping result correctly reflects the semantic location of the suspicious item.

[0093] Example a <![CDATA[P 背景 ]]> <![CDATA[P 头颈 ]]> <![CDATA[P 胸 ]]> <![CDATA[P 腹 ]]> <![CDATA[P 上臂 ]]> <![CDATA[P 小臂 ]]> <![CDATA[P 大腿 ]]> <![CDATA[P 小腿 ]]> Item 1 (Right Chest) 0.16 0.00 0.71 0.13 0.00 0.00 0.00 0.00 Item 2 (Right thigh) 0.15 0.00 0.00 0.00 0.00 0.00 0.85 0.00 Item 3 (left thigh) 0.09 0.00 0.00 0.00 0.00 0.00 0.91 0.00

[0094] Table 2 Probability distribution of item location (Example a)

[0095] exist Figure 7b1 In the given example, there are four suspicious items, located on the chest, abdomen, and both thighs of the human body. The probability distribution of the locations of these four items, calculated using the aforementioned method, is shown in Table 3. The mapping results of the suspicious items on the doll are as follows: Figure 7b2 As shown, the mapping result correctly reflects the semantic location of the suspicious item.

[0096] Example b <![CDATA[P 背景 ]]> <![CDATA[P 头颈 ]]> <![CDATA[P 胸 ]]> <![CDATA[P 腹 ]]> <![CDATA[P 上臂 ]]> <![CDATA[P 小臂 ]]> <![CDATA[P 大腿 ]]> <![CDATA[P 小腿 ]]> Item 1 (Chest) 0.07 0.00 0.92 0.01 0.00 0.00 0.00 0.00 Item 2 (abdomen) 0.00 0.00 0.00 0.68 0.00 0.00 0.32 0.00 Item 3 (Right Thigh) 0.11 0.00 0.00 0.00 0.00 0.00 0.89 0.00 Item 4 (left thigh) 0.00 0.00 0.00 0.00 0.00 0.00 1.00 0.00

[0097] Table 3. Probability distribution of item location (Example b)

[0098] exist Figure 7c1 In the given example, there are four suspicious items, located on the abdomen, left forearm, and both sides of the thigh. The probability distribution of the four item locations, calculated using the aforementioned method, is shown in Table 4. The mapping results of the suspicious items on the doll are as follows: Figure 7c2 As shown, the mapping result correctly reflects the semantic location of the suspicious item.

[0099] Example c <![CDATA[P 背景 ]]> <![CDATA[P 头颈 ]]> <![CDATA[P 胸 ]]> <![CDATA[P 腹 ]]> <![CDATA[P 上臂 ]]> <![CDATA[P 小臂 ]]> <![CDATA[P 大腿 ]]> <![CDATA[P 小腿 ]]> Item 1 (abdomen) 0.10 0.00 0.18 0.72 0.00 0.00 0.00 0.00 Item 2 (left forearm) 0.13 0.00 0.00 0.00 0.00 0.87 0.00 0.00 Item 3 (Right Thigh) 0.00 0.00 0.00 0.00 0.00 0.00 1.00 0.00 Item 4 (left thigh) 0.00 0.00 0.00 0.00 0.00 0.00 1.00 0.00

[0100] Table 4. Probability distribution of item location (Example c)

[0101] exist Figure 7d1 In the given example, there are two suspicious items, located on the thighs of the human body. The probability distribution of the locations of the four items, calculated using the aforementioned method, is shown in Table 4. The mapping results of the suspicious items on the doll are as follows: Figure 7d2 As shown, the mapping result correctly reflects the semantic location of the suspicious item.

[0102] Example d <![CDATA[P 背景 ]]> <![CDATA[P 头颈 ]]> <![CDATA[P 胸 ]]> <![CDATA[P 腹 ]]> <![CDATA[P 上臂 ]]> <![CDATA[P 小臂 ]]> <![CDATA[P 大腿 ]]> <![CDATA[P 小腿 ]]> Item 3 (Right Thigh) 0.00 0.00 0.00 0.00 0.00 0.00 1.00 0.00 Item 4 (left thigh) 0.00 0.00 0.00 0.00 0.00 0.00 1.00 0.00

[0103] Table 5. Probability distribution of item location (Example d)

[0104] Compared with the prior art, the significant advantages of this invention are:

[0105] (1) The method proposed in this invention can reduce the ambiguity of item location and improve the efficiency of security inspectors' manual inspection.

[0106] (2) The method proposed in this invention is based on convolutional neural network technology and has a strong feature extraction capability. Under the condition that the model is fully trained, the accuracy of human body part recognition is very high. It can adapt to millimeter wave human body images of different genders and body types. Moreover, it can reduce the requirements for the accuracy of the passenger's posture when the passenger is being inspected.

[0107] (3) The present invention can be quickly deployed and applied after the model training is completed. It does not require iterative calculation. The computational complexity of human body part division and object positioning is low. It can be processed and displayed in real time and has high practical value.

[0108] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for semantic localization of hidden objects based on convolutional neural networks, characterized in that, Includes the following steps: A model is constructed, which adopts a fully convolutional neural network structure; The constructed model is trained; the goal of training is to optimize the specific parameters of each convolution operation in the fully convolutional neural network so that the loss function of the input training images satisfies the conditions; the training dataset comes from millimeter-wave images of experimenters of different body sizes. Input the image to be located into the trained model to obtain the segmentation result; Detect and locate hidden objects in the image to be located; Perform semantic localization on hidden objects in the image to be located; Based on the semantic localization results of the concealed object, the area of ​​the concealed object on the human body is obtained; The fully convolutional neural network takes a human security inspection image as input. Through convolution, pooling, deconvolution, and feature map copying and stitching operations, a feature map is obtained. The size of the feature map is [size missing]. Where H and W are the height and width of the input image, respectively, and C represents the number of channels in the feature map, i.e. the region to which the pixel belongs; The semantic positioning of the item is specifically as follows: The specific formula for the semantic location I of the hidden object on the human body is: in, The probability of a certain hidden item being in region U is given by the following formula: in, , For the region The set of pixels, The operation means calculating the number of pixels in the set. D is the set of pixels occupied by a hidden item, obtained from the coordinates of the item detection box. The coordinates of the item detection box are obtained during the detection and positioning of the item.

2. The method for semantic localization of hidden objects based on convolutional neural networks according to claim 1, characterized in that, The loss function for the input training image is the average of the loss functions of each pixel in the image; the specific formula for the loss function of each pixel is as follows: in, Let represent the loss function for the pixel at coordinates (i,j). For manually labeled pixel categories, their values ​​are either 0 or 1; The pixel value of each pixel in the normalized feature map is represented by the following formula: in, Let i, j, and k be the pixel values ​​of the feature map, where i, j, and k are the coordinates of the image height, width, and number of channels, respectively.

3. The method for semantic localization of hidden objects based on convolutional neural networks according to claim 2, characterized in that, The specific process of training the constructed model is as follows: Update the training images and input them into the constructed model; Enhance the image; The forward propagation network is used to compute the output feature map; Combine the results of manual annotation, calculate the loss function, and if the loss function of the input training image is less than the set standard, then the training ends; Otherwise, the backpropagation algorithm is used to update the parameters and optimize the gradient. The above steps are repeated until the loss function of the input training image is less than the set standard, and then the training ends.

4. The method for semantic localization of hidden objects based on convolutional neural networks according to claim 3, characterized in that, The enhancement operations performed on images specifically include image cropping, affine transformation, contrast adjustment, brightness adjustment, and random combinations thereof.

5. The method for semantic localization of hidden objects based on convolutional neural networks according to claim 1, characterized in that, The region to which the hidden object belongs is subdivided according to the semantic location I of the hidden object on the human body. Specifically, the center position of the object detection box is compared with the center position of the region to which it belongs. If the center position of the object detection box is smaller than the center position of the region to which it belongs, the object is located on the left half of the human body; otherwise, it is located on the right half of the human body.

6. The method for semantic localization of hidden objects based on convolutional neural networks according to claim 3, characterized in that, The trained model is evaluated using intersection-union comparison, and the model evaluation specifically involves: [The text abruptly ends here, so the translation stops.] Let be the number of pixels in the u-th region that are inferred to be the v-th region. The number of pixels that accurately classify the u-th region; when hour, This represents the number of pixels in region u that were incorrectly classified as other regions. The number of pixels in other regions that were incorrectly classified into the u-th region; For the u-th class region, the intersection-union ratio (IoU) between the model partitioning result and the manually labeled result is... for: 。

Images