Multi-modal target detection method based on graph network information interaction

Abstract

The invention provides a multi-modal target detection method based on graph network information interaction, and the method comprises the steps: collecting visible light and infrared images through a camera and an infrared thermal imaging head, and extracting multi-scale features through a feature extraction module, and the multi-modal feature interaction module performs inter-modal and intra-modal information interaction on the bimodal multi-scale features and enhances feature representation, the gated fusion module fuses the interacted features to generate multi-modal fusion features, and the multi-modal fusion feature detection head outputs a prediction result. According to the method, a multi-modal feature interaction module based on a graph network is designed, complementary information and long-range spatial dependence among multi-modal data are captured through graph reasoning, then the performance of salient target detection is improved, and the whole process is divided into two stages, namely inter-modal graph reasoning and intra-modal graph reasoning; the two stages act together, so that information between modals can be fully fused, and space structures in the modals are strengthened.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of multi-modal target detection, and particularly relates to a multi-modal target detection method based on graph network information interaction. BACKGROUND

[0002] Target detection is a basic topic in computer vision, aiming to locate and identify various targets in images or videos. It has a wide range of applications in autonomous driving, monitoring and remote sensing. However, visible object detection has encountered great challenges under adverse conditions such as rain, fog, cloud and poor lighting, mainly due to the inherent limitations of RGB sensors. Therefore, alternative vision sensors, especially infrared cameras, are increasingly used to supplement RGB cameras to overcome these problems. At present, common multi-modal detectors can be roughly divided into three categories according to the fusion stage: early fusion, mid-fusion and late fusion.

[0003] Early fusion, also known as data layer fusion, directly splices or combines the original data from different modalities at the input layer. This method is simple and direct, but in practical applications it needs to handle data alignment, scale differences and noise of each modality; Mid-fusion, also known as feature layer fusion, extracts features of data from different modalities and fuses them at the feature layer. This method can better capture complementary information between modalities and improve information processing, but it has high requirements for model design and computational overhead, and may mask the scarce modality signal; Late fusion, also known as decision-level fusion, weights and fuses the results of each modality according to the output score, which can replace single modality models and is most robust to missing or noisy modalities, but the improvement is usually limited and only at the result layer can it utilize fine-grained complementary features. SUMMARY

[0004] In view of the above situation, the main purpose of the present application is to provide a multi-modal target detection method based on graph network information interaction to solve the above technical problems.

[0005] The present application provides a multi-modal target detection method based on graph network information interaction, which comprises the following steps: Step 1, a feature extraction module is constructed based on a pre-trained Swin-Transformer network sharing two parameters, a multi-modal feature interaction module is constructed based on a graph network, a gated fusion module is constructed based on a gated fusion mechanism, and a multi-modal target detection model is formed by the feature extraction module, the multi-modal feature interaction module, the gated fusion module and a detection head; Step 2, the current scene is collected by using a camera and an infrared thermal imaging head respectively to obtain a visible light image and an infrared image respectively; multi-scale features of the visible light image and the infrared image are extracted by using a feature extraction module to obtain a visible light multi-scale feature map and an infrared multi-scale feature map; Step 3, the multi-modal feature interaction module is used to establish inter-modal and intra-modal information interaction of the visible light multi-scale feature map and the infrared multi-scale feature map to obtain visible light modal graph network information interaction features and infrared modal graph network information interaction features respectively; Step 4, the visible light modal graph network information interaction features and the infrared modal graph network information interaction features are input into a gate fusion module for fusion to obtain multi-modal fusion features; Step 5, the multi-modal fusion features are input into a detection head to obtain a predicted bounding box, a classification loss function, a confidence loss function and a positioning loss function are constructed based on the predicted bounding box, the multi-modal target detection model is trained by using the classification loss function, the confidence loss function and the positioning loss function, a trained multi-modal target detection model is obtained, and the final class information and the bounding box information are determined by using the trained multi-modal target detection model.

[0006] Compared with the prior art, the present application has the following advantages: The present application designs a multi-modal feature interaction module based on a graph network and a gate fusion module; the multi-modal feature interaction module based on the graph network mainly captures complementary information and long-range spatial dependencies between multi-modal data through graph reasoning, thereby improving the performance of salient object detection; the whole process is divided into two stages: inter-modal graph reasoning and intra-modal graph reasoning; the two stages work together to enable the information between the modes to be fully fused and the spatial structure within the modes to be strengthened; the gate fusion module mainly weights and fuses the features from different modes through a gating attention mechanism, and uses 3D convolution to capture the relationship between the spatial, channel and modal dimensions; the specific process is to first fuse the feature maps of each mode, then selectively enhance the information of useful modes and suppress the influence of low-quality modes through the gating mechanism.

[0007] Additional aspects and advantages of the present application will be given in part in the following description, part will become apparent from the following description, or will be understood by those skilled in the art through embodiments of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0008] Figure 1 A step flowchart of a multi-modal target detection method based on graph network information interaction is provided for the present application; Figure 2A general framework diagram of a multi-modal target detector based on graph network information interaction is provided for an embodiment of the present application. Figure 3 A principle diagram of a multi-modal feature interaction module is provided for an embodiment of the present application. Figure 4 A principle diagram of a gating fusion module is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0009] Embodiments of the present application are described in detail below with reference to examples shown in the accompanying drawings, in which the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application and cannot be understood as a limitation of the present application.

[0010] These and other aspects of embodiments of the present application will become apparent from the following description and the accompanying drawings. In these descriptions and drawings, some specific implementations of embodiments of the present application are specifically disclosed to represent some ways in which the principles of embodiments of the present application can be implemented, but it should be understood that the scope of embodiments of the present application is not limited thereto.

[0011] Referring to Figure 1 The present embodiment provides a multi-modal target detection method based on graph network information interaction, which comprises the following steps: Step 1, a feature extraction module is constructed based on a pre-trained Swin-Transformer network sharing two parameters, a multi-modal feature interaction module is constructed based on a graph network, a gating fusion module is constructed based on a gating fusion mechanism, and a multi-modal target detection model is formed by the feature extraction module, the multi-modal feature interaction module, and the gating fusion module and a detection head.

[0012] Step 2, a camera and an infrared thermal imaging head are used to respectively capture the current scene to obtain visible light images and infrared images; a feature extraction module is used to extract multi-scale features of the visible light images and the infrared images to obtain visible light multi-scale feature maps and infrared multi-scale feature maps.

[0013] Referring to Figure 2 In step 2, a feature extraction module is used to extract multi-scale features of the visible light images and the infrared images to obtain visible light multi-scale feature maps and infrared multi-scale feature maps, which specifically comprises the following sub-steps: Based on the pre-trained Swin-Transformer network sharing two parameters, one of the pre-trained Swin-Transformer networks is used to extract multi-scale features of the visible light images to obtain visible light multi-scale feature maps; A pre-trained Swin-Transformer network is used to extract multi-scale features of the infrared image to obtain an infrared multi-scale feature map.

[0014] In step 3, a multi-modal feature interaction module is used to establish inter-modal and intra-modal information interaction between the visible light multi-scale feature map and the infrared multi-scale feature map to obtain visible light modal graph network information interaction features and infrared modal graph network information interaction features, respectively, and the specific steps include the following sub-steps:

[0015] Please refer to Figure 3 In step 3, a multi-modal feature interaction module is used to establish inter-modal and intra-modal information interaction between the visible light multi-scale feature map and the infrared multi-scale feature map to obtain visible light modal graph network information interaction features and infrared modal graph network information interaction features, respectively, and the specific steps include the following sub-steps: The visible light multi-scale feature map and the infrared multi-scale feature map are divided into two groups of sub-feature maps according to the channel, and global average pooling processing is performed on each sub-feature map to obtain node features; All node features are merged to obtain total node embedding; the similarity between modes is calculated based on the total node embedding to construct an adjacency matrix, and the following relationship exists in the corresponding process: ; Among them, Adjacency matrix is represented by A, Normalization is represented by N, Total node embedding is represented by Z, Transposition is represented by T; Graph reasoning is performed based on the adjacency matrix and the total node embedding to capture the mutual relationship between nodes to obtain new node embedding after graph reasoning, and the following relationship exists in the corresponding process: ; Among them, New node embedding after graph reasoning is represented by Z', Nonlinear activation function processing is represented by σ, A learnable parameter is represented by W; The new node embedding after graph reasoning is projected back to the channel dimension to obtain channel weight ; Among them, Visible light modal channel weight is represented by Wvis, Infrared modal channel weight is represented by Wir; The visible light modal channel weight is multiplied by the visible light multi-scale feature map, and multi-modal channel complementary information is introduced to obtain a visible light feature map with multi-modal information interaction, and the following relationship exists in the corresponding process: ; wherein, represents a visible light feature map with multimodal information interaction, represents a visible light multiscale feature map, represents element-wise addition, represents element-wise multiplication, represents processing through a fully connected layer with a Sigmoid activation function; The infrared modal channel weight is multiplied by the infrared multiscale feature map, and the multimodal channel complementary information is introduced to obtain an infrared feature map with multimodal information interaction. There is a relationship as follows in the corresponding process: ; wherein, represents an infrared feature map with multimodal information interaction, represents an infrared multiscale feature map; The visible light feature map with multimodal information interaction is converted into a spatial dimension node embedding through graph projection, and the relevance of the context semantic information of different spatial regions is modeled by using graph reasoning to obtain a visible light modal graph network information interaction feature. There is a relationship as follows in the corresponding process: ; wherein, represents a visible light modal graph network information interaction feature, represents a graph re-projection function, represents a graph reasoning operation, represents a graph projection function; The visible light feature map with multimodal information interaction is converted into a spatial dimension node embedding through graph projection, and the relevance of the context semantic information of different spatial regions is modeled by using graph reasoning to obtain an infrared modal graph network information interaction feature. There is a relationship as follows in the corresponding process: ; wherein, represents an infrared modal graph network information interaction feature.

[0016] Step 4, input the visible light modal graph network information interaction feature and the infrared modal graph network information interaction feature into the gate fusion module for fusion to obtain a multimodal fusion feature.

[0017] Please refer to Figure 4 , in step 4, the visible light modal graph network information interaction feature and the infrared modal graph network information interaction feature are input into the gate fusion module for fusion to obtain a multimodal fusion feature, which specifically includes the following sub-steps: The visible light modality graph network information interaction feature and the infrared modality graph network information interaction feature are stacked along the channel dimension to obtain stacked multi-modal features, and there is a relationship as follows in the corresponding process: ; wherein, represents the stacked multi-modal features, represents stacking along the channel dimension; The stacked multi-modal features are sequentially processed by a 3D convolution layer and a Sigmoid activation function to obtain feature weights; The feature weights are multiplied by the stacked multi-modal features, and the stacked multi-modal features are connected in residual to obtain features filtered by a gating mechanism, and there is a relationship as follows in the corresponding process: ; wherein, represents the features filtered by the gating mechanism, represents processing by the Sigmoid activation function, represents processing by the 3D convolution layer; The features filtered by the gating mechanism are sequentially processed by feature reshaping and a 3D convolution layer to obtain multi-modal fusion features, and there is a relationship as follows in the corresponding process: ; wherein, represents the multi-modal fusion features, represents processing by the feature reshaping.

[0018] Step 5, input the multi-modal fusion features into the detection head to obtain a predicted bounding box, based on the predicted bounding box, a classification loss function, a confidence loss function and a positioning loss function are constructed respectively, the multi-modal target detection model is trained using the classification loss function, the confidence loss function and the positioning loss function, and a trained multi-modal target detection model is obtained; the final class information and the bounding box information are determined by using the trained multi-modal target detection model.

[0019] In step 5, based on the predicted bounding box, a classification loss function, a confidence loss function and a positioning loss function are constructed respectively, which specifically includes the following sub-steps: The classification loss function is constructed based on the predicted bounding box, and there is a relationship as follows in the corresponding process: ; wherein, represents the classification loss, represents a weight factor, represents a label value, This represents the model's predicted probability of the true class. This indicates adjustable focus parameters. Indicates taking the logarithm; The confidence loss function is constructed based on the predicted bounding boxes, and the following relationship exists in the corresponding process: ; in, Indicates confidence loss. Indicates the number of samples. Indicates the index of the target to be detected. Indicates the first The true label of each sample Indicates the first The prediction confidence level of each sample; The localization loss function is constructed based on the predicted bounding box, and the following relationship exists in the corresponding process: ; in, This indicates the consistency used to measure the ratio of bounding box heights. This indicates taking the arctangent value. This represents the width of the actual bounding box. This represents the height of the actual bounding box. This indicates the width of the predicted bounding box. Indicates the height of the predicted bounding box. Represents the weighting function. This represents the intersection-union ratio (IoU) between the predicted bounding box and the ground truth bounding box. This represents the Euclidean distance between the predicted point and the actual point. Indicates the center point of the predicted bounding box. This represents the center point of the actual bounding box. This represents the diagonal length of the outermost rectangle that minimizes the difference between the predicted and actual bounding boxes. Indicates positioning loss; The multimodal object detection model is trained using classification loss function, confidence loss function and localization loss function to obtain the trained multimodal object detection model. The trained multimodal object detection model is then used to obtain the final category information and bounding box information.

[0020] Furthermore, the total training loss function is: ; in, This represents the total training loss.

[0021] Specifically, during the training of the multimodal object detection model using classification loss function, confidence loss function, and localization loss function, the learning rate is adjusted according to the training stage. This adjustment includes two stages: the first stage is the warm-up stage, where the current epoch number is less than the preset value. The values ​​correspond to the following relationship: ; in, Indicating the first stage The learning rate for each training round. This indicates the current training round number in the first phase. This indicates the learning rate at the start of warm-up. This represents the initial learning rate after the warm-up period ends. Indicates the number of preheating cycles; The second phase is the learning rate scheduling phase, which refers to the phase where the current epoch number is greater than the preset number. The learning rate is set to a value, at which point the cosine annealing method is used as the scheduling strategy, and the following relationship exists in the corresponding process: ; in, Indicating the second stage The learning rate for each training round. This indicates the current training round number in the second phase. This represents the minimum learning rate. This represents the maximum learning rate. This indicates taking the cosine value. This indicates the maximum number of training rounds.

[0022] It should be noted that the maximum number of training rounds... Set the initial learning rate to 150. Set to 0.01.

[0023] It should be understood that although the steps in the flowcharts of the various embodiments of the present invention are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the various embodiments may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0024] It should be appreciated that portions of the present application can be implemented in hardware, software, firmware, or combinations thereof. In the embodiments described above, various steps or methods can be implemented, in part, or in whole, in software, or firmware that is stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any of the above-described functionality can be implemented with or without the use of

[0025] In the description of the present application, the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" are intended to mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. The illustrative appearances of the above- described terms in various places in the specification are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0026] The above-described embodiments are merely illustrative of the present application and are not to be taken in a limiting sense. It is contemplated that various modifications and changes can be made to the present application without departing from the spirit and scope of the application. Therefore, the scope of the present application is not to be limited to the specific embodiments described herein. Rather, the scope of the present application is to be accorded the broadest scope of the claims appended hereto.

Claims

1. A multimodal target detection method based on graph network information interaction, characterized in that, The method includes the following steps: Step 1: Construct a feature extraction module based on a pre-trained Swin-Transformer network with two shared parameters, construct a multimodal feature interaction module based on a graph network, construct a gated fusion module based on a gated fusion mechanism, and use the feature extraction module, multimodal feature interaction module, and gated fusion module with the detection head to form a multimodal target detection model; Step 2: Use a camera and an infrared thermal imaging head to capture images of the current scene to obtain visible light and infrared images respectively; use a feature extraction module to extract multi-scale features from the visible light and infrared images to obtain visible light multi-scale feature maps and infrared multi-scale feature maps. Step 3: Use the multimodal feature interaction module to establish intermodal and intramodal information interaction between the visible light multi-scale feature map and the infrared multi-scale feature map, so as to obtain the network information interaction features of the visible light modal map and the network information interaction features of the infrared modal map respectively. Step 4: Input the visible light modal graph network information interaction features and the infrared modal graph network information interaction features into the gated fusion module for fusion to obtain multimodal fusion features; Step 5: Input the multimodal fusion features into the detection head to obtain the predicted bounding box. Based on the predicted bounding box, construct the classification loss function, confidence loss function, and localization loss function respectively. Use the classification loss function, confidence loss function, and localization loss function to train the multimodal object detection model to obtain the trained multimodal object detection model. Use the trained multimodal object detection model to determine the final category information and bounding box information.

2. The multimodal target detection method based on graph network information interaction according to claim 1, characterized in that, In step 2, the feature extraction module is used to extract multi-scale features from the visible light image and the infrared image respectively, so as to obtain the visible light multi-scale feature map and the infrared multi-scale feature map. Specifically, it includes the following sub-steps: Based on two pre-trained Swin-Transformer networks with shared parameters, one of the pre-trained Swin-Transformer networks is used to extract multi-scale features of visible light images to obtain a visible light multi-scale feature map. Another pre-trained Swin-Transformer network is used to extract multi-scale features from infrared images to obtain infrared multi-scale feature maps.

3. The multimodal target detection method based on graph network information interaction according to claim 2, characterized in that, In step 3, the multimodal feature interaction module is used to establish intermodal and intramodal information interaction between the visible light multi-scale feature map and the infrared multi-scale feature map, so as to obtain the network information interaction features of the visible light modality map and the network information interaction features of the infrared modality map, respectively. Specifically, it includes the following sub-steps: The visible light multi-scale feature map and the infrared multi-scale feature map are divided into two groups of sub-feature maps according to the channel, and global average pooling is performed on each sub-feature map to obtain node features. All node features are merged to obtain the total node embedding; The similarity between modes is calculated based on the total node embedding to construct the adjacency matrix; Graph reasoning is performed based on the adjacency matrix and the total node embedding to capture the relationships between nodes and obtain new node embeddings after graph reasoning. The new nodes, after graph inference, are embedded and reprojected onto the channel dimension to obtain the channel weights. ; in, Represents the visible light mode channel weights. Indicates the infrared modal channel weights; The visible light modal channel weights are multiplied with the visible light multi-scale feature map, and complementary information of the multi-modal channels is introduced to obtain a visible light feature map with multi-modal information interaction. The infrared modal channel weights are multiplied with the infrared multi-scale feature map, and complementary information of the multi-modal channels is introduced to obtain an infrared feature map with multi-modal information interaction. The visible light feature map with multimodal information interaction is converted into spatial dimension node embedding by graph projection, and the correlation of contextual semantic information in different spatial regions is modeled by graph reasoning to obtain the network information interaction features of visible light modal map. The visible light feature map with multimodal information interaction is converted into spatial dimension node embedding by graph projection, and the correlation of contextual semantic information in different spatial regions is modeled by graph reasoning to obtain the information interaction features of infrared modal graph network.

4. The multimodal target detection method based on graph network information interaction according to claim 3, characterized in that, In the steps of merging all node features to obtain the total node embedding, and calculating the similarity between modes based on the total node embedding to construct the adjacency matrix, the following relationship exists: ； in, Represents the adjacency matrix. This indicates that the normalization operation has been performed. Indicates the embedding of the main node. Indicates transpose; In the step of graph reasoning based on the adjacency matrix and total node embeddings to capture the relationships between nodes and obtain new node embeddings after graph reasoning, the following relationship exists: ； in, This represents the new node embedding after graph reasoning. This indicates that the process has been performed using a non-linear activation function. This represents a learnable parameter; In the step of multiplying the visible light modal channel weights with the visible light multi-scale feature map and introducing complementary information from the multi-modal channels to obtain a visible light feature map with multi-modal information interaction, the following relationship exists: ； in, This represents a visible light feature map with multimodal information interaction. Represents a multi-scale feature map of visible light. This indicates element-wise addition. This indicates element-wise multiplication. This indicates that the process has passed through a fully connected layer with a Sigmoid activation function. In the step of multiplying the infrared modal channel weights with the infrared multi-scale feature map and introducing complementary information from the multi-modal channels to obtain an infrared feature map with multi-modal information interaction, the following relationship exists: ； in, This represents an infrared feature map with multimodal information interaction. This represents an infrared multi-scale feature map.

5. The multimodal target detection method based on graph network information interaction according to claim 4, characterized in that, In the steps of converting a visible light feature map with multimodal information interaction into spatial dimension node embeddings through graph projection, and modeling the correlation of contextual semantic information in different spatial regions using graph reasoning to obtain the network information interaction features of the visible light modality map, the following relationship exists: ； in, This represents the information interaction characteristics of visible light modal graph networks. The function representing the reprojection of a graph. This indicates that graph reasoning operations have been performed. The projection function of the graph; In the steps of converting visible light feature maps with multimodal information interaction into spatial dimension node embeddings through graph projection, and modeling the correlation of contextual semantic information in different spatial regions using graph reasoning to obtain the information interaction features of infrared modal graph networks, the following relationship exists: ； in, This represents the information interaction characteristics of infrared modal graph networks.

6. The multimodal target detection method based on graph network information interaction according to claim 5, characterized in that, In step 4, the visible light modal graph network information interaction features and the infrared modal graph network information interaction features are input into the gated fusion module for fusion to obtain multimodal fusion features. This specifically includes the following sub-steps: The information interaction features of the visible light modal graph network and the information interaction features of the infrared modal graph network are stacked along the channel dimension to obtain the stacked multimodal features. The stacked multimodal features are sequentially processed through 3D convolutional layers and a Sigmoid activation function to obtain feature weights; The feature weights are multiplied with the stacked multimodal features and then residual-connected with the stacked multimodal features to obtain the features filtered by the gating mechanism. The features filtered by the gating mechanism are then processed sequentially through feature reshaping and 3D convolutional layers to obtain multimodal fusion features.

7. The multimodal target detection method based on graph network information interaction according to claim 6, characterized in that, In the step of stacking the visible light modality map network information interaction features and the infrared modality map network information interaction features along the channel dimension to obtain the stacked multimodal features, the following relationship exists: ； in, This represents the stacked multimodal features. This indicates stacking along the channel dimension; In the steps of multiplying the feature weights with the stacked multimodal features and then performing a residual connection with the stacked multimodal features to obtain the features filtered by the gating mechanism, the following relationship exists: ； in, This indicates the features after filtering by the gating mechanism. This indicates that the process has been performed using the Sigmoid activation function. This indicates that the process has been performed using 3D convolutional layers; In the process of sequentially reshaping and processing the gated features through feature reshaping and 3D convolutional layers to obtain multimodal fused features, the following relationship exists: ； in, Indicates multimodal fusion features, This indicates that the feature has been reshaped.

8. The multimodal target detection method based on graph network information interaction according to claim 7, characterized in that, In step 5, the multimodal fusion features are input into the detection head to obtain predicted bounding boxes. Based on the predicted bounding boxes, classification loss functions, confidence loss functions, and localization loss functions are constructed respectively. The multimodal object detection model is trained using the classification loss function, confidence loss function, and localization loss function to obtain the trained multimodal object detection model. The final category information and bounding box information are determined using the trained multimodal object detection model. The expression for the classification loss function is: ； in, Represents classification loss, Indicates the weighting factor. Indicates the tag value. This represents the model's predicted probability of the true class. This indicates adjustable focus parameters. Indicates taking the logarithm; The expression for the confidence loss function is: ； in, Indicates confidence loss. Indicates the number of samples. Indicates the index of the target being detected. Indicates the first The true label of each sample Indicates the first The prediction confidence level of each sample; The expression for the localization loss function is: ； in, This indicates the consistency used to measure the ratio of bounding box heights. This indicates taking the arctangent value. This represents the width of the actual bounding box. This represents the height of the actual bounding box. This indicates the width of the predicted bounding box. Indicates the height of the predicted bounding box. Represents the weighting function. This represents the intersection-union ratio (IoU) between the predicted bounding box and the ground truth bounding box. This represents the Euclidean distance between the predicted point and the actual point. Indicates the center point of the predicted bounding box. This represents the center point of the actual bounding box. This represents the diagonal length of the outermost rectangle that minimizes the difference between the predicted and actual bounding boxes. This indicates positioning loss.

Images