A target detection method based on global context feature fusion knowledge graph

Abstract

The application discloses a target detection method based on a global context feature fusion knowledge graph, constructs a neural network model based on a global context feature and a knowledge graph, trains the neural network model by using a COCO2017 dataset, predicts a prediction box category probability matrix of a to-be-detected image through the neural network model, obtains a semantic consistency matrix through quantitative processing of the knowledge graph, updates the prediction box category probability matrix through the semantic consistency matrix, and finally outputs the position and category label of each target in the image.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of target detection, and more specifically relates to a target detection method based on global context feature fusion knowledge graph. BACKGROUND

[0002] In recent years, artificial intelligence, as an important force driving the upgrading of traditional industries and technological revolution, has made many industries flourish. With the continuous upgrading of computer hardware, people can use more complex algorithms to process and analyze digital images.

[0003] In computer vision tasks, the basic problems to be solved include image classification, target detection, and image segmentation. Among them, image classification can identify the category of objects in a given image; target detection can not only identify the category of objects in a given image, but also predict the position of each object. Image segmentation can be regarded as a special setting of target detection, in which localization of the target is not required by the bounding box, but pixel-level localization is required. Therefore, as one of the most basic and most challenging problems in computer vision, target detection is the basis for solving complex or advanced tasks.

[0004] Convolutional neural networks can convert raw pixels into higher-level semantic information and automatically learn from a large amount of training data, thereby effectively solving various complex problems in computer vision. After achieving outstanding results in image classification tasks using convolutional neural networks, target detection technology based on deep learning has also made major breakthroughs. In particular, for some specific target detection tasks, such as medical image analysis, autonomous driving, and face recognition, these tasks have unique data characteristics and application requirements, and require deep learning models to have better generalization ability and adaptability. At present, advanced target detectors mostly use deep learning networks as the backbone and continuously improve and optimize them to improve the performance of the model.

[0005] Although the target detection model can automatically detect the position and category information of objects in the image, there is still a significant gap between the learning methods of machines and humans. The existing most advanced target detection algorithm only focuses on utilizing the features of the image itself, largely ignoring a large amount of background knowledge about the real world. In large-scale, multi-class target detection tasks, it is difficult to identify objects that are severely occluded and of small size, but humans can instantly judge them. The above is not because humans can better draw conclusions from target features, but because humans can rely on prior knowledge (such as the relationship between objects and context information) to make judgments or inferences and identify the specific category of the object. SUMMARY

[0006] The purpose of this invention is to overcome the shortcomings of the prior art and provide a target detection method based on global context feature fusion knowledge graph, which makes full use of the image's own features and captures the relationship between target objects in the image to improve detection performance.

[0007] To achieve the above-mentioned objectives, this invention provides a target detection method based on global contextual feature fusion knowledge graph, characterized by comprising the following steps:

[0008] (1) Download the dataset

[0009] Download a dataset containing object detection for N categories, with multiple images for each category, and all images are uniformly scaled to the same size; then add ground truth bounding boxes to the objects in each image and label them with category labels;

[0010] Download a knowledge base containing M knowledge items and extract the head entity s of each knowledge item. i Tail entity e i and relation r i Then combine them into triples (s i ,r i ,e i ), i = 1, 2, ..., M;

[0011] (2) Construct a neural network model based on global context features;

[0012] The neural network model based on global context features consists of three parts: backbone, bottleneck, and detection.

[0013] The main body includes a Focus module, five convolutional modules (CBL), four global context feature modules (GCF), a fast spatial pyramid pooling module (SPPF), and an upsampling module.

[0014] Each CBL consists of a convolutional layer, a batch normalization layer, and a LeakyReLU activation function;

[0015] Each GCF consists of three 1×1 convolutional layers and two activation functions. The output of the first convolutional layer is connected to the softmax function, and the output of the second convolutional layer is connected to the LeakyReLU activation function.

[0016] At the output of Focus, CBL and GCF are connected in series in sequence, and CBL and GCF are connected alternately; at the output of GCF, SPPF, CBL and upsampling module are connected in series in sequence.

[0017] The bottleneck section adopts a cross-stage local structure, including three branches;

[0018] Specifically, the third CBL at the Focus output end in the main section is spliced ​​with the output of the upsampling module in the channel direction, and then input to the upsampling module after passing through GCF and CBL. Then, the output of the upsampling module is spliced ​​with the second CBL output at the Focus output end in the channel direction. Finally, the splicing result is fused through GCF to obtain the first branch output of the bottleneck section.

[0019] The output of the first branch is concatenated with the output of the CBL in the first branch in the channel direction after passing through CBL. The concatenated result is then fused through GCF to obtain the output of the second branch of the bottleneck part.

[0020] The output of the second branch is concatenated with the output of the last CBL at the output end of the main branch in the channel direction after passing through CBL. The concatenation result is then fused through GCF to obtain the third branch output of the bottleneck part.

[0021] The detection part includes three convolutional modules (CBLs), each CBL connected to the output of three branches. The fused prediction box category probability matrix P is obtained from the three branches of the bottleneck part through convolution operations.

[0022] (3) Construct a knowledge graph and quantify its processing;

[0023] (3.1) Constructing a knowledge graph;

[0024] Based on the object detection dataset, extract all triples containing N categories of the object detection dataset from the knowledge base to form a knowledge graph of the object detection dataset;

[0025] (3.2) Quantitative knowledge graph;

[0026] (3.2.1) If the head and tail entities of triples of the same category in the knowledge graph are uniformly labeled as entity nodes v, then the entity node labeled for the triple of the i-th category in the knowledge graph is denoted as v. i , i = 1, 2, ..., N;

[0027] (3.2.2) Use the restart random walk algorithm to traverse the knowledge graph and calculate the semantic consistency index between any two entity nodes;

[0028]

[0029]

[0030] Among them, s ij P(v) represents the semantic consistency index between the i-th entity node and the j-th entity node, where i,j[1,N] and i≠j; j |v i:α) indicates from entity node v i After t steps, the entity node v is reached. j The probability; α represents the probability of returning to entity node v after each step. i ;

[0031] (3.2.3) Construct the semantic consistency matrix S;

[0032]

[0033] (4) Train a neural network model based on global context features;

[0034] In each round of training, an image is randomly selected from the object detection dataset, denoted as X, and then image X is input into a neural network model based on global context features;

[0035] In the backbone network, the input image X is first enhanced by Focus. Then the image The data flows sequentially through several cascaded CBLs and GCFs in tensor form, then through SPPF and upsampling layer modules before entering the bottleneck section. In the bottleneck section, feature extraction is performed using cross-stage local structures to obtain three prediction maps X of different sizes. k k = 1, 2, 3; In the detection part, the predicted graph X k The convolution operation is used to obtain the fused prediction box category probability matrix P from the three branches of the bottleneck part;

[0036]

[0037] Where, p il Let l represent the probability that the l-th predicted box belongs to the i-th category, where l = 1, 2, ..., L, and L represents the number of predicted boxes.

[0038] Calculate the loss function value (Loss) after this round of training:

[0039] Loss=λ1L cls +λ2L obj +λ3L loc

[0040]

[0041]

[0042]

[0043]

[0044]

[0045] Where λ1, λ2, λ3 are balance coefficients; L cls L represents the classification loss. obj L represents the target loss. loc Indicates positioning loss; O il ∈[0,1], when O il When = 0, it means that there is no target of type i in the l-th prediction box; when = 0, it means that there is no target of type i in the l-th prediction box. il =1 indicates that the i-th type of target exists in the l-th prediction box; l c represents the overlap area between the l-th predicted bounding box and its corresponding ground truth bounding box; l The confidence level of the l-th predicted box; ρ(x l ,y l ) represents the center point x of the l-th prediction box. l The center point y of the corresponding real target bounding box l Euclidean distance; d l This represents the diagonal length of the smallest bounding rectangle of the l-th prediction box; w represents the length and width of the l-th prediction box. l h l CIoU represents the length and width of the l-th predicted bounding box and its corresponding ground truth bounding box. l υ l As an intermediate variable;

[0046] Determine whether the total loss value (Loss) of the network has converged or whether the current iteration number has reached the preset maximum value. If both conditions are met, the iteration stops, and a trained neural network model based on global context features is obtained. Otherwise, backpropagation is performed using the Loss function, the network parameters are updated, and then the next round of training is carried out.

[0047] (5) Real-time target detection based on global context feature fusion knowledge graph;

[0048] (5.1) In real time, an image is acquired and input into a trained neural network model based on global context features, thereby outputting a prediction box category probability matrix P;

[0049] (5.2) Update the predicted box category probability matrix P based on the semantic consistency matrix S;

[0050]

[0051] in, p il The updated value, where ε is a hyperparameter, ε∈(0,1);

[0052] (5.3) In the updated predicted bounding box category probability matrix In this process, the final classification label for each predicted bounding box is calculated;

[0053]

[0054] The final output image contains the location and category label of each target.

[0055] The objective of this invention is achieved as follows:

[0056] This invention presents a target detection method based on global contextual features fused with a knowledge graph. First, a neural network model and a knowledge graph based on global contextual features are constructed. Then, the neural network model is trained using the COCO2017 dataset. The neural network model predicts the probability matrix of the bounding box category of the image to be detected. Next, the semantic consistency matrix is ​​obtained by quantizing the knowledge graph. The semantic consistency matrix is ​​then used to update the probability matrix of the bounding box category. Finally, the location and category label of each target in the image are output.

[0057] Meanwhile, the object detection method based on global contextual feature fusion knowledge graph of the present invention also has the following beneficial effects:

[0058] (1) Compared with traditional object detection networks, neural networks based on global context features can capture the relationship information between different spatial locations of the image while focusing on the image's own features, and enhance the original features by aggregating the same global features to improve the performance of the detector.

[0059] (2) Compared with traditional object detection networks, this invention constructs a knowledge graph using the ConceptNet dataset and integrates a neural network based on global context features with the knowledge graph. This fully utilizes the image's own features while capturing the relationship features between objects in the image to improve the detector's performance. Attached Figure Description

[0060] Figure 1 This is a flowchart of the target detection method based on global contextual feature fusion knowledge graph of the present invention;

[0061] Figure 2 It is a diagram of a neural network model based on global context features;

[0062] Figure 3 It is a flowchart for quantifying knowledge graphs;

[0063] Figure 4 This is a comparison chart of the test results of the YOLOv5s model and the ODFKG model on a kitchen scene graph.

[0064] Figure 5 This is a comparison chart of the test results of the YOLOv5s model and the ODFKG model on a traffic scene map. Detailed Implementation

[0065] The specific embodiments of the present invention will now be described with reference to the accompanying drawings to enable those skilled in the art to better understand the invention. It should be particularly noted that in the following description, detailed descriptions of known functions and designs that might obscure the main content of the invention will be omitted here.

[0066] Example

[0067] Figure 1 This is a flowchart of the target detection method based on global context feature fusion knowledge graph of the present invention.

[0068] In this embodiment, as Figure 1 As shown, the present invention provides a target detection method based on global contextual feature fusion knowledge graph, comprising the following steps:

[0069] S1. Download the dataset

[0070] Download the COCO2017 dataset, which contains 80 categories and includes complex everyday scenes and common objects in natural environments, making it more realistic. Only about 10% of the images in the COCO2017 dataset contain a single object category, while the remaining images contain an average of 3.5 categories per image. Each category has multiple images, all uniformly scaled to the same size; then, bounding boxes are added to the objects in each image, and category labels are added.

[0071] COCO2017 has highly concentrated and small objects, which increases the difficulty of detection and also better verifies the performance of the method proposed in this invention.

[0072] ConceptNet is a large, free knowledge graph that focuses on the common-sense meanings of words used in natural language. It is also a knowledge graph with a greater emphasis on constructing relational knowledge. Approaching natural language description, ConceptNet includes all categories from the COCO2017 dataset and the relationships between them.

[0073] Each piece of knowledge in the ConceptNet knowledge base, in CSV format, is processed into a triple of head entity, tail entity, and relation, and then negative relations are filtered out before storage.

[0074] S2. Construct a neural network model based on global context features;

[0075] In this embodiment, as Figure 2 As shown, the neural network model based on global context features consists of three parts: the backbone, the bottleneck, and the detection.

[0076] The main body includes a Focus module, five convolutional modules (CBL), four global context feature modules (GCF), a fast spatial pyramid pooling module (SPPF), and an upsampling module.

[0077] Each CBL consists of a convolutional layer, a batch normalization layer, and a LeakyReLU activation function;

[0078] Each GCF consists of three 1×1 convolutional layers and two activation functions. The output of the first convolutional layer is connected to the softmax function, and the output of the second convolutional layer is connected to the LeakyReLU activation function.

[0079] At the output of Focus, CBL and GCF are connected in series in sequence, and CBL and GCF are connected alternately; at the output of GCF, SPPF, CBL and upsampling module are connected in series in sequence.

[0080] The bottleneck section adopts a cross-stage local structure, including three branches;

[0081] Specifically, the third CBL at the Focus output end in the main section is spliced ​​with the output of the upsampling module in the channel direction, and then input to the upsampling module after passing through GCF and CBL. Then, the output of the upsampling module is spliced ​​with the second CBL output at the Focus output end in the channel direction. Finally, the splicing result is fused through GCF to obtain the first branch output of the bottleneck section.

[0082] The output of the first branch is concatenated with the output of the CBL in the first branch in the channel direction after passing through CBL. The concatenated result is then fused through GCF to obtain the output of the second branch of the bottleneck part.

[0083] The output of the second branch is concatenated with the output of the last CBL at the output end of the main branch in the channel direction after passing through CBL. The concatenation result is then fused through GCF to obtain the third branch output of the bottleneck part.

[0084] The detection part includes three convolutional modules (CBLs), each CBL connected to the output of three branches. The fused prediction box category probability matrix P is obtained from the three branches of the bottleneck part through convolution operations.

[0085] S3. Construct a knowledge graph and quantify its processing.

[0086] S3.1 Constructing a knowledge graph;

[0087] like Figure 3As shown, based on the COCO2017 dataset, all triples containing 80 categories from the ConceptNet knowledge base are extracted to form the knowledge graph of the object detection dataset.

[0088] S3.2, Quantitative Knowledge Graph;

[0089] S3.2.1 Label the head and tail entities of triples of the same category in the knowledge graph as entity nodes v. Then, the entity node labeled for the triple of the i-th category in the knowledge graph is denoted as v. i , i = 1, 2, ..., N;

[0090] S3.2.2 Utilize the restart random walk algorithm to traverse the knowledge graph and calculate the semantic consistency index between any two entity nodes;

[0091]

[0092]

[0093] Among them, s ij P(v) represents the semantic consistency index between the i-th entity node and the j-th entity node, where i,j[1,N] and i≠j; j |v i :α) indicates from entity node v i After t steps, the entity node v is reached. j The probability; α represents the probability of returning to entity node v after each step. i ;

[0094] S3.2.3 Construct the semantic consistency matrix S;

[0095]

[0096] S4. Train a neural network model based on global context features;

[0097] In each round of training, an image is randomly selected from the object detection dataset, denoted as X, and then image X is input into a neural network model based on global context features;

[0098] In the backbone network, the input image X is first enhanced by Focus. Then the image The data flows sequentially through several cascaded CBLs and GCFs in tensor form, then through SPPF and upsampling layer modules before entering the bottleneck section. In the bottleneck section, feature extraction is performed using cross-stage local structures to obtain three prediction maps X of different sizes. k k = 1, 2, 3; In the detection part, the predicted graph Xk The convolution operation is used to obtain the fused prediction box category probability matrix P from the three branches of the bottleneck part;

[0099]

[0100] Where, p il Let l represent the probability that the l-th predicted box belongs to the i-th category, where l = 1, 2, ..., L, and L represents the number of predicted boxes.

[0101] Calculate the loss function value (Loss) after this round of training:

[0102] Loss=λ1L cls +λ2L obj +λ3L loc

[0103]

[0104]

[0105]

[0106]

[0107]

[0108] Where λ1, λ2, λ3 are balance coefficients; L cls L represents the classification loss. obj L represents the target loss. loc Indicates positioning loss; O il ∈[0,1], when O il When = 0, it means that there is no target of type i in the l-th prediction box; when = 0, it means that there is no target of type i in the l-th prediction box. il =1 indicates that the i-th type of target exists in the l-th prediction box; l c represents the overlap area between the l-th predicted bounding box and its corresponding ground truth bounding box; l The confidence level of the l-th predicted box; ρ(x l ,y l ) represents the center point x of the l-th prediction box. l The center point y of the corresponding real target bounding box l Euclidean distance; d l This represents the diagonal length of the smallest bounding rectangle of the l-th prediction box; w represents the length and width of the l-th prediction box. l h l CIoU represents the length and width of the l-th predicted bounding box and its corresponding ground truth bounding box. l υ l As an intermediate variable;

[0109] Determine whether the total loss value (Loss) of the network has converged or whether the current iteration number has reached the preset maximum value. If both conditions are met, the iteration stops, and a trained neural network model based on global context features is obtained. Otherwise, backpropagation is performed using the Loss function, the network parameters are updated, and then the next round of training is carried out.

[0110] S5. Real-time target detection network based on global contextual feature fusion knowledge graph;

[0111] S5.1. In real time, an image is acquired and input into a trained neural network model based on global context features, thereby outputting a prediction box category probability matrix P.

[0112] S5.2 Update the predicted bounding box category probability matrix P based on the semantic consistency matrix S;

[0113]

[0114] in, p il The updated value, where ε is a hyperparameter, ε∈(0,1);

[0115] S5.3, In the updated predicted bounding box category probability matrix In this process, the final classification label for each predicted bounding box is calculated;

[0116]

[0117] The final output image contains the location and category label of each target.

[0118] Experiment and Results Analysis

[0119] This embodiment uses a series of experiments to verify the effectiveness of the target detection method based on global context feature fusion knowledge graph.

[0120] Experimental setup

[0121] This embodiment is implemented based on the open-source machine learning library PyTorch, version 1.13.1. The experimental software environment is a 64-bit Ubuntu 22.04.1 LTS system, Python version 3.7, and CUDA version 11.6. The experimental hardware environment consists of an NVIDIA GeForce RTX 3090Ti GPU, a 14-core Intel(R) Xeon(R) E5-2680 CPU, and 16GB of RAM.

[0122] The object detection method based on global contextual feature fusion knowledge graph was tested and its performance analyzed on the COCO2017 dataset. The model was trained on the training set of the COCO2017 dataset and tested on the validation set of COCO2017.

[0123] Comparative experiment

[0124] To verify the effectiveness of the object detection method based on global context feature fusion knowledge graph proposed in this invention, this embodiment selects some representative models from the development of object detection technology for comparison, including the two-stage model Faster R-CNN, the single-stage model SSD, and the YOLO series. The input image size for the YOLOv5s model and the global context feature fusion knowledge graph-based object detection model ODFKG is 640×640, while the input image size for other models is 448×448. Experiments and performance analysis of these models were conducted on the COCO2017 dataset. Since this invention is based on the YOLOv5s model, for fair comparison, this embodiment reproduces the YOLOv5s model under the same hardware conditions as the ODFKG model. In this embodiment, all metrics are expressed as percentages, omitting the "%". The model with the best performance on each metric is highlighted in bold. The detection accuracy comparison of different models on the COCO2017 validation set is shown in Table 1, where "KG" represents the knowledge graph module and "GCF" represents the global context feature module.

[0125]

[0126] Table 1 Comparison of detection accuracy of different models on the COCO2017 validation set

[0127] When the IoU is 0.5, the ODFKG object detection model, based on global contextual feature fusion knowledge graph, achieves a detection accuracy of 55.9%, which is 13.2% higher than the two-stage Faster R-CNN model and 0.7% higher than the best-performing single-stage YOLOv5s model. When the IoU is averaged over {0.5:0.95}, the ODFKG model achieves a detection accuracy of 35.6%, which is 13.7% higher than the Faster R-CNN model and 0.3% higher than the YOLOv5s model.

[0128] The above analysis shows that the target detection model ODFKG based on global context feature fusion knowledge graph outperforms other single-stage and two-stage models, verifying the effectiveness of fusing knowledge graph and global context feature module.

[0129] In the COCO2017 dataset, over 90% of the images contain multi-class objects. If there are too many targets in an image, problems such as occlusion and target blurring will inevitably occur. However, compared with the YOLOv5s model, the ODFKG model showed good detection performance on the COCO2017 dataset.

[0130] To further analyze the detection performance of the ODFKG model, this embodiment compares the accuracy of the YOLOv5s model and the ODFKG model on some categories of the COCO2017 validation set, as shown in Table 2.

[0131]

[0132]

[0133] Table 2

[0134] The categories in Table 2 can be roughly divided into kitchen scenes and traffic scenes. The detection performance of the ODFKG model will be analyzed from these two scenes respectively.

[0135] The kitchen scenario likely includes classes such as microwave, oven, and refrigerator. In Table 2, for the microwave class, compared to the YOLOv5s model, the ODFKG model improves mAP by 2.2% and mAP@0.5 by 2.9%; for the oven class, compared to the YOLOv5s model, the ODFKG model improves mAP by 1.2% and mAP@0.5 by 1.9%; for the refrigerator class, compared to the YOLOv5s model, the ODFKG model improves mAP by 0.2% and mAP@0.5 by 0.4%.

[0136] In summary, in the kitchen scene, the ODFKG model outperforms the YOLOv5s model in detection accuracy across all categories, with significant improvements in the detection accuracy of microwave and oven. However, in traditional object detection tasks, the microwave and oven classes are generally easily occluded, making them difficult to identify; the refrigerator class typically occupies a large portion of the image and is easily recognized by the detector. The semantic consistency relationship between the refrigerator class and the microwave and oven classes enhances the detection of the microwave and oven classes. Figure 4 (a) shows the test results for the YOLOv5s model. Figure 4 (b) shows the test results of the ODFKG model. Compared with the YOLOv5s model, the ODFKG model has a significant improvement, with detection accuracies of 0.6, 0.5 and 0.7 for microwave, oven and refrigerator classes, respectively.

[0137] In addition to the "person" class, which has the most tags, traffic scenarios also include classes such as traffic light, bus, and car. In Table 2, compared to the YOLOv5s model, for the traffic light class, the ODFKG model improves mAP by 0.8% and mAP@0.5 by 3.7%; for the bus class, the ODFKG model improves mAP by 0.3% and mAP@0.5 by 0.4%; and for the car class, the ODFKG model improves mAP by 0.1% while mAP@0.5 remains unchanged.

[0138] In summary, in simple traffic scenarios, the ODFKG model outperforms the YOLOv5s model in detection accuracy across all categories, with the most significant improvement in traffic light detection accuracy. However, in traditional object detection tasks, the traffic light category typically occupies less than 1% of the image area, such as... Figure 5 As shown, it is difficult to detect accurately. Figure 5 (a) shows the test results for the YOLOv5s model. Figure 5 (b) shows the test results of the ODFKG model. This invention extracts semantic consistency features between traffic lights, buses, and cars by constructing a knowledge graph and capturing global contextual features, thereby enhancing the detection of traffic lights to an accuracy of 0.8.

[0139] Although the illustrative specific embodiments of the present invention have been described above to enable those skilled in the art to understand the invention, it should be understood that the invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the invention as defined and determined by the appended claims, and all inventions utilizing the concept of the present invention are protected.

Claims

1. A target detection method based on global contextual feature fusion knowledge graph, characterized in that, Includes the following steps: (1) Download the dataset; The download includes A dataset for object detection in several categories, with multiple images for each category, and each image is uniformly scaled to the same size; then, bounding boxes are added to the objects in each image, and category labels are added; The download includes The knowledge base contains entries, from which the head entity of each entry is extracted. Tail entity and relationships Then combine them into triples , ; (2) Construct a neural network model based on global context features; The neural network model based on global context features consists of three parts: backbone, bottleneck, and detection. The main body includes a Focus module, five convolutional modules (CBL), four global context feature modules (GCF), a fast spatial pyramid pooling module (SPPF), and an upsampling module. Each CBL consists of a convolutional layer, a batch normalization layer, and a LeakyReLU activation function; Each GCF consists of three The convolutional layer consists of two activation functions, where the output of the first convolutional layer is connected to the softmax function, and the output of the second convolutional layer is connected to the LeakyReLU activation function. At the output of the Focus module, CBL and GCF are connected in series in sequence, with CBL and GCF being connected alternately; at the output of the fourth GCF module, SPPF, CBL and upsampling module are connected in series in sequence. The bottleneck section adopts a cross-stage local structure, including three branches; Specifically, the third CBL at the Focus output end in the main section is spliced ​​with the output of the upsampling module in the channel direction, and then input to the upsampling module after passing through GCF and CBL. Then, the output of the upsampling module is spliced ​​with the second CBL output at the Focus output end in the channel direction. Finally, the splicing result is fused through GCF to obtain the first branch output of the bottleneck section. The output of the first branch is concatenated with the output of the CBL in the first branch in the channel direction after passing through CBL. The concatenated result is then fused through GCF to obtain the output of the second branch of the bottleneck part. The output of the second branch is concatenated with the output of the last CBL at the output end of the main branch in the channel direction after passing through CBL. The concatenation result is then fused through GCF to obtain the third branch output of the bottleneck part. The detection part includes three convolutional modules (CBLs), each CBL connected to the output of three branches. The fused prediction box category probability matrix P is obtained from the three branches of the bottleneck part through convolution operations. (3) Construct a knowledge graph and quantify it; (3.1) Constructing a knowledge graph; Based on the object detection dataset, extract information from the knowledge base that contains the object detection dataset. All triples of each category constitute the knowledge graph of the object detection dataset; (3.2) Quantitative knowledge graph; (3.2.1) If the head and tail entities of triples of the same category in the knowledge graph are uniformly labeled as entity nodes v, then the first... The entity nodes labeled with triples of each category are denoted as follows: , ; (3.2.2) Use the restart random walk algorithm to traverse the knowledge graph and calculate the semantic consistency index between any two entity nodes; ； ； in, Indicates the first The entity node and the first Semantic consistency metrics among entity nodes and ; Indicates from entity node go through Step to reach the entity node The probability of; This indicates that there is a certain number of moves when moving one step. The probability of returning to the entity node ; (3.2.3) Construct the semantic consistency matrix S; ； (4) Train a neural network model based on global context features; In each round of training, an image is randomly selected from the object detection dataset, denoted as . Then the image Input is fed into a neural network model based on global contextual features; In the backbone network section, the input image... First, obtain the enhanced image using Focus. Then the image The data flows sequentially through several cascaded CBLs and GCFs in tensor form, then through SPPF and upsampling layer modules before entering the bottleneck section. In the bottleneck section, feature extraction is performed using cross-stage local structures, resulting in prediction maps of three different sizes. , In the detection section, the prediction graph The convolution operation is used to obtain the fused prediction box class probability matrix from the three branches of the bottleneck part. ; ； in, Indicates the first The predicted bounding box belongs to the first... The probability of each category, , Indicates the number of prediction boxes; Calculate the loss function value after this round of training. : ； ； ； ； ； ； in, This is the balance coefficient; Represents classification loss, Indicates target loss. Indicates positioning loss; ,when When =0, it means the first... The first prediction box does not exist. Class target, when When =1, it means the first... The prediction box contains the first... Class target; Indicates the first The overlap area between each predicted bounding box and its corresponding ground truth bounding box; No. Confidence level of each predicted bounding box; Indicates the first The center point of each prediction box The center point of its corresponding real target bounding box The Euclidean distance; Indicates the first The diagonal length of the smallest bounding rectangle of each prediction box; , Indicates the first The length and width of each prediction box, , Indicates the first The length and width of each predicted bounding box and its corresponding ground truth bounding box; , As an intermediate variable; Determine whether the total loss value (Loss) of the network has converged or whether the current iteration number has reached the preset maximum value. If both conditions are met, the iteration stops, and a trained neural network model based on global context features is obtained. Otherwise, backpropagation is performed using the Loss function, the network parameters are updated, and then the next round of training is carried out. (5) Real-time target detection based on global context feature fusion knowledge graph; (5.1) A real-time image is acquired and input into a trained neural network model based on global context features, thereby outputting a prediction box category probability matrix P; (5.2) Update the predicted box category probability matrix P based on the semantic consistency matrix S; ； in, express The updated value, For hyperparameters, ; (5.3) In the updated predicted bounding box category probability matrix In this process, the final classification label for each predicted bounding box is calculated; ； The final output image contains the location and category label of each target.

Images