A method for object detection in urban environments

Abstract

A method for target detection in an urban environment belongs to the technical field of target detection, and specifically relates to the technical field of target detection in an urban environment. It solves the problem that it is difficult for a single-stage detector to combine classification and detection regression into one stage, which leads to inaccurate results when performing target detection in a city. The method of the present invention improves the YOLOv7 model, reduces the complexity of the model, and realizes the lightweight of the model. After obtaining the target coordinates, target category and confidence by improving the YOLOv7 model, these data are processed, the confidence is recalculated, the real confidence is obtained, and all the information is organized into a training data set for constructing a heterogeneous graph neural network (SeHGNN network). Finally, the inference confidence obtained by the heterogeneous graph neural network is used to replace the confidence obtained by the improved YOLOv7 model, thereby improving the accuracy of target detection.

Description

Technical Field

[0001] The present invention belongs to the field of target detection technology, and in particular to the field of target detection technology in urban environments. Background Art

[0002] In urban environments, due to the large number of obstructions such as buildings, obstacles, vegetation and facilities, the detection targets often maneuver, dodge and hide behind them. How to quickly and accurately identify the detection targets in complex urban environments has become a research hotspot in recent years.

[0003] In urban environments, "transient targets" often lead to poor recognition and low detection accuracy. Targets usually move quickly and appear blurred, or hide behind obstructions. Although the detector recognizes the object in the frame, its confidence is very low due to the ambiguity and limited features of the object. For example, the confidence of recognizing a person is 0.11. Such targets exist in large numbers in urban environments, posing a huge challenge to the target detection task. In order to identify these blurred objects, humans first build an "object location map" based on the category information and location information of other clear objects in the image.

[0004] For example, if the confidences of the recognizable people are 0.78 and 0.62, and there is a clear car nearby with a confidence of 0.93, then the conclusion is that the blurry object is likely to be a "person", which indicates that the confidence is much higher than 0.11. Faced with the same complex scenes of blur and occlusion, humans can judge whether the results given by the detector are correct. This is because humans have the ability to reason with contextual information that current detection systems do not have.

[0005] Currently, deep learning-based object detection algorithms are mainly divided into two categories. The first category is two-stage algorithms, such as Faster R-CNN, Fast R-CNN, and R-CNN. Although these algorithms have high accuracy, they are slow due to large computational complexity. The second category is one-stage algorithms, such as SSD and YOLO, which provide faster detection speeds despite lower accuracy. In the demand for urban object recognition, since high real-time performance is required, one-stage algorithms are more popular due to their superior speed and wide application.

[0006] Current research often regards detecting objects in images as an independent task, ignoring the rich context and semantic relationships. To solve this problem, it is particularly important to enhance the detector by imitating human recognition. Existing technologies mainly adopt the following approaches: introducing a global and local attention mechanism (GAL) to combine local feature analysis with global context; proposing a context dependency mining network (CDMN) for low-cost feature construction; adding an additional shallow feature (ESF) mapping to enhance context; and there are also many studies that enhance the model's ability to aggregate information by adding context-aware modules.

[0007] We can see from the research progress that the potential of combining graph network structures with object detection can be achieved. This combination can not only model object relationships, but also utilize global and local context information to improve detection accuracy. Currently, most research focuses on object detection in general scenes. Although these methods perform well in daily environments, general models may not meet the accuracy requirements in specific application scenarios (such as dense cities).

[0008] In addition, most visual reasoning tasks adopt a two-stage detector because it divides the detection process into two stages: category detection and prediction box detection, which helps to incorporate common sense knowledge and context, thereby improving prediction results. However, a single-stage detector combines classification and detection regression into one stage, making similar improvements difficult. Summary of the invention

[0009] In order to solve the problem that it is difficult for a single-stage detector to combine classification and detection regression into one stage, thereby causing inaccurate results when performing target detection in a city, the present invention provides a target detection method for an urban environment, the method comprising the following steps:

[0010] S1. Use the improved YOLOv7 model to detect the image containing the target and obtain the target coordinates, target category and confidence level.

[0011] S2. Number each detected target to obtain a unique target number;

[0012] S3, setting a credible threshold and a fuzzy threshold, and dividing the target into a credible target and a fuzzy target according to the relationship between the confidence and the credible threshold and the fuzzy threshold;

[0013] S4. For each blurred target, compare its detection frame with the real target frame in the original data and calculate the intersection-over-union ratio;

[0014] S5. Calculate the true confidence of each fuzzy target according to the intersection-over-union ratio and the confidence;

[0015] S6, generating a node file, which includes a target unique number, target category, confidence and true confidence;

[0016] S7, target relationship calculation, according to the target category and target coordinates, calculate the spatial relationship between targets, including intersection, proximity and distance;

[0017] S8, generating an edge file to record the spatial relationship between the targets, wherein the distance relationship is not recorded;

[0018] S9. Input the node file and edge file into the improved SeHGNN network and obtain the final target detection result.

[0019] Furthermore, in improving the YOLOv7 model, the specific improvements to the YOLOv7 model include:

[0020] Improve the SPPCSPC module in the YOLOv7 model by deleting the two CBS modules before the Maxpool layer, replacing the remaining CBS module with a CBS module with a convolution kernel size of 1×1, changing the window size of the Maxpool layer from (5, 9, 13) to (5, 5, 5), and changing the SiLU function to the ReLU function;

[0021] Replace the upsample module in the YOLOv7 model with the CARAFE module;

[0022] Replace the 3×3 convolutional layer in the ELAN module of the YOLOv7 model backbone network with the DSConv layer;

[0023] Replace IoU in the YOLOv7 model with Shape-IoU.

[0024] Furthermore, the target coordinates are specifically the coordinates (x, y) of the center point of the detection frame and the width w and height h of the detection frame.

[0025] Furthermore, when the confidence is greater than or equal to the credible threshold, the target is a credible target, and when the confidence is less than the credible threshold and greater than or equal to the fuzzy threshold, the target is a fuzzy target.

[0026] Furthermore, the real target frame in the original data is a target frame obtained by manually annotating a picture containing the target.

[0027] Further, according to the intersection-over-union ratio and the confidence, the true confidence of each fuzzy target is calculated as follows:

[0028] If the intersection-and-union ratio is zero, then S = Penalty1 * d_score, where S represents the true confidence of each fuzzy target, Penalty1 represents the penalty coefficient applied when the intersection-and-union ratio is 0, d_score represents the confidence, and "*" represents the product;

[0029] If the target category does not match the true category, then S =Penalty2 * d_score, where Penalty2 represents the penalty coefficient imposed when the target category does not match the true category;

[0030] If the IoU is greater than the set IoU threshold and the target category matches the true category, then S = d_score + k * (IoU - threshold) , where k is a coefficient used to enhance the impact of the IoU when it is greater than the set IoU threshold; threshold is the set IoU threshold.

[0031] Furthermore, the real category is the category to which the target belongs obtained by manually classifying the picture containing the target.

[0032] Furthermore, the spatial relationship between the targets is calculated based on the target categories and target coordinates, and the spatial relationship includes intersection, proximity and distance. Specifically, when the detection frames represented by the two target coordinates have overlapping parts, the spatial relationship between the targets is considered to be intersection; when the detection frames represented by the two target coordinates do not have overlapping parts and the distance between the center points of the detection frames represented by the two target coordinates is less than a set distance threshold, the spatial relationship between the targets is considered to be proximity; in other cases, the spatial relationship between the targets is considered to be distance.

[0033] Furthermore, the specific improvement to the SeHGNN network is to make changes to the dataset loading code of the SeHGNN network, specifically:

[0034] For function definition: introduce the load_custom_dataset function, which is used to read and process custom datasets from the specified path;

[0035] For file paths: The paths of node and edge data files are constructed using the os.path.join() function;

[0036] For reading data: use the pd.read_csv function to read node and edge data, specify space as the delimiter, and assign custom column names;

[0037] For processing edges: create a reverse edge by swapping the source and target columns in the edge data, and connect it with the original edge data;

[0038] For tensor conversion: convert the test_conf feature to a PyTorch tensor, test_conf represents the confidence; use the torch.isnan() function to create a mask for real_conf to identify valid nodes, real_conf represents the true confidence;

[0039] For edge index conversion: convert edge index to tensor format to be compatible with PyTorch operations;

[0040] For data object creation: Create a data object, including:

[0041] x:test_conf feature;

[0042] edge_index: edge connection;

[0043] mask: mask indicating valid nodes based on real_conf;

[0044] node_id: the converted node ID tensor;

[0045] For prediction saving: Introduce a new function save_predictions to write the predictions of each node to the specified output file, with each line containing the node ID and its corresponding inference confidence.

[0046] The beneficial effects of the method of the present invention are:

[0047] Aiming at the difficulty of target detection in urban environment, the YOLOv7 model is improved to improve the detection efficiency of urban targets. By introducing SPPFPC structure, CARAFE structure and DSConv, the complexity of the model is reduced and lightweight. At the same time, the latest Shape-IoU method is used to optimize the accuracy of the regression box and significantly reduce the positioning loss of the model. These improvements make the model improve mAP50 by 1.1% on the PASCAL VOC2012 dataset, reduce the amount of calculation by 25.3%, and reach FPS of 156.6 compared with the original YOLOv7 model. Then, the detection task in the actual environment is simulated using urban pictures. The test results show that the detection effect of the detector on blurred objects is insufficient, mainly due to the low confidence caused by the blurred objects. In order to solve this problem, it is proposed to use the spatial position information between objects to recalculate the confidence. The confidence is recalculated by processing the data and improving the heterogeneous graph neural network architecture. The experiments carried out in the specific implementation method verify the feasibility and significant improvement of this concept. It shows that the method of the present invention can further improve mAP50 by 0.7% in urban images, with significant effect.

[0048] The method of the present invention can be applied in the field of target detection technology and the field of target tracking technology in complex urban environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is a comparison diagram of the SPPCSPC structure before and after improvement in the embodiment of the present invention;

[0050] Figure 2 It is a schematic diagram of an embodiment of the present invention when there is an overlap between the detection frames represented by two target coordinates;

[0051] Figure 3 This is a comparison diagram of the original SeHGNN network and the modified network in an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of the present invention.

[0053] Embodiment 1,

[0054] The present embodiment provides a method for object detection in an urban environment. Most visual reasoning tasks use a two-stage detector because it divides the detection process into two stages: category detection and prediction box detection, which helps to incorporate common sense knowledge and context, thereby improving the prediction effect. However, a single-stage detector combines classification and detection regression into one stage, making similar improvements difficult. To address this problem, the present embodiment focuses on the confidence results after detection. The detector generates confidence for each prediction as a sorting criterion. High-quality predictions should have higher confidence. Using confidence changes on a single-stage target detector can improve performance. Therefore, after the detector completes image recognition, it continues to reason based on its results to obtain a more accurate final result.

[0055] The design concept of the target detection method in this embodiment is specifically introduced as follows:

[0056] First, the YOLOv7 model is improved to reduce the complexity of the model and make it lightweight. The regression box is optimized using the latest Shape-IoU method, which effectively reduces the positioning loss of the model.

[0057] Secondly, after obtaining the target coordinates, target category and confidence by improving the YOLOv7 model, these data are processed. First, the detection results are divided into credible targets and fuzzy targets according to the confidence. For fuzzy targets, their confidence is recalculated to obtain the true confidence to more accurately reflect their actual situation. The spatial relationship information between targets is obtained according to the target coordinates, and all the information is organized into a training data set for building a heterogeneous graph neural network (SeHGNN network).

[0058] Finally, changes are made to the dataset loading code of the heterogeneous graph neural network, so that the modified heterogeneous graph neural network has enhanced capabilities in data reading, processing, and model training structure. The improved heterogeneous graph neural network learns through target categories, confidence, and spatial relationships between targets, and calculates an inferred confidence, named inference confidence. The inference confidence is compared with the true confidence, and the obtained loss value is used to judge the inference effect of the improved heterogeneous graph neural network, and then continuously adjusts itself and trains to achieve the optimal result.

[0059] The inference confidence obtained by the improved heterogeneous graph neural network is used to replace the confidence obtained in the improved YOLOv7 model, thereby overcoming the problem of low confidence caused by object blur in the detection process of the improved YOLOv7 model and improving the accuracy of target detection results.

[0060] The method comprises the following steps:

[0061] S1. Use the improved YOLOv7 model to detect the image containing the target and obtain the target coordinates, target category and confidence level. The target coordinates are the coordinates of the center point of the detection box (x, y) and the width w and height h of the detection box.

[0062] S2. Number each detected target to obtain a unique target number;

[0063] S3, setting a credible threshold and a fuzzy threshold, and dividing the target into a credible target and a fuzzy target according to the relationship between the confidence and the credible threshold and the fuzzy threshold;

[0064] S4. For each blurred target, compare its detection frame with the real target frame in the original data and calculate the intersection-over-union ratio;

[0065] S5. Calculate the true confidence of each fuzzy target according to the intersection-over-union ratio and the confidence;

[0066] S6, generating a node file, which includes a target unique number, target category, confidence and true confidence;

[0067] S7, target relationship calculation, according to the target category and target coordinates, calculate the spatial relationship between targets, including intersection, proximity and distance;

[0068] S8, generating an edge file to record the spatial relationship between the targets, wherein the distance relationship is not recorded;

[0069] S9. Input the node file and edge file into the improved SeHGNN network and obtain the final target detection result. (SeHGNN: Simple and Efficient Heterogeneous Graph Neural Network)

[0070] Embodiment 2,

[0071] This embodiment is a further limitation of Embodiment 1. The YOLOv7 model is improved to make it lightweight and improve the detection speed to meet the real-time requirements in urban environments. First, the SPPCSPC structure in YOLOv7 is improved: the convolution layer before the two pooling layers is cut off, and only a 1×1 convolution is used to change the number of channels; the pooling structure is changed, and the pooling window size is changed from the original (5, 9, 13) to three serial structure pooling layers with a window size of 5; and the original SiLU function is replaced by the ReLU function. The simplified design is called the SPPFPC structure, which can reduce the model complexity and improve the model detection accuracy compared to the SPPCSPC structure. Figure 1 The comparison of the structure before and after improvement is shown.

[0072] SPPCSPC is a module in the field of deep learning, especially in the convolutional neural network (CNN) architecture, which combines the concepts of Spatial Pyramid Pooling (SPP) and Cross Stage Partial Networks (CSPN).

[0073] The CARAFE structure is used to replace the upsample structure in the YOLOv7 structure, and a distribution shift convolution (DSConv) is used to replace the 3×3 convolution layer with a large computational overhead in the ELAN module in the YOLOv7 model backbone network; and a new IoU calculation method is introduced: Shape-IoU. Although the existing improved bounding box regression methods (GIoU, SIoU, CIoU, etc.) consider the influence of the distance, shape and angle of the GT box and the anchor box on the bounding box regression, they often ignore the fact that the shape and scale of the bounding box itself will also have an impact during the bounding box regression process. Shape-IoU can calculate the loss by focusing on the shape and scale of the bounding box itself, making the bounding box regression more accurate.

[0074] Embodiment 3,

[0075] This embodiment is a further limitation of Embodiment 1. Steps S2 to S8 are a process of processing data.

[0076] After we obtain data including target categories, target coordinates and confidence through target detection technology, we need to organize these data and use them to construct a training dataset for heterogeneous graph neural networks. In heterogeneous graph neural networks, we use these targets as nodes of the graph, and the edges between nodes are defined according to the spatial or logical relationship between the targets. The dataset constructed in this way will be used in the training process of the network, so that the model can learn and predict the corresponding attributes of new data.

[0077] Each detected target is numbered to obtain a unique target number.

[0078] A credible threshold and a fuzzy threshold are set, and the target is divided into a credible target and a fuzzy target according to the relationship between the confidence and the credible threshold and the fuzzy threshold; specifically, when the confidence is greater than or equal to the credible threshold, the target is a credible target, and when the confidence is less than the credible threshold and greater than or equal to the fuzzy threshold, the target is a fuzzy target.

[0079] For each blurred target, its detection frame is compared with the real target frame in the original data, and the intersection-over-union ratio is calculated; the real target frame in the original data is the target frame obtained by manually annotating the picture containing the target.

[0080] According to the intersection-over-union ratio and the confidence, the true confidence of each fuzzy target is calculated; specifically:

[0081] If the intersection-and-union ratio is zero, then S = Penalty1 * d_score, where S represents the true confidence of each fuzzy target, Penalty1 represents the penalty coefficient applied when the intersection-and-union ratio is 0, d_score represents the confidence, and "*" represents the product;

[0082] If the target category does not match the true category, S = Penalty2 * d_score, where Penalty2 represents the penalty coefficient imposed when the target category does not match the true category; the true category is the category to which the target belongs obtained by manually classifying the picture containing the target.

[0083] If the IoU is greater than the set IoU threshold and the target category matches the true category, then S = d_score + k * (IoU - threshold) , where k is a coefficient used to enhance the impact of the IoU when it is greater than the set IoU threshold; threshold is the set IoU threshold.

[0084] Generate a node file, which includes the target unique number, target category, confidence and true confidence;

[0085] Target relationship calculation, based on the target category and target coordinates, calculates the spatial relationship between targets. Spatial relationships include intersection, proximity, and distance. Specifically, when the detection boxes represented by the two target coordinates have overlapping parts, such as Figure 2 In the two cases shown, the spatial relationship between the targets is considered to be intersecting. When the detection frames represented by the two target coordinates do not overlap, and the distance between the center points of the detection frames represented by the two target coordinates is less than the set distance threshold, the spatial relationship between the targets is considered to be close. In other cases, the spatial relationship between the targets is considered to be far away.

[0086] A method for setting the distance threshold is given: Assume that the two targets are a person and a car, and the coordinates of the person are (x 1 ,y 1 , w 1 ,h 1 ), respectively, the coordinates of the center point of the human detection frame (x 1 ,y 1 ) and the width w of the human detection box 1 and high h 1 ; The coordinates of the car are (x 2 ,y 2 , w 2 ,h 2 ), respectively, the coordinates of the center point of the car detection frame (x 2 ,y 2 ) and the width w of the car detection box 2 and high h 2 ;

[0087] Set a multiple n, the distance threshold is n times w 1 , where n is set based on comparing the sizes of the two detection boxes and the target category.

[0088] Generate an edge file based on the calculated target relationship type. Generate an edge file to record the relationship between targets, including the number of target 1, the number of target 2, and the spatial relationship type between targets. Do not record distant relationships, that is, distant relationships do not exist as edges when constructing the graph.

[0089] Through the above series of detailed and precise processing steps, useful information can be effectively extracted from the YOLOv7 model target detection data. The heterogeneous graph neural network reads the node file (node.txt), in which each target is treated as a separate node. The attributes of the node include the unique number of the target, the target category, the confidence, and the possible true confidence (because the true confidence is only calculated when it is a fuzzy target). Next, the network reads the edge file (link.txt), which contains information about the edges based on the spatial relationship between the targets. These edges are associated by the number of the target nodes, clearly specifying which two nodes (targets) have a relationship and the type of this relationship (intersection, proximity). In this way, the heterogeneous graph neural network can construct a complete graph in which the nodes represent the detected targets and the edges represent the spatial relationship between these targets.

[0090] Embodiment 4,

[0091] This embodiment is a further limitation of Embodiment 1. Improving the graph neural network refers to making changes to the data set loading code of the SeHGNN network, specifically:

[0092] (1) Function definition:

[0093] Introduced the load_custom_dataset function, which is used to read and process custom datasets from the specified paths (node.txt and link.txt);

[0094] (2) File path:

[0095] The paths to the node and edge data files are constructed using the os.path.join() function;

[0096] (3) Read data:

[0097] Use the pd.read_csv function to read node and edge data, specify space as the delimiter, and assign custom column names;

[0098] Node data: ['node_id', 'type', 'test_conf', 'real_conf']

[0099] Edge data: ['source', 'target', 'edge_type']

[0100] (4) Processing edges:

[0101] Create a reverse edge by swapping the source and target columns in the edge data and connect it with the original edge data;

[0102] (5) Tensor conversion:

[0103] Convert the test_conf feature to a PyTorch tensor, test_conf represents the confidence; use the torch.isnan() function to create a mask for real_conf to identify valid nodes, real_conf represents the true confidence;

[0104] (6) Edge index conversion:

[0105] Convert edge indices to a tensor format compatible with PyTorch operations, using source and target columns.

[0106] (7) Data object creation:

[0107] Create a data object consisting of:

[0108] x:test_conf feature;

[0109] edge_index: edge connection;

[0110] mask: mask indicating valid nodes based on real_conf;

[0111] node_id: the converted node ID tensor;

[0112] (8) Delete unused variables to simplify the code;

[0113] (9) Mean square error calculation:

[0114] Adjusted the compute_mse function to correctly handle the shape of the output tensor and ensure compatibility with masks;

[0115] (10) Prediction preservation:

[0116] A new function save_predictions is introduced to write the predictions of each node to the specified output file, with each line containing the node ID and its corresponding prediction.

[0117] The above changes are to change the way the network reads data. For node classification tasks, the cross entropy loss function is usually used, and the labels are one-hot encoded. However, for regression tasks, a continuous value needs to be predicted, so the MSE (mean square error) loss function is selected to measure the difference between the predicted value and the true value. After each training cycle, the model is evaluated and the current model state and node data are saved. The purpose of this is to be able to use the optimal model for prediction after training, ensuring the accuracy and stability of the model in practical applications.

[0118] In terms of data reading, the original data is properly processed to make it more suitable for regression tasks. Specifically, a dataset containing node features (such as test_conf) and true confidence (real_conf) is loaded and converted into a format suitable for model input. In addition, a mask mechanism is introduced in the model so that only those blurred targets are considered when calculating test_conf during training.

[0119] The Chinese name of SeHGNN is simple and efficient heterogeneous graph neural network, which mainly consists of three parts: neighbor aggregation, feature projection and transformer-based semantic fusion. Figure 3 This is the comparison between the original SeHGNN network and our modified network. Figure 3 In the figure, Customized SeHGNN represents the improved SeHGNN network. From the comparison in the figure, we can see that in the neighbor aggregation step, the original SeHGNN network only targets nodes of one category P (Target node), while the improved SeHGNN network expands the category to three categories P, C, and G (mask) in order to more comprehensively construct the dependency relationship between objects, and adjust the complexity of the model by separately controlling the number of meta-path hops of each category.

[0120] The Transformer module of SeHGNN is relatively complex and includes a multi-head attention mechanism. Figure 3The attention is calculated in parallel (represented by H in the figure), and concatenation and linear transformation are performed after calculation. And its residual connection and dropout mechanism are also more complex and detailed in implementation. Considering that the task of the method in this embodiment is a regression task, there are fewer node types, the relationship between nodes is not complicated, and the node features mainly have only one test_conf (confidence), semantic fusion is simplified for this purpose. The Transformer module with a single-head attention mechanism is used to reduce the computational complexity and the number of model parameters, thereby shortening the training time and reducing resource consumption. Secondly, it is optimized for regression tasks. Through feature projection and a simple attention mechanism, it focuses on the test_conf attribute of the node, which makes the model more efficient when processing regression tasks. In the case of limited data, the simplified model structure also reduces the risk of overfitting.

[0121] Embodiment 5,

[0122] This embodiment further limits the embodiment 1 and introduces the beneficial effects of the method with specific experimental data.

[0123] Comparison of the improved YOLOv7 model with other target detection networks: The operating system is Ubuntu20.04 server, the graphics card is Nvdia 3090, CUDA11.6, and the operating environment is built using Python language based on the Pytorch framework.

[0124] In order to comprehensively evaluate the improved YOLOv7 model, experiments were conducted on the PASCAL VOC2012 dataset and the self-built dataset MILITARGETS dataset, and compared with the current advanced target detectors. The results are shown in Table 1, where the accuracy P, recall R, average precision mAP50, frame rate FPS, parameter amount Params and floating-point operation number GFLOPs are used to evaluate the model performance.

[0125] Table 1:

[0126]

[0127] In the PASCAL VOC2012 dataset, compared with other object detection models, including the most advanced yolov9 and yolov10, the improved YOLOv7 model (denoted by Ours in the table) has the best recall and mAP50. Compared with other models, mAP50 is 1.1%~11.6% higher. At the same time, the improved YOLOv7 model has the highest frame rate of 156.6 while keeping the computational complexity to a minimum and reducing the number of parameters. This highlights that the improved YOLOv7 model has higher accuracy and lighter weight. In the MILITARGETS dataset, compared with other models, the recall and mAP50 of the improved YOLOv7 model are still at the best level. The recall rate is 3.5%~19.3% higher than other models, and the mAP50 is 1.7%~16.9% higher, and the computational complexity is the lowest and the frame rate is still the highest at 137. It can be seen that the improved SPPFPC module and the introduced Shape-IoU, DSConv, and CARAFE methods are also lightweight and effective on the MILITARGETS dataset.

[0128] Through the analysis of comparative experiments on the PASCAL VOC2012 dataset and the MILITARGETS dataset, we clearly observe that the performance of the improved YOLOv7 model has been significantly improved. The wide application and different characteristics of these two datasets have verified the improvements in a wider range of scenarios, further consolidating the effectiveness and reliability of the improvement measures. Among the current popular detectors, the improved YOLOv7 model performs outstandingly, showing higher accuracy and stronger robustness. However, since the targets in the city pictures selected in the experiment are relatively blurred, it brings great obstacles to the detection task. Table 2 uses the trained yolov7 model and the improved YOLOv7 model to test the training set pictures in the city. It can be seen that even though the model has been optimized to a great extent in the above datasets, the mAP50 of the model on the MILITARGETS dataset can reach 84.1% and 85.8% respectively. However, when detecting city pictures, the mAP50 is still only 46.8 and 48.5. Due to the small number of samples and the relatively blurred targets, which are easily interfered with the surrounding environment, the recall rate (R) of the target category Gun is very low.

[0129] Table 2:

[0130]

[0131] In Table 2, the detection accuracy and recall are not ideal due to the ambiguity of the target. To this end, a series of parameter tuning experiments were conducted to re-score the confidence using only the information of the predicted coordinate box and the predicted score. The goal is to adjust the confidence score to a more appropriate level to ensure that subsequent reasoning on the heterogeneous graph can be based on a more accurate target value, thereby improving the effect of the entire detection system.

[0132] After completing the confidence re-scoring, the model evaluation steps in the original YOLOv7 model are followed to re-enter the adjusted confidence into the model. Doing so allows us to obtain updated data for various performance indicators, including precision, recall, and average precision. In this way, we can intuitively see the specific impact of re-scoring on model performance, thereby verifying whether our adjustments are effective and further optimizing the model to cope with complex detection environments. Table 3 shows the values ​​of each parameter and the improved performance based on the detection results of the yolov7 model and the improved YOLOv7 model, where Conf1 represents the credible threshold, Conf2 represents the fuzzy threshold, Penalty1 represents the penalty coefficient applied when the intersection-and-union ratio is 0, k is a coefficient used to enhance the influence of the intersection-and-union ratio when it is greater than the set intersection-and-union ratio threshold, threshold is the set intersection-and-union ratio threshold, and Penalty2 represents the penalty coefficient applied when the target category does not match the true category.

[0133] Table 3:

[0134]

[0135] In Table 3, when the credible threshold is set to 0.75 and the fuzzy threshold is set to 0.15, the performance of the YOLOv7 model and the improved YOLOv7 model is significantly improved by recalculating the confidence. Specifically, the best mAP50 of YOLOv7 is increased from 46.8% to 52.3%, while the improved YOLOv7 model is increased from 48.5% to 57.3%. This result not only demonstrates the effectiveness of parameter settings, but also verifies the preliminary feasibility and practical value of the confidence recalculation strategy. Table 4 shows the detection results of various target categories at the best effect. It is worth noting that compared with Table 2, it can be found that after recalculation, although the recall rate of the gun and truck categories has not changed much, the accuracy has shown a significant improvement. For further exploration, the threshold range is expanded to 1 to 0.01, which covers almost all detection results. Under this setting, the mAP50 of the YOLOv7 model and the improved YOLOv7 model are improved by 12.9 and 16.6 percentage points respectively.

[0136] Table 4:

[0137]

[0138] In the previous step, we recalculated the confidence of the object and obtained a more appropriate set of confidence values. These updated values ​​are input into the heterogeneous graph neural network and used as target values. At this point, a graph relationship consisting of clear objects and blurred objects is formed in the graph neural network. Using the powerful node-to-node relationship capture capability of the graph neural network, the confidence of the blurred object is inferred and calculated to simulate the process of human inference based on the category and position relationship of the object. A graph dataset suitable for heterogeneous graph networks is constructed using city images, including 4485 nodes and 11260 edges. The heterogeneous graph reasoning results are shown in Table 5 through training with the training set and validation set.

[0139] Table 5:

[0140]

[0141] According to the data in Table 5, the mAP50 of the improved yolov7 model is 46.7 after testing. On this basis, recalculating the confidence can increase the mAP50 by 7.4. After the inference processing of the heterogeneous graph neural network, it can be seen that the mAP50 has increased by 0.7, the accuracy has increased, but the recall has decreased. The accuracy of most categories has increased significantly, among which the accuracy of the gun and people categories has increased significantly.

[0142] The experiment believes that reasoning through spatial position relationships may provide more contextual information, which helps to filter out some misdetected objects or objects with low confidence, thereby improving the accuracy. However, due to the stricter screening of objects, the recall rate has decreased. The results show that our method can effectively filter out some misdetected objects, thereby improving the accuracy of the system and the quality of the detection results. In order to avoid insufficient samples due to too many node types and relationship types when inputting heterogeneous graph neural networks, the three categories of car, truck, and chariot are merged into one category of all. The improvement has shown that the current model has shown good performance in processing complex urban battlefield data. Especially in terms of the model's ability to capture complex relationships between nodes, it has been able to effectively simulate and infer the confidence of fuzzy targets, showing the effectiveness and adaptability of the model. This proves that the method of the present invention is feasible and competitive in meeting the challenges of actual application scenarios.

Claims

1. A method for target detection in an urban environment, characterized in that: The method comprises the following steps: S1. Use the improved YOLOv7 model to detect the image containing the target and obtain the target coordinates, target category and confidence level. S2. Number each detected target to obtain a unique target number; S3, setting a credible threshold and a fuzzy threshold, and dividing the target into a credible target and a fuzzy target according to the relationship between the confidence and the credible threshold and the fuzzy threshold. When the confidence is greater than or equal to the credible threshold, the target is a credible target, and when the confidence is less than the credible threshold and greater than or equal to the fuzzy threshold, the target is a fuzzy target; S4. For each blurred target, compare its detection frame with the real target frame in the original data and calculate the intersection-over-union ratio; S5. Calculate the true confidence of each fuzzy target according to the intersection-over-union ratio and the confidence; S6, generating a node file, which includes a target unique number, target category, confidence and true confidence; S7, target relationship calculation, according to the target category and target coordinates, calculate the spatial relationship between targets, including intersection, proximity and distance; S8, generating an edge file to record the spatial relationship between the targets, wherein the distance relationship is not recorded; S9. Input the node file and the edge file into the improved SeHGNN network. The improved SeHGNN network calculates the reasoning confidence through the node file and the edge file, and uses the reasoning confidence to replace the confidence obtained in step S1, thereby obtaining the final target detection result.

2. The target detection method for an urban environment according to claim 1, characterized in that: In improving the YOLOv7 model, the specific improvements to the YOLOv7 model include: Improve the SPPCSPC module in the YOLOv7 model by deleting the two CBS modules before the Maxpool layer, replacing the remaining CBS module with a CBS module with a convolution kernel size of 1×1, changing the window size of the Maxpool layer from (5, 9, 13) to (5, 5, 5), and changing the SiLU function to the ReLU function; Replace the upsample module in the YOLOv7 model with the CARAFE module; Replace the 3×3 convolutional layer in the ELAN module of the YOLOv7 model backbone network with a DSConv layer; Replace IoU in the YOLOv7 model with Shape-IoU.

3. The target detection method for an urban environment according to claim 2, characterized in that: The target coordinates are specifically the coordinates (x, y) of the center point of the detection frame and the width w and height h of the detection frame.

4. The target detection method for an urban environment according to claim 3, characterized in that: The real target frame in the original data is the target frame obtained by manually annotating the picture containing the target.

5. The target detection method for an urban environment according to claim 4, characterized in that: According to the intersection-over-union ratio and confidence, the true confidence of each fuzzy target is calculated as follows: If the intersection-and-union ratio is zero, then S = Penalty1 * d_score, where S represents the true confidence of each fuzzy target, Penalty1 represents the penalty coefficient applied when the intersection-and-union ratio is 0, d_score represents the confidence, and "*" represents the product; If the target category does not match the true category, then S =Penalty2 * d_score, where Penalty2 represents the penalty coefficient imposed when the target category does not match the true category; If the IoU is greater than the set IoU threshold and the target category matches the true category, then S = d_score + k* (IoU - threshold) , where k is a coefficient used to enhance the effect of the IoU when it is greater than the set IoU threshold; threshold is the set IoU threshold.

6. The target detection method for an urban environment according to claim 5, characterized in that: The real category is the category to which the target belongs obtained by manually classifying the picture containing the target.

7. The target detection method for an urban environment according to claim 6, characterized in that: The spatial relationship between the targets is calculated based on the target categories and target coordinates. The spatial relationship includes intersection, proximity and distance. Specifically, when the detection frames represented by the two target coordinates have overlapping parts, the spatial relationship between the targets is considered to be intersection; when the detection frames represented by the two target coordinates do not have overlapping parts and the distance between the center points of the detection frames represented by the two target coordinates is less than the set distance threshold, the spatial relationship between the targets is considered to be proximity; in other cases, the spatial relationship between the targets is considered to be distance.

8. The target detection method for an urban environment according to claim 7, characterized in that: The specific improvements to the SeHGNN network are changes to the dataset loading code of the SeHGNN network, specifically: For function definition: introduce the load_custom_dataset function, which is used to read and process custom datasets from the specified path; For file paths: The paths of node and edge data files are constructed using the os.path.join() function; For reading data: use the pd.read_csv function to read node and edge data, specify space as the delimiter, and assign custom column names; For processing edges: create a reverse edge by swapping the source and target columns in the edge data, and connect it with the original edge data; For tensor conversion: convert the test_conf feature to a PyTorch tensor, test_conf represents the confidence; use the torch.isnan() function to create a mask for real_conf to identify valid nodes, real_conf represents the true confidence; For edge index conversion: convert edge index to tensor format to be compatible with PyTorch operations; For data object creation: Create a data object, including: x:test_conf feature; edge_index: edge connection; mask: mask indicating valid nodes based on real_conf; node_id: the converted node ID tensor; For prediction saving: Introduce a new function save_predictions to write the predictions of each node to the specified output file, with each line containing the node ID and its corresponding inference confidence.

Images