Target detection method based on dense feature point splicing and improved YOLOV 5

Abstract

The application discloses a target detection method based on dense feature point splicing and an improved YOLOV5, and the method comprises the following steps: acquiring multiple images of a target area through the adjacent installation and adjustment of multiple cameras; wherein the multiple images are adjacent and have overlaps; splicing the multiple images based on the overlapping areas; and performing target detection on the spliced images. The application uses a dense area matching mode, fully utilizes global feature information for matching, fully utilizes background information of unmanned aerial vehicle detection for network learning, avoids the fragmentation of the target caused by block detection without affecting real-time performance, and can prevent obvious distortion of the picture after splicing and improve the accuracy of target detection.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and in particular to a target detection method based on dense feature point stitching and an improved YOLOv5. Background Technology

[0002] With the rapid development of the drone industry, drone applications are emerging like mushrooms after rain, ranging from aerial photography to logistics, and from individuals to government agencies. How to uniformly manage drones is crucial for the further popularization and development of the drone industry, as well as airspace safety, especially the low-altitude airspace safety in cities and key areas. Effective drone management requires solving the problem of efficient drone detection, and ultra-wide-angle drone detection offers a practical and feasible technical contribution to drone management.

[0003] Currently, most drone detection is aimed at non-ultra-wide-angle detection, which requires the simultaneous deployment of multiple monitoring points to detect drones in a certain direction of airspace. This is not only uneconomical, but also greatly affected by equipment errors in the later stages, and the accuracy requirements for multi-point target detection data fusion are also high.

[0004] Drone detection based on ultra-wide-angle camera imaging largely relies on a single ultra-wide-angle lens. Ultra-wide-angle lenses suffer from severe distortion, especially near the edges of the field of view. This distortion significantly interferes with target localization, and distortion correction is a complex and resource-intensive process. Due to the optical characteristics of the lens, the correction effect is often unsatisfactory, and post-processing is often required to crop the edge areas of the image to reduce the field of view.

[0005] Using ordinary lenses to stitch together an ultra-wide-angle lens is a feasible and economical solution. Currently, most technologies use feature point matching. However, in drone detection, the background of drones is mostly a sky area with no texture or weak texture. Using sparse feature point matching will result in a large error.

[0006] Ultra-wide-angle drone detection falls under the category of small target detection. Currently, most small target detection methods obtain information about small targets by improving the methods for extracting their features. Experiments have shown frequent missed detections in ultra-wide-angle drone detection because resizing the ultra-wide-angle image to 1280×1280 or 640×640 (common input resolutions in YOLOv5) further compresses the drone, which is already a small target, resulting in feature loss. Therefore, it is possible to divide the image into blocks to transform the small target into a larger target before detection.

[0007] Existing block detection techniques typically employ overlapping blocks to avoid splitting a target into two. However, this method requires calculating a large number of image sub-blocks, making it unsuitable for real-time target detection in ultra-wide-angle videos. Summary of the Invention

[0008] In view of this, the present invention provides a target detection method based on dense feature point stitching and an improved YOLOv 5 to solve the above-mentioned technical problems.

[0009] This invention discloses a target detection method based on dense feature point stitching and an improved YOLOv5, comprising the following steps:

[0010] Step 1: Acquire multiple images of the target area by installing and adjusting multiple cameras adjacent to each other; wherein, the multiple images overlap with each other.

[0011] Step 2: Based on the overlapping areas, stitch the multiple images together;

[0012] Step 3: Perform target detection on the stitched image.

[0013] Further, step 1 includes:

[0014] By installing and adjusting three cameras adjacent to each other, three images of the target area are acquired; wherein, the three images overlap with each other, that is, there are four overlapping areas.

[0015] The three cameras, when stitched together, have a field of view greater than 180°, and all three cameras have the same parameters. Frame synchronization and image quality alignment have been performed. The four overlapping regions each account for 10% to 20% of the FOV of their respective individual cameras.

[0016] Further, step 2 includes:

[0017] Step 21: Using the histogram matching method, with the image at the middle position of the three images as the reference, the images on both sides are matched towards the image at the middle position;

[0018] Step 22: Use the FAST feature detector to detect feature points in each overlapping region; sort the feature points in each overlapping region from largest to smallest response value, and retain the feature points corresponding to the top 5 largest response values ​​to form a sparse feature set; if there are fewer than 5 response values, retain all of them; where each feature point corresponds to one response.

[0019] Step 23: Calculate the optical flow of the sparse feature set using the Lucas-Kanade method. Use the image pyramid to match feature points in each feature dimension to obtain matching feature points in four overlapping regions. Then calculate the optical flow of the matching feature points. Each optical flow value is a two-dimensional vector representing the amount of motion in the horizontal and vertical directions.

[0020] Step 24: Based on the sparse feature set and matching feature points, obtain the optical flow value of each grid after subdividing each overlapping region into 15×3 grids;

[0021] Step 25: Stitch the three images together based on the optical flow value of each grid.

[0022] Further, step 25 includes:

[0023] Step 251: The optical flow value of the top left vertex of each grid is equal to the median smoothed optical flow value in the neighborhood of its 2×2 grid vertex; then, with each grid vertex as the center, median smoothing is performed in the 4×4 vertex neighborhood to obtain the optical flow value of each grid vertex.

[0024] Step 252: Subdivide each grid into N×N cells, and obtain the optical flow value of the vertex of each cell using the triangle interpolation method; take the optical flow value of the top left vertex of each cell as the optical flow value of all pixels in the cell, thereby obtaining the optical flow value of all pixels in the overlapping area, and then use the optical flow value of the overlapping area to stitch the three images together.

[0025] Further, step 252 includes:

[0026] Step 2521: N×N cells form a quadrilateral. Given the optical flow values ​​of the four vertices P1, P2, P3, and P4 of the quadrilateral, obtain the optical flow value of each cell vertex using triangular interpolation. The optical flow value P of point P in the upper triangle is... v Optical flow values ​​P corresponding to P1, P2, and P3 1v P 2v P 3v Interpolated from:

[0027] P v =(w1P 1v +w2P 2v +w3P 3v ) / (w1+w2+w3)

[0028]

[0029] w i =1 / D i

[0030] in, Representing the i-th vertex P respectively i x, y coordinates, P x P y Let D represent the x and y coordinates of point P in the triangle to be solved. i Indicates P to P i The distance between them, weight w i With D i Inversely proportional;

[0031] Step 2522: Following the same logic as step 2521, the optical flow values ​​P corresponding to P2, P3, and P4 are... 2v P 3v P 4v Interpolation is used to obtain the optical flow value of the vertex of the cell located in the lower triangle.

[0032] Further, step 3 includes:

[0033] Based on the fact that the target is likely to appear in the air region in the upper half of the image, the upper half of the image is cropped, and the size of the region is adjusted to be the same as the size of the original image.

[0034] The adjusted image and the original stitched image are used as training samples and input into the YOLOv5 network for learning.

[0035] Extract the first feature map corresponding to the adjusted image and the original stitched image, and stitch the first feature map to obtain a feature map group. Then extract the second feature map of the feature map group.

[0036] The second feature map is divided into multiple blocks for parallel detection. Then, the position of the detected target is correlated with its position in the original stitched image, and the final detection result is output.

[0037] Furthermore, when using image block detection, if a target spans two adjacent blocks, it is detected whether there are any connected target boxes in the adjacent blocks. If they are connected, then the two targets belong to the same target. In the final ultra-wide-angle image, the smallest bounding rectangle of the two connected target boxes is taken as its final output target box.

[0038] Furthermore, when using image block detection, when two targets are adjacent, new labels are created to check whether the targets are complete before detection.

[0039] The new label format is: [cls,x,y,w,h,c], where cls is the class, x and y are the coordinates of the target, w and h are the width and height of the target, respectively, and c is the target completeness, with a value of 0 or 1; 0 indicates an incomplete target, and 1 indicates a complete target. The sample set is labeled according to the new labels and then trained. The new loss function increases the completeness factor.

[0040] Loss(all)=Loss(confidence)+Loss(location)+Loss(class)+Loss(completeness)

[0041]

[0042] Where Loss(confidence) represents confidence loss, Loss(location) represents location loss, Loss(class) represents class loss, and Loss(completeness) represents completeness loss. 2 B represents the number of grid cells, and B represents the number of anchor boxes in each grid cell. c indicates the existence of a target. i It is the detected c value. It is the labeled truth value, λ comp It is a weighting coefficient that measures the loss of integrity.

[0043] Furthermore, iterate through the target bounding boxes near the dividing line between the two targets, convert the coordinate formats of the top left and bottom right corners of the target bounding boxes to (x1, y1) and (x2, y2) respectively, and then iterate through the coordinates within 10 pixels near the dividing line to check whether the coordinates on both sides of the dividing line are adjacent.

[0044] Furthermore, if the coordinates on both sides of the dividing line are not adjacent, the target box is directly output; if the coordinates on both sides of the dividing line are adjacent, it is further determined whether the two adjacent target boxes belong to the same target or belong to two different targets, which is determined by checking whether the target is complete in the label.

[0045] If the labels of two adjacent target boxes are both 0, it means that the targets on both sides of the dashed line are incomplete. In this case, the minimum bounding rectangle of the two adjacent target boxes is calculated and output as the merged target box. If they are not both 0, the two adjacent target boxes are output directly.

[0046] Due to the adoption of the above technical solutions, the present invention has the following advantages: (1) The present invention proposes a method for stitching ordinary lenses into ultra-wide-angle lenses. It uses dense optical flow estimation to register overlapping areas, which can overcome the adverse effects of the lack of matching feature points in the sky. It integrates the global features of the overlapping areas to stitch the images, preventing obvious distortion of the image after stitching; (2) The present invention makes full use of the special characteristics of the target background in the task, and incorporates more areas where the target appears in the sky into the network learning, thereby improving the detection accuracy; (3) The present invention uses a block detection method to innovatively propose new labels to judge the integrity of objects, and then combines whether the target boxes near the dividing line are connected to avoid the same target being divided into two target outputs. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments recorded in the embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0048] Figure 1 This is a schematic diagram of image stitching according to an embodiment of the present invention;

[0049] Figure 2 This is a schematic diagram of the grid in the overlapping area of ​​the image according to an embodiment of the present invention;

[0050] Figure 3 This is a schematic diagram of the dense optical flow estimation process according to an embodiment of the present invention;

[0051] Figure 4 This is a schematic diagram of the probability statistics of the target occurrence area in an embodiment of the present invention;

[0052] Figure 5 This is a schematic diagram illustrating the additional sky region capture and network learning in an embodiment of the present invention.

[0053] Figure 6 This is a schematic diagram of the target on the dividing line in an embodiment of the present invention;

[0054] Figure 7 This is a schematic diagram illustrating how two targets in an embodiment of the present invention are mistakenly identified as a single target.

[0055] Figure 8 This is a schematic diagram of incomplete target detection according to an embodiment of the present invention;

[0056] Figure 9 This is a flowchart illustrating a target detection method based on dense feature point stitching and an improved YOLOv 5 according to an embodiment of the present invention. Detailed Implementation

[0057] The present invention will be further described in conjunction with the accompanying drawings and embodiments. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art should fall within the protection scope of the present invention.

[0058] This invention addresses the unique characteristics of ultra-wide-angle drone detection: a large required field of view (FOV) and a lack of features in the sky. Therefore, dense optical flow estimation is employed for image registration and stitching. To further address the characteristics of drone-prone areas, the input to the YOLOv5 network is improved, incorporating more background information into the network's learning process. An innovative improvement to the label format allows for the detection of target integrity, and combining this with segmentation results avoids the indiscriminate splitting of targets during block detection.

[0059] See Figure 9 This invention provides an embodiment of a target detection method based on dense feature point stitching and an improved YOLOv5, which includes the following steps:

[0060] S1: Multiple images of the target area are acquired by installing and adjusting multiple cameras adjacent to each other; among them, there is overlap between adjacent images.

[0061] See Figure 1 You can use three identical camera modules to stitch together a picture with an ultra-wide field of view. For example, if you choose a single module with a field of view between 65° and 75°, the stitched image will have a field of view greater than 180°. When installing the camera modules, ensure that the three lenses are as perpendicular to the horizontal ground as possible, that the camera lenses are on the same horizontal line, and that the FOVs of the three cameras overlap. Figure 1 As shown.

[0062] Prerequisites for image stitching: The three modules have the same parameters and have already performed frame synchronization and image quality alignment (including optimization of noise, AWB, AE, Lenshading, AF, sharpening, etc. in the camera's ISP to ensure that the images generated by the three devices have the same quality).

[0063] The overlapping area accounts for approximately 10% to 20% of the FOV of a single camera, but not less than 10%, to ensure that there are enough feature points for matching.

[0064] The target area can be the area where the drone appears.

[0065] S2: Based on overlapping regions, stitch together multiple images.

[0066] When stitching images, multiple images can be selected for stitching. Here are three methods for image stitching: the middle image remains stationary as a reference frame; the left and right images are simultaneously aligned and stitched with the left and right edge regions of the middle image, respectively; that is, regions 1 and 2 are matched and aligned, and regions 3 and 4 are matched and aligned. The overlapping area is divided into a 15×3 grid, resulting in 45 cells. Figure 2 As shown, regions 1, 2, 3, and 4 are overlapping areas of the three images, meaning there are four overlapping areas among the three images.

[0067] The image stitching module includes histogram matching, sparse feature point detection, sparse feature matching, dense optical flow estimation, and image registration.

[0068] Histogram matching: Histogram matching uses the middle image as a reference, and the images on both sides are matched towards the middle image. This step prevents inconsistencies in brightness and color among the three frames, which helps to eliminate visible color differences after stitching and facilitates feature matching during stitching.

[0069] Sparse Feature Point Detection: The FAST feature detector is used to detect feature points in each cell of regions 2 and 3. Each feature point corresponds to a response. The larger the response value, the more obvious the feature point is, and the more likely it is to be a feature point. Then, the feature points in each cell are sorted from largest to smallest response value, and the top 5 feature points with the largest response value are retained. All feature points with fewer than 5 responses are retained.

[0070] Sparse Feature Matching: The Lucas-Kanade method is used to calculate the optical flow of the sparse feature set. Each optical flow value is a two-dimensional vector representing the amount of motion in the horizontal and vertical directions. Feature points are matched and optical flow is calculated using an image pyramid across each feature dimension. This step calculates the matching feature points between regions 1 and 2, and between regions 4 and 3.

[0071] Dense optical flow estimation: Next, optical flow filling is performed, such as... Figure 3 As shown in Figure A, the numbers in the cells represent the number of detected and retained feature points and the corresponding number of optical flow value pairs. The optical flow of each vertex is equal to the median of the optical flow values ​​in its 2×2 cell neighborhood plus median smoothing. Then, median smoothing is performed in a 4×4 vertex neighborhood centered on each vertex, resulting in a smoothed optical flow value for each vertex.

[0072] Each cell is further subdivided into a 3×3 grid, as shown in Figure B. The optical flow at each vertex of the subdivided grid is obtained using triangular interpolation, as shown in Figure C. The optical flow values ​​corresponding to P1, P2, P3, and P4 are known, and the optical flow value of point P in the upper triangle is P0. v Optical flow values ​​P corresponding to P1, P2, and P3 1v P 2v P 3v Interpolated from:

[0073] P v =(w1P 1v +w2P 2v +w3P 3v ) / (w1+w2+w3)

[0074]

[0075] w i =1 / D i

[0076] in, Representing the i-th vertex P respectively i x, y coordinates, P x P yLet represent the x and y coordinates of point P in the triangle to be determined. (D) i Indicates P to P i The distance between them, weight w i With D i Inversely proportional.

[0077] Similarly, using the method described above, the optical flow values ​​P corresponding to P2, P3, and P4 can be obtained. 2v P 3v P 4v Interpolation is used to obtain the optical flow value of the vertex of the cell located in the lower triangle.

[0078] The optical flow value of each subdivided vertex is filled into the subdivided cell in its lower right corner (D diagram, different colors represent different values, this is for illustrative purposes only). At this point, each pixel in the cell is assigned an optical flow value. Thus, feature-based sparse matching evolves into dense pixel matching.

[0079] All the above steps are performed offline. After stitching is completed, during real-time target detection, to prevent stitching misalignment caused by factors such as device jitter, the displacement of sparse matching feature points in overlapping areas is detected during real-time operation. When the cumulative displacement of more than 50% of the sparse feature points exceeds 0.5 pixels, the average displacement of the sparse matching feature points is calculated, and the entire image is shifted in the opposite direction by the corresponding displacement.

[0080] S3: Perform target detection on the stitched image.

[0081] Object detection is performed using yolov5-tiny. For this application scenario, two innovative improvements are proposed: feature extraction and block detection.

[0082] Feature extraction: Since the background of drones is very likely to be the sky, more drone areas with the sky as the background can be added to the feature extraction to increase the contribution of the sky background to target detection.

[0083] like Figure 4 As shown, the probability of a drone appearing in the upper half of an image is much higher than that in the lower half in the statistical sample set. Therefore, at the input end, the upper half of the image can be cropped and resized to the original image size. Figure 1 It serves as input for the network to learn. Figure 5This refers to the input of the YOLOv5 network and the three hidden layers connecting it. The input image is resized to 640×640×3, the upper half is cropped, grayscale is applied, and the same operation is performed on the original image to obtain a 320×320×64 feature map. These feature maps are then combined to obtain a 320×320×128 feature map group, which is then used for the next feature extraction operation. The original network, following the direction of the dashed arrow, directly extracts features from the original image to obtain a 320×320×64 feature map, and then further extracts features to obtain a 160×160×128 feature map.

[0084] Block detection: Since the stitched image is large and the target is relatively small compared to the image, YOLOv5 input requires image compression. However, compression results in the loss of more target details, hindering detection. Therefore, this method divides the image into four blocks for parallel detection, then maps the detected target positions to their corresponding positions in the original image, and finally outputs the final detection result.

[0085] When using image patch detection, if a target spans two adjacent patches, the merged detection results may generate two bounding boxes for one target, such as... Figure 6 As shown.

[0086] Therefore, it can detect whether adjacent blocks have connected bounding boxes (i.e., the edges of the bounding boxes are close together but do not overlap). If they are connected and both targets belong to the same target, the smallest bounding rectangle of the two connected bounding boxes is taken as the final output bounding box in the final ultra-wide-angle image, thus avoiding the detection of the same target as two separate targets. To enhance robustness, a redundancy of ±2 pixels can be added to the determination of the edges of the left and right bounding boxes being connected.

[0087] If only the above method is used, when two targets are adjacent, they will ultimately be detected as one target, such as... Figure 7 As shown.

[0088] This issue can be avoided by creating special labels to check for target incompleteness. The official YoloV5 label format is [cls,x,y,w,h]. Now, we add a dimension [cls,x,y,w,h,c], where c takes the value 0 or 1. 0 indicates an incomplete target, and 1 indicates a complete target. Label the sample set according to the new labels and then learn the new system. To improve efficiency, the steps for detecting incomplete targets are as follows: Figure 8 As shown. The new loss function adds an integrity term; the rest is expressed as the loss function in Ultralytics' YOLOv5 documentation.

[0089] Loss(all)=Loss(confidence)+Loss(location)+Loss(class)+Loss(completeness)

[0090]

[0091] Where Loss(confidence) represents confidence loss, Loss(location) represents location loss, Loss(class) represents class loss, and Loss(completeness) represents completeness loss. 2 B represents the number of grid cells, and B represents the number of anchor boxes in each grid cell. c indicates the existence of a target. i It is the detected c value. It is the labeled truth value, λ comp It is a weighting coefficient that measures the loss of integrity.

[0092] Traversing the bounding boxes near the dividing line involves converting the x, y, w, and h coordinates of the bounding boxes to the coordinate format of the top-left and bottom-right corners. Then, iterating through the coordinates within a 10-pixel radius of the dividing line checks if any of the coordinates on either side of the dividing line are adjacent. If they are not adjacent, the bounding box is directly output. If they are adjacent, the process continues to determine whether the two adjacent bounding boxes belong to the same target or two separate targets. This step uses the incompleteness indicator 'c' in the label to determine this. If both 'c' values ​​of the two adjacent bounding boxes are 0, it indicates that the targets on either side of the dashed line are incomplete. In this case, the minimum bounding rectangle of the two adjacent bounding boxes is calculated and used as the merged bounding box for output. Otherwise, if both 'c' values ​​are not 0, the two adjacent bounding boxes are directly output.

[0093] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A target detection method based on dense feature point stitching and an improved YOLOv5, characterized in that, Includes the following steps: Step 1: Acquire multiple images of the target area by installing and adjusting multiple cameras adjacent to each other; wherein, the multiple images overlap with each other. Step 2: Based on the overlapping areas, stitch the multiple images together; Step 3: Perform target detection on the stitched image; Step 1 includes: By installing and adjusting three cameras adjacent to each other, three images of the target area are acquired; wherein, the three images overlap with each other, that is, there are four overlapping areas. The three cameras, when stitched together, have a field of view greater than 180°, and all three cameras have the same parameters. Frame synchronization and image quality alignment have been performed. The four overlapping regions each account for 10% to 20% of the FOV of their respective individual cameras. Step 2 includes: Step 21: Using the histogram matching method, with the image at the middle position of the three images as the reference, the images on both sides are matched towards the image at the middle position; Step 22: Use the FAST feature detector to detect feature points in each overlapping region; sort the feature points in each overlapping region from largest to smallest response value, and retain the feature points corresponding to the top 5 largest response values ​​to form a sparse feature set; if there are fewer than 5 response values, retain all of them; where each feature point corresponds to one response. Step 23: Calculate the optical flow of the sparse feature set using the Lucas-Kanade method. Use the image pyramid to match feature points in each feature dimension to obtain matching feature points in four overlapping regions. Then calculate the optical flow of the matching feature points. Each optical flow value is a two-dimensional vector representing the amount of motion in the horizontal and vertical directions. Step 24: Based on the sparse feature set and matching feature points, obtain the optical flow value of each grid after subdividing each overlapping region into 15×3 grids; Step 25: Stitch the three images based on the optical flow value of each grid; Step 25 includes: Step 251: The optical flow value of the top left vertex of each grid is equal to the median smoothed optical flow value in the neighborhood of its 2×2 grid vertex; then, with each grid vertex as the center, median smoothing is performed in the 4×4 vertex neighborhood to obtain the optical flow value of each grid vertex. Step 252: Subdivide each grid into N×N cells, and obtain the optical flow value of the vertex of each cell using the triangle interpolation method; take the optical flow value of the top left vertex of each cell as the optical flow value of all pixels in the cell, thereby obtaining the optical flow value of all pixels in the overlapping area, and then use the optical flow value of the overlapping area to stitch the three images together.

2. The method according to claim 1, characterized in that, Step 252 includes: Step 2521: N×N cells form a quadrilateral. Given the optical flow values ​​of the four vertices P1, P2, P3, and P4 of the quadrilateral, obtain the optical flow value of each cell vertex using triangular interpolation. The optical flow value P of point P in the upper triangle is... v Optical flow values ​​P corresponding to P1, P2, and P3 1v P 2v P 3v Interpolated from: P v =(w1P 1v +w2P 2v +w3P 3v ) / (w1+w2+w3) w i =1 / D i in, Representing the i-th vertex P respectively i x, y coordinates, P x P y Let D represent the x and y coordinates of point P in the triangle to be solved. i Indicates P to P i The distance between them, weight w i With D i Inversely proportional; Step 2522: Following the same logic as step 2521, the optical flow values ​​P corresponding to P2, P3, and P4 are... 2v P 3v P 4v Interpolation is used to obtain the optical flow value of the vertex of the cell located in the lower triangle.

3. The method according to claim 1, characterized in that, Step 3 includes: Based on the fact that the target is likely to appear in the air region in the upper half of the image, the upper half of the image is cropped, and the size of the region is adjusted to be the same as the size of the original image. The adjusted image and the original stitched image are used as training samples and input into the YOLOv5 network for learning. Extract the first feature map corresponding to the adjusted image and the original stitched image, and stitch the first feature map to obtain a feature map group. Then extract the second feature map of the feature map group. The second feature map is divided into multiple blocks for parallel detection. Then, the position of the detected target is correlated with its position in the original stitched image, and the final detection result is output.

4. The method according to claim 3, characterized in that, When using image block detection, if a target spans two adjacent blocks, it checks whether there are any connected target boxes in the adjacent blocks. If they are connected, the two targets belong to the same target. Then, the smallest bounding rectangle of the two connected target boxes is taken as the final output target box on the final ultra-wide-angle image.

5. The method according to claim 3, characterized in that, When using image patch detection, when two targets are adjacent, new labels are created to check whether the targets are complete before detection. The new label format is: [cls,x,y,w,h,c], where cls is the class, x and y are the coordinates of the target, w and h are the width and height of the target, respectively, and c is the target completeness, with a value of 0 or 1; 0 indicates an incomplete target, and 1 indicates a complete target. The sample set is labeled according to the new labels and then trained. The new loss function increases the completeness factor. Loss(all)=Loss(confidence)+Loss(location)+Loss(class)+Loss(completeness) Where Loss(confidence) represents confidence loss, Loss(location) represents location loss, Loss(class) represents class loss, and Loss(completeness) represents completeness loss. 2 B represents the number of grid cells, and B represents the number of anchor boxes in each grid cell. c indicates the existence of a target. i It is the detected c value. It is the labeled truth value, λ comp It is a weighting coefficient that measures the loss of integrity.

6. The method according to claim 5, characterized in that, Iterate through the bounding boxes of the targets near the dividing line between the two targets, convert the coordinates of the top left and bottom right corners of the target boxes to (x1, y1) and (x2, y2) respectively, and then iterate through the coordinates within 10 pixels near the dividing line to check whether the coordinates on both sides of the dividing line are adjacent.

7. The method according to claim 6, characterized in that, If the coordinates on both sides of the dividing line are not adjacent, the target box is output directly; if the coordinates on both sides of the dividing line are adjacent, it is further determined whether the two adjacent target boxes belong to the same target or belong to two different targets by checking whether the target is complete in the label. If the labels of two adjacent target boxes are both 0, it means that the targets on both sides of the dashed line are incomplete. In this case, the minimum bounding rectangle of the two adjacent target boxes is calculated and output as the merged target box. If they are not both 0, the two adjacent target boxes are output directly.

Images