Engineering target detection method and device for high-resolution image

Abstract

The invention relates to an engineering target detection method and device for a high-resolution image. The engineering target detection method for the high-resolution image comprises the following steps: carrying out overlapping cutting on a to-be-detected high-resolution image to obtain a plurality of small-size overlapped to-be-detected images; detecting the plurality of small-size overlapped to-be-detected images by adopting a target detection model to obtain a plurality of small-size detection result images; splicing and combining the plurality of small-size detection result images to obtain a large-resolution detection result image; and sequentially carrying out filtering and frame fusion on the overlapped region of the high-resolution detection result image by adopting an area-based target frame fusion strategy to obtain an accurate high-resolution detection result image. According to the engineering target detection method for the high-resolution image, through overlapping slicing, target frame splicing and area-based target frame fusion strategies, the problem that a large target frame is easily filtered by mistake when an overlapping region is processed in the prior art is effectively solved.

Description

Technical Field

[0001] The present invention relates to the field of multi-category and multi-target detection of large-resolution images, and in particular to a method, device, electronic equipment and computer storage medium for engineering target detection of large-resolution images. Background Art

[0002] During the construction process, engineering technicians usually need to analyze the construction area on the project map in order to accurately estimate the project cost and material preparation, thereby assisting engineering technicians to reasonably schedule resources, optimize construction plans, and improve overall construction efficiency.

[0003] To speed up the analysis of construction areas, traditional technologies use a template matching-based approach to analyze satellite images provided by a Geographic Information System (GIS). However, while template matching performs well when target features (such as shape and size) are fixed, its effectiveness decreases significantly when the target is rotated, scaled, or partially occluded, making it difficult to cope with complex construction scenarios.

[0004] With the popularization of drone aerial photography technology and machine learning, it has become a common practice to use drones to collect high-resolution aerial images of construction scenes. Machine learning is used to detect and analyze the features of the construction area in the aerial images to detect changes or progress in the construction area, thereby assisting engineering technicians in making decisions based on changes in the construction area. However, conventional machine learning needs to scale or slice the aerial images when processing high-resolution aerial images, and scaling can easily lead to the loss of information about small targets in aerial photography, affecting the integrity and accuracy of target detection; while slicing can easily split targets at the edge, affecting the continuity of the target frame.

[0005] Based on this, the prior art proposes an overlapping slicing method (Slicing Aided Hyper Inference, SAHI) combined with deep learning for multi-target detection. The overlapping slicing method overlaps and slices a high-resolution aerial image into several small-size images, and inputs all the small-size images into the target detection model to detect the targets in the small-size images, and splices the detection results of each small image to generate a detection map consistent with the aerial image. Then, for the duplicate target frames that may exist in the overlapping areas of the spliced ​​combination, the prior art filters the target frames by combining non-maximum suppression (NMS), thereby retaining the most representative target frames. The target frame can be used by engineering and technical personnel to quickly analyze the area occupied by the construction area, and prepare materials and schedule resources accordingly.

[0006] However, when dealing with overlapping areas, NMS usually selects the target frames to be retained based on confidence and overlap (intersection-over-union ratio). When multiple target frames have similar confidence but large size differences, smaller target frames may be retained due to their higher confidence, resulting in larger target frames being filtered out. As a result, when the detected target is at the edge of an overlapping slice, its complete target frame is easily misfiltered, resulting in the target frame not being able to accurately fit the target area, which in turn makes it impossible for engineering technicians to analyze the construction area or prepare materials based on the accurate target frame. Summary of the invention

[0007] Based on this, the object of the present invention is to provide a method for detecting engineering targets in large-resolution images.

[0008] A method for detecting engineering targets in large-resolution images comprises the following steps:

[0009] S1: Overlap and cut the large-resolution image to be detected to obtain several small-size overlapping images to be detected;

[0010] S2: Use the target detection model to detect a number of small-sized overlapping images to be detected, and obtain a number of small-sized detection result images;

[0011] S3: splicing and combining several small-size detection result images to obtain a large-resolution detection result image;

[0012] S4: Using an area-based target frame fusion strategy to filter and frame fuse the overlapping areas of the large-resolution detection result map in sequence to obtain an accurate large-resolution detection result map; wherein the area-based fusion frame strategy is to screen according to the area ratio of the target frame to the overlapping area, the area ratio between the target frames, and the intersection-and-union ratio between the target frames.

[0013] The engineering target detection method for large-resolution images described in the present invention effectively solves the problem in the prior art that large target frames are easily misfiltered when processing overlapping areas through overlapping slices, target frame splicing and area-based target frame fusion strategies, and ensures that no matter the target is located in the edge area of ​​the overlapping slice or spans multiple slices, the target frame can accurately fit the corresponding target area, and avoids the phenomenon of overlapping frames through area fusion, thereby significantly improving the accuracy and reliability of target detection in large-resolution images, and further ensuring the integrity of the target frame of large targets in engineering scenes.

[0014] Furthermore, the step S4 includes the following sub-steps:

[0015] S41: performing overlapping area analysis on the large-resolution detection result image to obtain a number of frames to be analyzed;

[0016] Wherein, the overlapping area analysis is expressed as:

[0017] overlap={area i ∪area j |i∈[1,n x -1],j∈[1,n y -1]}

[0018] In the formula, overlap is the set of all overlapping areas; area i and area j are the horizontal overlap area and the vertical overlap area respectively; n x and n y are the number of horizontal and vertical cut blocks of the small-sized overlapping images to be detected, respectively. The specific calculation is as follows:

[0019]

[0020] Where W and H are the width and height of the large-resolution image to be detected, respectively; is the rounding operation; w and h are the width and height of the small-sized overlapping image to be detected; th is the overlapping area threshold for cutting;

[0021] Next, the target frame that intersects with all overlapping areas is obtained to obtain several frames to be analyzed, which are specifically expressed as follows:

[0022] Box={box k |box k ∩overlap}

[0023] In the formula, Box represents the set of all boxes to be analyzed; box k ∩overlap represents the kth target box k There is a geometric intersection with any overlapping area; where k∈N, N is the total number of target boxes that intersect with all overlapping areas;

[0024] S42: Filtering a number of frames to be analyzed by using area-based non-maximum suppression to obtain a number of frames to be fused;

[0025] The specific operation of filtering based on area-based non-maximum suppression is as follows:

[0026]

[0027] In the formula, Fusion boxis the set of all frames to be fused; remove(·) is used to remove the input frame to be analyzed from the large-resolution detection result map; keep(·) is used to retain the input frame to be analyzed in the large-resolution detection result map; The current box to be analyzed k The intersection area with the overlapping area and the box to be analyzed k The ratio between the areas of the current box to be analyzed. k The area ratio with other frames to be analyzed is expressed as follows:

[0028]

[0029] In the formula, area(box k ∩box i ) indicates the current box to be analyzed k The intersection area with other boxes to be analyzed; area(box k ) is the current box to be analyzed k area;

[0030] S43: filtering a plurality of frames to be fused by using intersection-and-union ratio, and fusing the filtered frames to be fused to obtain an accurate high-resolution detection result image; wherein the specific operation of filtering by using intersection-and-union ratio is as follows:

[0031]

[0032] In the formula, Fusion filter is the set of all filtered frames to be fused; IoU is the intersection over union ratio, which is calculated as follows:

[0033]

[0034] In the formula, area(box k ∪box i ) represents the current frame to be fused box k The area combined with other frames to be merged;

[0035] Next, Fusion filter There are intersecting target boxes that are fused to obtain accurate high-resolution detection result images for the i-th target box. i Intersect with the corresponding j-th target box j The coordinates of the upper left corner and lower right corner of the fused target box are expressed as:

[0036] (min(x i L ,x j L),min(y i L ,y j L ))

[0037] (max(x i R ,x j R ),max(y i R ,y j R ))

[0038] In the formula, (x i L ,y i L ) and (x i R ,y i R ) are represented by box i The upper left corner coordinates and lower right corner coordinates (x j L ,y j L ) and (x j R ,y j R ) are represented by box j The upper left corner coordinates and lower right corner coordinates of the image; min(·) and max(·) represent the minimum value and maximum value, respectively.

[0039] The present invention removes unnecessary small target frames in the overlapping area in an area-based manner to avoid the situation where small target frames are over-retained and large targets are mistakenly filtered. At the same time, the present invention can effectively ensure that small target frames can be accurately extracted in the overlapping area based on the IoA and overlapping area ratio thresholds set based on experience, reduce the risk of mistaken removal, and improve the detection accuracy of small targets. In addition, the set IoU threshold can ensure the accuracy of multi-target detection, avoid the mistaken fusion of independent targets, and ensure that large targets can still be fully detected when spanning multiple slices.

[0040] Furthermore, the specific calculation of the overlapping cutting is as follows:

[0041] I slide ={img i,j =Crop((w x (i),h y (j)),(w,h))|i∈[0,n x -1],j∈[0,n y -1]}

[0042] In the formula, I slide is a collection of several small-sized overlapping images to be detected; img i,j represents the small-size overlapping image to be detected in the i-th row and j-th column; Crop((w x (i),h y (j)), (w,h)) represents the starting position of the image to be detected from the large resolution (w x (i),h y (j)) Start cutting and cut out small overlapping images to be detected with a size of w×h; w x (i) is calculated by the following method:

[0043] w x (i) = i × w × (1-th)

[0044] Where w is the preset cutting width; th is the overlapping area threshold of the cutting; and h y (j) Calculated as follows:

[0045] h y (j) = j × h × (1-th)

[0046] Where h is the preset cutting height.

[0047] The present invention improves the coverage of the target through overlapping cutting. Especially when the target spans multiple image blocks, the overlapping area acts as a buffer to prevent the target from being lost due to image segmentation. At the same time, it improves the coverage of target detection and ensures the continuity and accuracy of target detection.

[0048] Furthermore, before adopting the target detection model, a number of small-sized overlapping images to be detected are grouped according to a number of threads, and the grouped small-sized overlapping images to be detected are assigned to the target detection model in the corresponding thread for detection; after the detection is completed, the detection results in all threads are summarized to obtain a number of small-sized detection result images;

[0049] Wherein, the number of threads is a preset parameter.

[0050] The present invention groups several small-sized overlapping images to be detected and assigns them to a multi-threaded target detection model to effectively improve the parallel processing capability of detection, significantly speed up the detection speed of large-resolution images, and ensure the efficient processing of large-scale image processing tasks. In addition, the target detection model can be further deployed on different devices through multi-threading to achieve distributed processing, thereby significantly speeding up the processing of large-scale images, and can also make the target detection model adapt to a variety of hardware environments and application scenarios.

[0051] An engineering target detection device for large-resolution images, comprising an image overlapping cutting unit, an image detection unit, an image stitching unit and a target frame fusion unit;

[0052] The image overlapping cutting unit is used to perform overlapping cutting on the large-resolution image to be detected to obtain a plurality of small-size overlapping images to be detected;

[0053] The image detection unit is used to detect a plurality of small-sized overlapping to-be-detected images using a target detection model to obtain a plurality of small-sized detection result images;

[0054] The image stitching unit is used to stitch together a number of small-size detection result images to obtain a large-resolution detection result image;

[0055] The target frame fusion unit is used to sequentially filter and frame fuse the overlapping areas of the large-resolution detection result map using an area-based target frame fusion strategy to obtain an accurate large-resolution detection result map;

[0056] The area-based fusion frame strategy is to perform screening according to the area ratio between the target frame and the overlapping area, the area ratio between the target frames, and the intersection-and-union ratio between the target frames.

[0057] Furthermore, the target frame fusion unit includes an overlapping area analysis module, a target frame filtering module and a target frame fusion module;

[0058] The overlapping region analysis module is used to perform overlapping region analysis on the high-resolution detection result image to obtain a number of frames to be analyzed; wherein the overlapping region analysis is expressed as:

[0059] overlap={area i ∪area j |i∈[1,n x -1],j∈[1,n y -1]}

[0060] In the formula, overlap is the set of all overlapping areas; area i and area j are the horizontal overlap area and the vertical overlap area respectively; n x and n y are the number of horizontal and vertical cut blocks of the small-sized overlapping images to be detected, respectively. The specific calculation is as follows:

[0061]

[0062] Where W and H are the width and height of the large-resolution image to be detected, respectively; is the rounding operation; w and h are the width and height of the small-sized overlapping image to be detected; th is the overlapping area threshold for cutting;

[0063] Next, the target frame that intersects with all overlapping areas is obtained to obtain several frames to be analyzed, which are specifically expressed as follows:

[0064] Box={box k |box k ∩overlap}

[0065] In the formula, Box represents the set of all boxes to be analyzed; box k ∩overlap represents the kth target box k There is a geometric intersection with any overlapping area; where k∈N, N is the total number of target boxes that intersect with all overlapping areas;

[0066] The target frame filtering module is used to filter a plurality of frames to be analyzed by using area-based non-maximum suppression to obtain a plurality of frames to be fused;

[0067] The specific operation of filtering based on area-based non-maximum suppression is as follows:

[0068]

[0069] In the formula, Fusion box is the set of all frames to be fused; remove(·) is used to remove the input frame to be analyzed from the large-resolution detection result map; keep(·) is used to retain the input frame to be analyzed in the large-resolution detection result map; The current box to be analyzed k The intersection area with the overlapping area and the box to be analyzed k The ratio between the areas of the current box to be analyzed. k The area ratio with other frames to be analyzed is expressed as follows:

[0070]

[0071] In the formula, area(box k ∩box i ) indicates the current box to be analyzed k The intersection area with other boxes to be analyzed; area(box k ) is the current box to be analyzed k area;

[0072] The target frame fusion module is used to filter several frames to be fused by intersection and union ratio, and fuse the filtered frames to be fused to obtain an accurate large-resolution detection result image; wherein the specific operation of filtering by intersection and union ratio is as follows:

[0073]

[0074] In the formula, Fusion filter is the set of all filtered frames to be fused; IoU is the intersection over union ratio, which is calculated as follows:

[0075]

[0076] In the formula, area(box k ∪box i ) represents the current frame to be fused box k The area combined with other frames to be merged.

[0077] Next, Fusion filter There are intersecting target boxes that are fused to obtain accurate high-resolution detection result images for the i-th target box. i Intersect with the corresponding j-th target box j The coordinates of the upper left corner and lower right corner of the fused target box are expressed as:

[0078] (min(x i L ,x j L ),min(y i L ,y j L ))

[0079] (max(x i R ,x j R ),max(y i R ,y j R ))

[0080] In the formula, (x i L ,y i L ) and (x i R ,y i R ) are represented by box i The upper left corner coordinates and lower right corner coordinates (x jL ,y j L ) and (x j R ,y j R ) are represented by box j The upper left corner coordinates and lower right corner coordinates of ; min(I) and max(·) represent the selection of the minimum value and the selection of the maximum value, respectively.

[0081] Furthermore, the image overlapping cutting unit includes an overlapping cutting module for performing overlapping cutting, and the specific calculation thereof is as follows:

[0082] I slide ={img i,j =Crop((w x (i),h y (j)),(w,h))|i∈[0,n x -1],j∈[0,n y -1]}

[0083] In the formula, I slide is a collection of several small-sized overlapping images to be detected; img i,j represents the small-size overlapping image to be detected in the i-th row and j-th column; Crop((w x (i),h y (j)), (w,h)) represents the starting position of the image to be detected from the large resolution (w x (i),h y (j)) Start cutting and cut out small overlapping images to be detected with a size of w×h; w x (i) is calculated by the following method:

[0084] w x (i) = i × w × (1-th)

[0085] Where w is the preset cutting width; th is the overlapping area threshold of the cutting; and h y (j) Calculated as follows:

[0086] h y (j) = j × h × (1-th)

[0087] Where h is the preset cutting height.

[0088] Furthermore, the engineering target detection device for large-resolution images also includes a thread number unit, which is used to group a number of small-sized overlapping images to be detected according to the number of threads before using the target detection model, and assign the grouped number of small-sized overlapping images to be detected to the target detection model in the corresponding thread for detection; after the detection is completed, the detection results in all threads are summarized to obtain a number of small-sized detection result images;

[0089] Wherein, the number of threads is a preset parameter.

[0090] For better understanding and implementation, the present invention is described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] Figure 1 It is a schematic diagram of the overall structure of the engineering target detection device for large-resolution images according to the present invention;

[0092] Figure 2 It is a schematic diagram of the overall process of the engineering target detection method of large-resolution images according to the present invention;

[0093] Figure 3 A schematic diagram showing a simple example of target boxes for different targets in the overlapping area;

[0094] Figure 4 and Figure 5 This is a schematic diagram showing an example of the comparison between the engineering target detection method for large-resolution images described in the present invention and other detection methods. DETAILED DESCRIPTION

[0095] In order to solve the problem in the prior art that when the detected target is in the edge area of ​​the overlapping slice, its complete target frame is easily mis-filtered, resulting in the target frame being unable to accurately fit the target area, the present invention overlaps and cuts the large-resolution image to be detected to obtain a number of small-sized overlapping images to be detected; and uses a target detection model to detect a number of small-sized overlapping images to be detected to obtain a number of small-sized detection result images, and splices and combines them to generate a large-resolution detection result image, and uses an area-based target frame fusion strategy to analyze, filter and fuse the overlapping area to generate an accurate large-resolution detection result image. Accordingly, the present invention uses an area-based target frame fusion strategy to ensure that targets at the edge are mis-filtered due to traditional NMS, and through the fusion of the target frame, the target frame is accurately fitted to the target area, thereby improving the accuracy and reliability of the target detection results at a large resolution.

[0096] Please also see Figure 1 and Figure 2 , Figure 1 is a schematic diagram of the overall structure of the engineering target detection device for large-resolution images according to the present invention, Figure 2 The figure is a schematic diagram of the overall process of the engineering target detection method for large-resolution images described in the present invention.

[0097] Based on the above design, the present invention proposes a method for detecting engineering targets in large-resolution images, and based on the method, proposes a device for detecting engineering targets in large-resolution images.

[0098] The engineering target detection device for large-resolution images comprises an image overlapping cutting unit 1, an image detection unit 2, an image stitching unit 3 and a target frame fusion unit 4.

[0099] The image overlapping cutting unit 1 is used to execute step S1: overlapping and cutting the large-resolution image to be detected to obtain a plurality of small-size overlapping images to be detected.

[0100] Specifically, the specific calculation of the overlapping cutting is as follows:

[0101] I slide ={img i,j =Crop((w x (i),h y (j)),(w,h))|i∈[0,n x -1],j∈[0,n y -1]}

[0102] In the formula, I slide is a collection of several small-sized overlapping images to be detected; img i,j represents the small-size overlapping image to be detected in the i-th row and j-th column; Crop((w x (i),h y (j)), (w,h)) represents the starting position of the image to be detected from the large resolution (w x (i),h y (j)) Start cutting and cut out small overlapping images to be detected with a size of w×h; w x (i) is calculated by the following method:

[0103] w x (i) = i × w × (1-th)

[0104] Where w is the preset cutting width; th is the overlapping area threshold of the cutting, which defaults to 0.2; and h y (j) Calculated as follows:

[0105] h y (j) = j × h × (1-th)

[0106] Where h is the preset cutting height; and n x and n yare the number of horizontal and vertical cut blocks of the small-sized overlapping images to be detected, respectively. The specific calculation is as follows:

[0107]

[0108] Where W and H are the width and height of the large-resolution image to be detected, respectively; This is a round-up operation.

[0109] When the target in the large-resolution image to be detected is located at the edge of a slice or spans multiple slices, part of the target may be ignored due to the cutting of the target, thereby affecting the accuracy of the target detection model. Therefore, the present invention overlaps the slices to ensure that when the target in the large-resolution image to be detected is at the edge of a slice, the target detection model will not be lost, missed or misdetected.

[0110] The image detection unit 2 is used to execute step S2: using the target detection model to detect a number of small-sized overlapping to-be-detected images to obtain a number of small-sized detection result images.

[0111] Specifically, the detection operation is expressed as follows:

[0112] R small = {r i,j =Model(I slide )|i∈[0,n x -1],j∈[0,n y -1]}

[0113] In the formula, R small is a collection of several small-sized detection result images; r i,j It is represented as a small-sized detection result image of the i-th row and j-th column; Model(·) is represented as a target detection model, which can be selected as YOLOv10 (You Only Look Once version10), and the target detection model includes a backbone network, a neck network and a detection head network; the backbone network is used to continuously extract features from the input small-sized overlapping images to be detected to obtain shallow and middle-layer feature maps, and perform feature extraction and spatial feature pyramid pooling on the middle-layer feature map to obtain a deep-layer feature map; the neck network is used to use a feature pyramid to fuse the shallow, middle and deep feature maps to obtain shallow, middle and deep fused feature maps; the detection head network is used to perform feature classification and positioning on the shallow, middle and deep fused feature maps respectively to obtain small-, medium- and large-scale detection results, and integrate all the detection results to obtain a small-sized detection result image.

[0114] In order to enable the target detection model to effectively detect engineering targets, this embodiment further trains the target detection model.

[0115] In the training preparation stage, the present invention processes a batch of high-resolution engineering construction pictures through step S1 to obtain a number of small-sized overlapping images to be detected, and selects the targets in the several small-sized overlapping images to be detected manually or using annotation tools, and adds corresponding labels to generate training sets, test sets and verification sets.

[0116] In the first stage of training, the original small-sized overlapping images to be detected in the training set are input into the target detection model for target detection to obtain a small-sized detection result image; the positioning loss function and the classification loss function are used to calculate the loss values ​​of the small-sized detection result image and the corresponding target boxes and labels in the training set; and the weights of the current target detection model are updated according to the loss values.

[0117] Among them, the positioning loss function can select different intersection-over-union loss calculations such as IOU loss, GIOU loss, CIOU loss or EIOU loss; the classification loss function can select the cross entropy loss function. The selection of the specific loss function can be adjusted according to the actual application scenario. The present invention does not specifically limit the selection of its loss function.

[0118] In the second stage of training, the original small target overlapping to-be-detected image in the verification set is input into the target detection model for target detection to obtain a small-size detection result image; the small-size detection result image is compared with the target box and label in the verification set, the accuracy is statistically calculated, and it is determined whether the accuracy meets a training threshold: if not, return to the first stage of training; if so, enter the third stage of training.

[0119] The training threshold is a custom standard, and the loss value of the optional validation set converges, or the accuracy reaches a set standard, such as training rounds (Epochs).

[0120] In the third stage of training, the original small target overlapping to-be-detected image in the test set is input into the target detection model for target detection to obtain a small-size detection result image; and it is determined whether the current small-size detection result image meets a detection standard; if not, it returns to the first stage of training; if so, the training is completed to obtain the final target detection model.

[0121] In this embodiment, the detection standard requires that the target detection model achieve a detection rate of at least 70% in the validation set and a missed detection rate of less than 5% to ensure that subsequent frame fusion can meet the expected standards for engineering detection. However, this standard is set based on actual application requirements, and the present invention does not limit the setting of its detection standard.

[0122] Since there are many choices of target detection models, such as R-CNN series, YOLO series, SSD series and Swim-Transformer, YOLOv10 is selected in this embodiment, but other models can be replaced according to needs in actual applications. Therefore, the present invention is not limited to a specific target detection model.

[0123] Furthermore, before adopting the target detection model, a number of small-sized overlapping images to be detected are grouped according to a number of threads, and the grouped small-sized overlapping images to be detected are assigned to the target detection model in the corresponding thread for detection; after the detection is completed, the detection results in all threads are summarized to obtain a number of small-sized detection result images.

[0124] Among them, the number of threads is a preset parameter and is customized according to actual device resources; a number of small-sized overlapping images to be detected are detected through multi-threading, thereby accelerating the speed of data processing and model reasoning.

[0125] In addition, since the main detection object of the present invention is the engineering target, and the engineering target usually has a regular geometric shape. Although instance segmentation can accurately outline the target boundary, and the overlapping problem can be easily solved by semantic identity / fusion, in engineering detection, too precise target boundaries are actually not conducive to subsequent regional calculations and construction planning. Therefore, a rectangular target box can more concisely characterize the engineering target area, and compared with instance segmentation, the rectangular box requires fewer computing resources. Accordingly, the present invention needs to use a rectangular target box to represent the construction area to simplify the detection process and ensure that the detection results can be directly applied to engineering planning and decision-making, thereby providing effective information for subsequent resource allocation and construction management.

[0126] The image stitching unit 3 is used to execute step S3: stitching and combining a plurality of small-size detection result images to obtain a detection result image with a large resolution.

[0127] Specifically, the splicing combination is specifically represented as follows:

[0128]

[0129] In the formula, R full It is a high-resolution detection result image, which contains several detection targets; is a translation operation used to convert the small-size detection result image r i,j Translate to the starting coordinates of the global coordinates (w x (i),h y (j)), thereby realizing the splicing combination of small-size detection result images; It is used to indicate that all small-sized detection result images are translated to the position of the global coordinate according to indexes i and j in sequence to form a large image.

[0130] Based on this, the small-size detection result images after detection are spliced ​​according to the rules of overlapping slicing to avoid target omissions or discontinuous detection due to the segmentation of small-size detection result images, and the global target detection results of the large-resolution image to be detected are effectively restored to ensure the integrity and accuracy of the overall detection of the large-resolution image.

[0131] The target frame fusion unit 4 is used to execute step S4: using an area-based target frame fusion strategy to sequentially filter and frame fuse the overlapping areas of the large-resolution detection result image to obtain an accurate large-resolution detection result image.

[0132] Specifically, the area-based fusion frame strategy is to perform screening according to the area ratio of the target frame to the overlapping area, the area ratio between the target frames, and the intersection-and-union ratio between the target frames, so as to determine the target frames that need to be retained, deleted, and fused.

[0133] The target frame fusion unit 4 includes an overlapping region analysis module 41 , a target frame filtering module 42 and a target frame fusion module 43 .

[0134] The overlapping region analysis module 41 is used to execute step S41: performing overlapping region analysis on the high-resolution detection result image to obtain a plurality of frames to be analyzed.

[0135] Specifically, the overlapping area analysis can be expressed as:

[0136] overlap={area i ∪area j |i∈[1,n x -1],j∈[1,n y -1]}

[0137] In the formula, overlap is the set of all overlapping areas; area i and area j are horizontal overlapping areas and vertical overlapping areas respectively. Next, the target frame that intersects with all overlapping areas is obtained to obtain several frames to be analyzed, which are specifically expressed as:

[0138] Box={box k |box k ∩overlap}

[0139] In the formula, Box represents the set of all boxes to be analyzed; box k ∩overlap represents the kth target box k There is a geometric intersection with any overlapping region; where k∈N, N is the total number of target boxes that intersect with all overlapping regions.

[0140] Since there are overlapping areas, that is, the same target is detected multiple times, it is necessary to focus on extracting the target frames in the overlapping areas to analyze the target overlap phenomenon.

[0141] The target frame filtering module 42 is used to execute step S42: filtering a plurality of frames to be analyzed by using area-based non-maximum suppression to obtain a plurality of frames to be fused.

[0142] Specifically, the specific operation of filtering based on area-based non-maximum suppression is as follows:

[0143]

[0144] In the formula, Fusion box is the set of all frames to be fused; remove(·) is used to remove the input frame to be analyzed from the large-resolution detection result map; keep(·) is used to retain the input frame to be analyzed in the large-resolution detection result map; The current box to be analyzed k The intersection area with the overlapping area and the box to be analyzed k The ratio between the areas of the current box to be analyzed. k The area ratio with other frames to be analyzed is expressed as follows:

[0145]

[0146] In the formula, area(box k ∩box i ) indicates the current box to be analyzed k The intersection area with other boxes to be analyzed; area(box k ) is the current box to be analyzed k area.

[0147] In the overlapping area, there are mainly two different situations for the target box, such as Figure 3 As shown, the target boxes corresponding to the small and large targets in the overlapping area are:

[0148] For small targets, since most of the target area is in the overlapping area, it can be considered that there is a target frame that almost completely covers the small target. Therefore, we only need to remove the target frame with a small area in the small target and keep the target frame with the largest area to obtain a target frame that completely fits the current small target.

[0149] For large targets, especially when they span overlapping areas, a single target frame cannot completely fit the entire target. Therefore, these target frames will be marked as frames to be fused after filtering for further analysis and processing in subsequent steps.

[0150] The target frame fusion module 43 is used to execute step S43: filtering a plurality of frames to be fused by intersection and union comparison, and fusing the filtered frames to be fused to obtain an accurate high-resolution detection result image.

[0151] Specifically, the specific operation of filtering by intersection-and-union ratio is as follows:

[0152]

[0153] In the formula, Fusion filter is the set of all filtered frames to be fused; IoU is the intersection over union ratio, which is calculated as follows:

[0154]

[0155] In the formula, area(box k ∪box i ) represents the current frame to be fused box k The area combined with other frames to be merged.

[0156] Next, Fusion filter There are intersecting target boxes that are fused to obtain accurate high-resolution detection result images for the i-th target box. i Intersect with the corresponding j-th target box j The coordinates of the upper left corner and lower right corner of the fused target box are expressed as:

[0157] (min(x i L ,x j L ),min(y i L ,y j L ))

[0158] (max(x i R ,x j R ),max(y i R ,y j R ))

[0159] In the formula, (x i L ,y i L ) and (x i R ,y i R) are represented by box i The upper left corner coordinates and lower right corner coordinates (x j L ,y j L ) and (x j R ,y j R ) are represented by box j The upper left corner coordinates and lower right corner coordinates of the image; min(·) and max(·) represent the minimum value and maximum value, respectively.

[0160] Since the target frames of large targets in overlapping areas are prone to overlap due to cross-regional reasons, the present invention excludes the detection frames of small targets and further uses the intersection-and-union ratio to analyze and filter the frames to be fused. Through actual testing, the present invention learned that when the intersection-and-union ratio between the frames to be fused is less than 0.5, it is considered that the two target frames are detection results representing different targets respectively, so no further fusion is required; and when the intersection-and-union ratio between the frames to be fused is greater than or equal to 0.5, it is considered that the two target frames represent the detection results of the same target, and accordingly, the two target frames need to be fused at this time. The present invention uses the maximum and minimum values ​​of the upper left corner and the lower right corner to dynamically adjust the boundaries of the target frame, thereby automatically adapting to the fusion between multiple target frames under large targets, thereby optimizing the existing target detection model for large-resolution image detection efficiency and positioning accuracy.

[0161] Compared with the prior art, the present invention analyzes the target frame of the overlapping area based on area non-maximum suppression, and eliminates useless overlapping frames while ensuring that the target frame can be correctly framed even if the small target is in the edge area of ​​the slice, thereby significantly improving the recognition accuracy of the target detection model for small targets in complex scenes. At the same time, the present invention filters the frames to be fused by the intersection ratio and fuses them, ensuring that large targets can be fully detected even if they span multiple slice areas, avoiding the situation of incomplete target frame segmentation. This strategy significantly improves the detection accuracy of large targets in high-resolution images, especially in complex engineering scenes.

[0162] In addition, since the present invention adopts an overlapping slicing method to perform target detection on a small-sized image to be detected, even if the target is located in multiple slice areas, as long as the target is successfully detected in any small image, it can be framed and accurately presented in a large-resolution detection result image through an area-based target frame fusion strategy, reducing possible missed detections due to detection errors or targets spanning multiple slices, thereby significantly improving the detection rate of target detection, and the detection rate has reached 98%.

[0163] Please also see Figure 4 and Figure 5 , where labels 1, 2 and 3 are comparison diagrams of not analyzing the overlapping area, performing NMS processing on the overlapping area and performing processing using the area-based target frame fusion strategy of the present invention. It can be seen that label 3 can frame the target more accurately without the phenomenon of overlapping target frames.

[0164] Based on the same inventive concept, the present application also provides an electronic device, which may be a terminal device such as a server, a desktop computing device or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet computer, a netbook, etc.). The device includes one or more processors and a memory, wherein the processor is used to execute a program to implement the engineering target detection method for a large-resolution image according to an embodiment of the present invention; and the memory is used to store a computer program executable by the processor.

[0165] Based on the same inventive concept, the present application also provides a computer-readable storage medium, corresponding to the embodiments of the aforementioned engineering target detection method for large-resolution images, wherein the computer-readable storage medium stores a computer program thereon, and when the program is executed by a processor, the steps of the engineering target detection method for large-resolution images recorded in any of the aforementioned embodiments are implemented.

[0166] The present application may take the form of a computer program product implemented on one or more storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program code. Computer-usable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0167] The above-mentioned embodiments only express several implementation methods of the present invention, and the description is relatively specific and detailed, but it cannot be understood as limiting the scope of the invention patent. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, and the present invention is also intended to include these modifications and modifications.

Claims

1. A method for detecting engineering targets in large-resolution images, characterized in that: The following steps are involved: S1: Overlap and cut the large-resolution image to be detected to obtain several small-size overlapping images to be detected; S2: Use the target detection model to detect a number of small-sized overlapping images to be detected, and obtain a number of small-sized detection result images; S3: splicing and combining several small-size detection result images to obtain a large-resolution detection result image; S4: Adopting the area-based target frame fusion strategy to filter and frame-fuse the overlapping areas of the large-resolution detection result map in turn, and obtain an accurate large-resolution detection result map; The area-based fusion frame strategy is to perform screening according to the area ratio between the target frame and the overlapping area, the area ratio between the target frames, and the intersection-and-union ratio between the target frames.

2. The engineering target detection method of large-resolution images according to claim 1 is characterized in that: The step S4 comprises the following sub-steps: S41: Perform overlapping area analysis on the high-resolution detection result image to obtain a number of frames to be analyzed; wherein the overlapping area analysis is expressed as: overlap={area i ∪area j |i∈[1,n x -1],j∈[1,n y -1]} In the formula, overlap is the set of all overlapping areas; area i and area j are the horizontal overlap area and the vertical overlap area respectively; n x and n y are the number of horizontal and vertical cut blocks of the small-sized overlapping images to be detected, respectively. The specific calculation is as follows: Where W and H are the width and height of the large-resolution image to be detected, respectively; is the rounding operation; w and h are the width and height of the small-sized overlapping image to be detected; th is the overlapping area threshold for cutting; Next, the target frame that intersects with all overlapping areas is obtained to obtain several frames to be analyzed, which are specifically expressed as follows: Box={box k |box k ∩overlap} In the formula, Box represents the set of all boxes to be analyzed; box k ∩overlap represents the kth target box k There is a geometric intersection with any overlapping area; where k∈N, N is the total number of target boxes that intersect with all overlapping areas; S42: Filtering a number of frames to be analyzed by using area-based non-maximum suppression to obtain a number of frames to be fused; The specific operation of filtering based on area-based non-maximum suppression is as follows: In the formula, Fusion box is the set of all frames to be fused; remove(·) is used to remove the input frame to be analyzed from the large-resolution detection result map; keep(·) is used to retain the input frame to be analyzed in the large-resolution detection result map; The current box to be analyzed k The intersection area with the overlapping area and the box to be analyzed k The ratio between the areas of the current box to be analyzed. k The area ratio with other frames to be analyzed is expressed as follows: In the formula, area(box k ∩box i ) indicates the current box to be analyzed k The intersection area with other boxes to be analyzed; area(box k ) is the current box to be analyzed k area; S43: filtering a plurality of frames to be fused by using intersection-and-union ratio, and fusing the filtered frames to be fused to obtain an accurate high-resolution detection result image; wherein the specific operation of filtering by using intersection-and-union ratio is as follows: In the formula, Fusion filter is the set of all filtered frames to be fused; IoU is the intersection over union ratio, which is calculated as follows: In the formula, area(box k ∪box i ) represents the current frame to be fused box k The area combined with other frames to be merged; Next, Fusion filter There are intersecting target boxes that are fused to obtain accurate high-resolution detection result images for the i-th target box. i Intersect with the corresponding j-th target box j The coordinates of the upper left corner and lower right corner of the fused target box are expressed as: In the formula, (x i L ,y i L ) and (x i R ,y i R ) are represented by box i The upper left corner coordinates and lower right corner coordinates (x j L ,y j L ) and (x j R ,y j R 0 represents box j The upper left corner coordinates and lower right corner coordinates of the image; min(·) and max(·) represent the minimum value and maximum value, respectively.

3. The engineering target detection method of large-resolution images according to claim 2 is characterized in that: The specific calculation of the overlapping cutting is as follows: I slide ={img i,j =Crop((w x (i),h y (j)),(w,h))|i∈[0,n x -1],j∈[0,n y -1]} In the formula, I slide is a collection of several small-sized overlapping images to be detected; img i,j represents the small-size overlapping image to be detected in the i-th row and j-th column; Crop((w x (i),h y (j)), (w,h)) represents the starting position of the image to be detected from the large resolution (w x (i),h y (j)) Start cutting and cut out small overlapping images to be detected with a size of w×h; w x (i) is calculated by the following method: In x (i)=i×w×(1-th) Where w is the preset cutting width; th is the overlapping area threshold of the cutting; and h y (j) Calculated as follows: h y (j)=j×h×(1-th) Where h is the preset cutting height.

4. The engineering target detection method of large-resolution images according to claim 3 is characterized in that: Before using the target detection model, a number of small-sized overlapping images to be detected are grouped according to the number of threads, and the grouped small-sized overlapping images to be detected are assigned to the target detection model in the corresponding thread for detection; after the detection is completed, the detection results in all threads are summarized to obtain a number of small-sized detection result images; Wherein, the number of threads is a preset parameter.

5. An engineering target detection device for large-resolution images, characterized in that: It includes an image overlapping cutting unit, an image detection unit, an image stitching unit and a target frame fusion unit; The image overlapping cutting unit is used to perform overlapping cutting on the large-resolution image to be detected to obtain a plurality of small-size overlapping images to be detected; The image detection unit is used to detect a plurality of small-sized overlapping to-be-detected images using a target detection model to obtain a plurality of small-sized detection result images; The image stitching unit is used to stitch together a number of small-size detection result images to obtain a large-resolution detection result image; The target frame fusion unit is used to use an area-based target frame fusion strategy to filter and fuse the overlapping areas of the large-resolution detection result image in sequence to obtain an accurate large-resolution detection result image; wherein the area-based fusion frame strategy is to screen according to the area ratio of the target frame to the overlapping area, the area ratio between the target frames, and the intersection-and-union ratio between the target frames.

6. The engineering target detection device of large-resolution image according to claim 5, characterized in that: The target frame fusion unit includes an overlapping area analysis module, a target frame filtering module and a target frame fusion module; The overlapping region analysis module is used to perform overlapping region analysis on the high-resolution detection result image to obtain a number of frames to be analyzed; wherein the overlapping region analysis is expressed as: overlap={area i ∪area j |i∈[1,n x -1],j∈[1,n y -1]} In the formula, overlap is the set of all overlapping areas; area i and area j are the horizontal overlap area and the vertical overlap area respectively; n x and n y are the number of horizontal and vertical cut blocks of the small-sized overlapping images to be detected, respectively. The specific calculation is as follows: Where W and H are the width and height of the large-resolution image to be detected, respectively; is the rounding operation; w and h are the width and height of the small-sized overlapping image to be detected; th is the overlapping area threshold for cutting; Next, the target frame that intersects with all overlapping areas is obtained to obtain several frames to be analyzed, which are specifically expressed as follows: Box={box k |box k ∩overlap} In the formula, Box represents the set of all boxes to be analyzed; box k ∩overlap represents the kth target box k There is a geometric intersection with any overlapping area; where k∈N, N is the total number of target boxes that intersect with all overlapping areas; The target frame filtering module is used to filter a plurality of frames to be analyzed by using area-based non-maximum suppression to obtain a plurality of frames to be fused; The specific operation of filtering based on area-based non-maximum suppression is as follows: In the formula, Fusion box is the set of all frames to be fused; remove(·) is used to remove the input frame to be analyzed from the large-resolution detection result map; keep(·) is used to retain the input frame to be analyzed in the large-resolution detection result map; The current box to be analyzed k The intersection area with the overlapping area and the box to be analyzed k The ratio between the areas of the current box to be analyzed. k The area ratio with other frames to be analyzed is expressed as follows: In the formula, area(box k ∩box i ) indicates the current box to be analyzed k The intersection area with other boxes to be analyzed; area(box k ) is the current box to be analyzed k area; The target frame fusion module is used to filter several frames to be fused by intersection and union ratio, and fuse the filtered frames to be fused to obtain an accurate large-resolution detection result image; wherein the specific operation of filtering by intersection and union ratio is as follows: In the formula, Fusion filter is the set of all filtered frames to be fused; IoU is the intersection over union ratio, which is calculated as follows: In the formula, area(box k ∪box i ) represents the current frame to be fused box k The area combined with other frames to be merged; Next, Fusion filter There are intersecting target boxes that are fused to obtain accurate high-resolution detection result images for the i-th target box. i Intersect with the corresponding k-th target box j The coordinates of the upper left corner and lower right corner of the fused target box are expressed as: In the formula, (x i L ,y i L ) and (x i R ,y i R ) are represented by box i The upper left corner coordinates and lower right corner coordinates (x j L ,y j L ) and (x j R ,y j R ) are represented by box j The upper left corner coordinates and lower right corner coordinates of the image; min(·) and max(·) represent the minimum value and maximum value, respectively.

7. The engineering target detection device of large-resolution image according to claim 6, characterized in that: The image overlapping cutting unit includes an overlapping cutting module for performing overlapping cutting, and the specific calculation is as follows: I slide ={img i,j =Crop((w x (i),h y (j)),(w,h))|i∈[0,n x -1],k∈[0,n y -1]} In the formula, I slide is a collection of several small-sized overlapping images to be detected; img i,j represents the small-size overlapping image to be detected in the i-th row and j-th column; Crop((w x (i),h y (j)), (w,h)) represents the starting position of the image to be detected from the large resolution (w x (i),h y (j)) Start cutting and cut out small overlapping images to be detected with a size of w×h; w x (i) is calculated by the following method: In x (i)=i×w×(1-th) Where w is the preset cutting width; th is the overlapping area threshold of the cutting; and h y (j) Calculated as follows: h y (j)=j×h×(1-th) Where h is the preset cutting height.

8. The engineering target detection device of large-resolution image according to claim 7, characterized in that: It also includes a thread number unit, which is used to group a number of small-sized overlapping images to be detected according to the number of threads before using the target detection model, and assign the grouped number of small-sized overlapping images to be detected to the target detection model in the corresponding thread for detection; after the detection is completed, the detection results in all threads are summarized to obtain a number of small-sized detection result images; Wherein, the number of threads is a preset parameter.

9. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the engineering target detection method for large-resolution images as described in any one of claims 1 to 4 when executing the computer program.

10. A computer-readable storage medium storing computer-executable instructions, characterized in that: The computer executable instructions are used in the engineering target detection method for large-resolution images as described in any one of claims 1 to 4.

Images