Multi-scale target adaptive detection method and system for high-position oblique viewing angle

Abstract

The application discloses a multi-scale target adaptive detection method and system for high-position inclined viewing angle, and relates to the technical field of computer vision and intelligent transportation. The method prevents video memory overflow by detecting hardware resource dynamic planning slice parameters, combining a video memory guard mechanism to real-time control inference load; the weighted non-maximum suppression fusion slice result is adopted, and the road geometric topology direction field is used to check the motor vehicle rotating frame angle and perform confidence penalty; according to the target longitudinal depth, the region is divided, and the partition non-maximum suppression is executed. The application also enhances the model generalization by perspective simulation and vanishing line constraint small target copy and paste. The application solves the problems of high multi-scale target missing detection rate, unstable cross-device deployment and insufficient detection accuracy under high-position inclined viewing angle, and realizes accurate real-time perception of more than ten times scale difference targets.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and intelligent transportation technology, specifically to a multi-scale adaptive target detection method and system for high-angle tilted viewpoints. Background Technology

[0002] Road intersections are key nodes in urban transportation networks and also areas prone to traffic congestion and accidents. With the rapid advancement of intelligent transportation construction, intersection target detection technology based on computer vision has become a core foundation for applications such as traffic flow statistics, signal timing optimization, and illegal behavior identification. Currently, intersection surveillance cameras are typically mounted at high positions (mostly 6-15 meters, with drone aerial photography reaching even higher altitudes) and generally have a downward shooting angle, resulting in target size differences of up to 10 times within the same surveillance frame: near-end vehicles can have pixel sizes exceeding 300 pixels, while far-end pedestrians and non-motorized vehicles occupy only 15-25 pixels, representing a typical small target detection scenario that poses a severe challenge to existing general-purpose detection algorithms.

[0003] To address the need for target detection at intersections, existing technologies have developed several solutions: Single-stage detection algorithms, represented by the YOLO series, achieve a balance between detection accuracy and speed through anchorless frame design and decoupled head structure. However, in high-level monitoring scenarios, they lack sufficient feature representation for small targets at a distance, resulting in a high false negative rate. The SAHI slice inference scheme effectively improves the recall rate of small targets by dividing the input image into overlapping sub-images, detecting them separately, and then merging the results. However, it uses fixed slice parameters, which cannot adapt to different resolution inputs and different hardware memory constraints. Small memory devices are prone to memory overflow, while large memory devices suffer from wasted computing power. Multi-target tracking algorithms such as ByteTrack maintain the temporal identity of targets by associating detection with trajectory. However, when multiple video streams share a tracker, ID interference is likely to occur, and there is a lack of effective trajectory prediction mechanisms when frames are lost. OBB-based rotated box detection methods can more tightly surround tilted targets, but their output format differs significantly from conventional horizontal boxes. Existing systems require separate development of post-processing logic, which significantly increases system complexity.

[0004] While the aforementioned existing technical solutions have achieved phased results in their respective dimensions, they lack collaborative optimization for intersection scenarios. They cannot simultaneously ensure small target recall, cross-device deployment robustness, and multi-model compatibility within a single system, specifically exhibiting the following shortcomings: First, there is no targeted enhancement for tilted perspectives; general data augmentation strategies (such as Mosaic, MixUp, and random horizontal flipping) cannot simulate perspective distortion at high tilt angles, leading to a severe domain offset between model training and actual deployment scenarios. Second, the fixed slice size lacks adaptive capability; slice parameters are not dynamically adjusted based on GPU memory capacity and target scale distribution. When memory is ample, excessively small slices increase redundant computation and decrease throughput; when memory is insufficient, even reducing the batch size can easily cause overflow crashes. Third, there is no OBB angle prediction capability. The following issues arise: First, the scene constraints and general post-processing methods fail to utilize prior information about vehicle orientation in road scenes, which is constrained by lane geometry. This can easily lead to physically unreasonable bounding box predictions, such as vehicles crossing lanes. Second, the globally uniform threshold is not adapted to scale changes. Using the same confidence threshold and non-maximum suppression parameters across the entire frame results in small distant targets being filtered out due to low confidence, while large nearby targets are mistakenly suppressed due to occlusion. Third, the CPU / GPU mode requires manual switching, lacking a hardware-enabled automatic detection and inference parameter end-to-end adaptive mechanism. Deploying to different devices requires manual configuration adjustments, leading to high implementation costs. Fourth, the small target copy-paste enhancement lacks semantic constraints. Existing methods paste small targets at random locations across the entire image, which can easily paste targets into non-road areas such as the sky and building facades, introducing semantically polluting noise samples and actually reducing detection accuracy.

[0005] In summary, existing technologies are ill-suited to the specific needs of high-angle monitoring scenarios at road intersections, and there is an urgent need for a multi-scale adaptive target detection scheme that balances detection accuracy, deployment robustness, and system compatibility. Summary of the Invention

[0006] This invention provides a multi-scale target adaptive detection method and system for high-angle viewing angles, which solves the problems of existing multi-scale target detection schemes for road intersections with high angle viewing angles. These schemes fail to dynamically adapt slice inference parameters according to hardware resources, optimize the rotation frame angle prediction by combining road geometric topology constraints, and adaptively adjust the post-processing threshold for target perspective scale differences. As a result, they have high small target false negative rates, high risk of memory overflow, insufficient detection accuracy, and poor stability when deployed across devices.

[0007] This invention is achieved through the following technical solution:

[0008] Firstly, this application provides a multi-scale adaptive target detection method for high-angle tilted viewpoints, including:

[0009] Acquire road scene images from a high-angle, tilted viewpoint and detect the available hardware resources of the computing device;

[0010] Based on the detected available hardware resources, the slice size, overlap rate, and batch size are selected, and a slice grid sequence covering the entire image is generated according to the image resolution.

[0011] The slices in the slice grid sequence are retrieved in batches, and the target detection model is called to perform forward inference to obtain the rotating box detection box corresponding to each slice. During the inference process, the memory usage is monitored synchronously. If the usage exceeds the preset threshold, a slice size reduction command or an inference backend switching command is issued immediately, and the grid parameters of the remaining unprocessed slices are replanned according to the command.

[0012] The rotated bounding boxes output from each slice are collected, and the candidate boxes in the overlapping areas are merged using a weighted nonmaximum suppression algorithm to obtain a merged set of full-image bounding boxes.

[0013] Traverse the entire set of detection boxes, perform periodic correction of the rotation box angle, and for motor vehicle targets, retrieve the pre-constructed road geometry topology orientation field data, verify the distance between the predicted angle and the circumferential angle of the orientation field. If the deviation exceeds the preset angle threshold, apply a penalty to the confidence of the detection box, and output the set of detection boxes after angle correction and confidence adjustment.

[0014] Based on the set of detection boxes, the image is divided into at least two regions distributed along the vertical direction according to the ordinate of the center point of the detection box. Different confidence thresholds and non-maximum suppression cross-union ratio thresholds are applied to different regions, and partitioned non-maximum suppression is performed to output the final detection result.

[0015] A further optimization scheme is that, in response to the detected available hardware resources, the slice size, overlap rate, and batch size of the dynamic slice are selected, specifically including:

[0016] Query the available video memory capacity of the computing device's GPU. When there is no dedicated GPU, the available video memory capacity is recorded as 0.

[0017] Match the available video memory capacity with the preset memory location range in the pre-calibrated video memory location table, and select the corresponding slice size, overlap ratio, and batch size.

[0018] The larger the video memory, the larger the slice size can be.

[0019] When the available video memory does not fall exactly on the boundary of the level, the slice size is determined by linear interpolation between adjacent levels, and the interpolation result must not exceed the upper limit of the slice size of the next higher level.

[0020] During inference, after each batch is processed, the current memory usage is checked. When the usage exceeds the preset threshold, a first-level degradation is triggered, reducing the size of the remaining unprocessed slices by a preset ratio, ensuring it is not lower than the current minimum threshold, and clearing the GPU compute cache. If a memory overflow exception still occurs, a second-level degradation is triggered, switching the remaining slices to the next lower threshold. If a memory overflow still occurs after multiple second-level degradations, the system automatically switches to the next inference backend.

[0021] A further optimized solution involves traversing the entire image detection box set and performing periodic adjustments to the box rotation angle, specifically including:

[0022] For each rotating frame detection box, when its predicted width is less than its predicted height, the predicted angle is corrected to the original predicted angle plus 90 degrees, and the width and height are swapped. After correction, the angle is normalized to the range of 0 to 180 degrees.

[0023] For motor vehicle targets, query the road direction field at the target's location in the image. The road direction field is obtained in advance by lane line detection or manual calibration. In the case of a crossroads, the road direction field is a discrete set of directions, while in the case of a roundabout or ramp, the road direction field is a continuous direction function.

[0024] The minimum difference between the predicted angle and the allowed direction of the road direction field modulo 180 degrees is calculated as the circumferential angle distance. When the circumferential angle distance exceeds the preset angle threshold, the confidence of the vehicle target detection box is multiplied by the preset penalty coefficient.

[0025] For non-motorized vehicles and pedestrians, only periodic corrections to the rotation frame angle are performed, without applying road direction field constraints.

[0026] A further optimized solution involves dividing the image into at least two longitudinally distributed regions based on the ordinate of the detection box center point, specifically including:

[0027] Let the image height be H. Divide the image vertically into at least two detection regions.

[0028] Among them, the larger the vertical coordinate of the center of the detection box, the higher the confidence threshold and the non-maximum suppression cross-union ratio threshold are for the bottom of the image.

[0029] For targets located near the region boundary, a continuous threshold is calculated by linear interpolation using the target center's ordinate.

[0030] Non-maximum suppression is performed using the interpolated threshold.

[0031] A further optimization scheme includes perspective simulation data augmentation steps during the training phase:

[0032] The training samples are randomly sampled for virtual camera height, tilt angle, and equivalent focal length, and the sampling range of the equivalent focal length covers the common focal length range of surveillance cameras.

[0033] Based on the pinhole camera model, the homography transformation matrix is ​​calculated, and bilinear interpolation perspective transformation is performed on the training image, while the same coordinate transformation is performed simultaneously on the four vertices of the rotated box annotation.

[0034] The vanishing line position is calculated based on the perspective transformation parameters. The candidate area for copying and pasting small targets is limited to the ground area below the vanishing line. The pasting position is selected in the low-density grid. When pasting, the small targets are scaled and edge-blended in accordance with the perspective depth. The angle of the new annotation box is taken as the median direction of the angles of similar targets near the pasting position.

[0035] A further optimization scheme is to perform perspective simulation enhancement with a preset first probability and small target copy-paste enhancement with a preset second probability in each round of training, where the first probability is greater than the second probability.

[0036] A further optimization scheme is that the weighted nonmaximum suppression algorithm is used to merge candidate boxes of overlapping regions, specifically including:

[0037] Within the overlapping area of ​​slices, the detection box that is closer to the center of its respective slice is retained. The confidence of the detection box is multiplied by a distance weight, which decreases as the distance from the target center to the center of its respective slice increases.

[0038] Secondly, this application provides a multi-scale adaptive target detection system for high-angle tilted viewpoints, including:

[0039] The hardware detection module is used to detect the available GPU memory capacity, the number of CPU logical cores, and the SIMD vector instruction set when the system starts up, and select the inference backend and model accuracy based on the detection results;

[0040] The dynamic slicing module is used to look up the slice size, overlap rate and batch size from the pre-calibrated parameter range table based on the available video memory output by the hardware detection module, generate a slice mesh, and call the inference execution module to perform target detection on each slice;

[0041] The memory guard module is used to monitor the GPU memory usage in real time during inference. When the usage exceeds the preset usage threshold, it triggers a reduction in slice size. When memory overflow occurs, it triggers a downgrade or switches the inference backend.

[0042] The rotating frame angle correction module is used to receive the set of detection frames output by the inference execution module, perform periodic angle correction on the detection frames, and perform angle rationality verification and confidence penalty on motor vehicle targets based on the road geometric topology orientation field.

[0043] The partition threshold module is used to receive the set of detection boxes processed by the rotation box angle correction module, divide the image into at least two detection regions distributed along the longitudinal direction according to the center ordinate of the detection box, and perform partition non-maximum suppression using different confidence thresholds and non-maximum suppression cross union ratio thresholds respectively.

[0044] The inference execution module is used to call the deep learning inference engine to perform forward propagation on each slice and return the rotated box detection box.

[0045] Further optimizations include a training enhancement module, used to perform perspective simulation data augmentation and small target copy-paste enhancement with road area constraints during offline training.

[0046] The perspective simulation data enhancement randomly samples the virtual camera height, tilt angle, and equivalent focal length of the training samples, calculates the homography transformation matrix based on the pinhole camera model, and performs bilinear interpolation perspective transformation, while simultaneously performing the same coordinate transformation on the four vertices of the rotating box annotation.

[0047] The small target copy-paste enhancement with road area constraints calculates the vanishing line position based on perspective transformation parameters, limits the paste candidate area to the ground area below the vanishing line, and selects the paste position in the low-density grid.

[0048] A further optimization scheme is that the hardware detection module automatically configures the full-link inference parameters based on the detected available GPU memory capacity, CPU logical core count, and SIMD vector instruction set. The full-link inference parameters include model accuracy, model size, inference resolution, slice parameters, batch size, and number of threads.

[0049] When a memory overflow error occurs multiple times consecutively during GPU inference, the system will automatically downgrade to the next lower inference configuration.

[0050] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0051] By combining targeted enhancement of distant small target features under high-angle tilt with adaptive threshold for partitioning, the detection recall rate of small targets in distant areas is improved and false positives are effectively suppressed under dense occlusion in the near view.

[0052] By leveraging the combined regulation of adaptive dynamic slicing of video memory and a real-time video memory guard mechanism, the risk of video memory overflow on the entire spectrum of hardware devices, from high-end GPUs to low-end CPUs, is significantly reduced, enabling stable cross-device deployment without manual intervention.

[0053] By relying on the dual verification of periodic correction of the rotating frame angle and road geometric topology orientation field constraints, the periodic jump error of angle prediction is eliminated and physically unreasonable vehicle orientation is filtered out, thereby improving the positioning accuracy of the detection frame and the scene fit.

[0054] By using perspective simulation with focal length sampling to enhance small target copying and pasting constrained by vanishing lines to expand the effective training samples, the domain offset problem between training and deployment scenarios is alleviated, and the generalization ability of the model in complex intersection environments is enhanced.

[0055] The various technical aspects work together to achieve a balance between detection accuracy and inference efficiency, ensuring real-time monitoring frame rate while enabling accurate perception of targets with scale differences of more than ten times. Attached Figure Description

[0056] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort. In the drawings:

[0057] Figure 1 This is an overall flowchart of the multi-scale target adaptive detection method for high-angle tilting view provided in the embodiments of this application;

[0058] Figure 2 A schematic diagram of the architecture and data interaction of a multi-scale adaptive target detection system for a high-angle tilting viewpoint provided in an embodiment of this application;

[0059] Figure 3 A schematic diagram illustrating the perspective simulation data enhancement and road area constraint principle based on vanishing lines provided in the embodiments of this application;

[0060] Figure 4 This is a flowchart illustrating the control of the video memory adaptive slicing inference and video memory guard mechanism provided in the embodiments of this application.

[0061] Figure 5 A schematic diagram illustrating the principle of periodic correction of the rotating frame angle and confidence penalty based on the road geometric topology orientation field provided in the embodiments of this application;

[0062] Figure 6 This is a visual diagram illustrating the detection region division and adaptive adjustment of partition thresholds based on the vertical depth of the target, provided in an embodiment of this application. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0064] First, some of the technical terms used in this application will be explained to help those skilled in the art understand this application.

[0065] Firstly, such as Figure 1 This application provides a multi-scale adaptive target detection method for high-angle tilted viewpoints, including:

[0066] Step S1: Acquire road scene images from a high-angle tilt view and detect the available hardware resources of the computing device;

[0067] Step S2: Select the slice size, overlap rate, and batch size based on the detected available hardware resources, and generate a slice grid sequence covering the entire image based on the image resolution;

[0068] Step S3: Retrieve slices from the slice grid sequence in batches, call the target detection model to perform forward inference, and obtain the rotating box detection box corresponding to each slice. During the inference process, monitor the memory usage rate simultaneously. If the usage rate exceeds the preset threshold, immediately issue a slice size reduction instruction or an inference backend switching instruction, and re-plan the grid parameters of the remaining unprocessed slices according to the instruction.

[0069] Step S4: Collect the rotated bounding boxes output from each slice, and use the weighted nonmaximum suppression algorithm to merge the candidate boxes in the overlapping areas to obtain the merged full-image bounding box set;

[0070] Step S5: Traverse the entire image detection box set, perform periodic correction of the rotation box angle, for motor vehicle targets, retrieve the pre-constructed road geometry topology direction field data, verify the distance between the predicted angle and the circumferential angle of the direction field, if the deviation exceeds the preset angle threshold, then apply a penalty to the confidence of the detection box, and output the detection box set after angle correction and confidence adjustment.

[0071] Step S6: Combine the detection box set and divide the image into at least two regions distributed along the vertical direction based on the ordinate of the center point of the detection box. Apply differentiated confidence thresholds and non-maximum suppression cross-union ratio thresholds to different regions, perform partitioned non-maximum suppression, and output the final detection result.

[0072] This embodiment employs a memory-adaptive dynamic slicing mechanism that intelligently matches slice size, overlap rate, and batch size from a pre-calibrated tier table based on the device's real-time available video memory. Combined with a real-time degradation strategy for video memory protection, this significantly reduces video memory overflow rate across a full spectrum of devices, from high-end graphics cards to CPUs without dedicated video memory, ensuring extreme system stability. Furthermore, a scale-partitioned adaptive thresholding mechanism dynamically adjusts the confidence level and non-maximum suppression cross-union ratio threshold based on the target's longitudinal position. This, combined with perspective enhancement for distant features and angle-constrained filtering for near-field false positives, collaboratively overcomes the detection challenge caused by significant target scale differences within a single frame at high-angle viewing angles. On a dedicated intersection dataset, this significantly improves the detection accuracy of small targets while ensuring high accuracy under dense near-field occlusion and high recall for small distant targets, achieving a balance between accuracy and efficiency.

[0073] like Figure 2 As shown, the multi-scale target adaptive detection method for high-angle tilting perspectives provided in this embodiment is divided into a pre-training offline stage and a subsequent online inference stage. The two stages work together to achieve a balance between detection accuracy and cross-device deployment stability. Before step S1, the following steps are also included:

[0074] Step S0: Generate target detection model weights adapted to high-angle tilted viewpoints, further including the following sub-steps:

[0075] Step S01: Collect the original monitoring images of the road intersection, label each type of traffic participant in the image with an OBB rotating bounding box, and construct a training dataset with precise spatial location and category labels;

[0076] Specifically, original surveillance images of road intersections are collected, and OBB rotating bounding boxes are labeled for six types of traffic targets in the images. The six types of targets include person (pedestrian), e-bike (electric bicycle), bicycle (bicycle), motorcycle (motorcycle), car (car), and bus (bus). A training dataset with OBB annotations is output.

[0077] Step S02: Read the original image and its corresponding OBB annotation, randomly sample the virtual camera parameters, calculate the homography transformation matrix based on the pinhole camera model, perform perspective transformation on the image and synchronously map the vertices of the annotation box to generate enhanced samples simulating different high-angle tilt angles;

[0078] Randomly sample virtual camera parameters: height h ∈ [6, 15] meters, depression angle θ ∈ [30°, 60°], and equivalent focal length f ∈ ​​[500, 2000] pixels (covering the focal length range of common surveillance cameras from wide-angle to telephoto to simulate the perspective convergence differences of different camera models). The corresponding expression is:

[0079] Equation (1)

[0080] In the formula, U represents a uniform distribution. Based on the pinhole camera model, the homography transformation matrix H is calculated, and bilinear interpolation perspective transformation is performed on the training image, while the same coordinate transformation is simultaneously performed on the four vertices of the OBB bounding box.

[0081] Wherein, the world coordinate depth Z corresponding to the vertical pixel coordinate y in the image plane satisfies:

[0082] Equation (2)

[0083] In the formula, ( , The principal point of the image (taken as the image center) is the core element of the homography matrix H. (Controlling longitudinal perspective intensity) and focal length Altitude h, Depression angle All are relevant:

[0084] Equation (3)

[0085] In the formula, The ground sampling distance corresponding to the vertical pixel resolution of the image (determined by h, θ, and f, approximately 0.02-0.05 meters / pixel at close range) is used. The remaining matrix elements are determined by constraining the four corner mapping relationships of the image and maintaining the continuity of the image boundaries.

[0086] like Figure 3 As shown, the above perspective transformation simulates the imaging characteristics of a high-angle viewpoint. The dashed line indicates the vanishing line calculated after the perspective transformation, which divides the image into an upper non-road area (such as the sky and buildings) and a lower road area. This scheme strictly limits subsequent small target copy-and-paste operations to only the road area below the vanishing line, thereby avoiding pasting targets to physically infeasible locations and improving the realism and effectiveness of the training samples. Figure 3 The OBB (skewed bounding box) is used to label targets such as vehicles and pedestrians; the area below the vanishing line is the road area (pasting is allowed), and the area above the vanishing line is the non-road area (pasting is prohibited).

[0087] By randomly sampling the focal length f, the enhanced image covers different degrees of perspective convergence effect, improving the model's generalization ability to different types of surveillance cameras.

[0088] Step S03: Extract the small target clipping region from the training samples, calculate the vanishing line position based on the perspective transformation parameters, limit the pasting region to the ground area, select low-density grid cells as pasting positions, perform scaling and edge blending in coordination with perspective depth, and assign new bounding boxes that conform to the prior angular attributes of the scene, thus obtaining an augmented dataset supplemented with distant small target samples; further, including the following steps:

[0089] Extract small target cropping regions with an area of ​​less than 32×32 pixels from the training set (mainly for pedestrians and non-motorized vehicles);

[0090] Next, calculate the position of the vanishing line based on the perspective transformation parameters. This formula, derived based on a pinhole camera model, represents the vertical coordinate of the sky-ground boundary in the image after perspective transformation, limiting the paste candidate region to the ground region where y > yh.

[0091] Avoid pasting targets onto the sky or building facades;

[0092] The existing target density D of each 32x32 grid cell within the road area is recalculated. In the low-density grid (D<0.3), the pasting position is randomly selected, excluding positions where the overlap area with the existing large target exceeds 30%.

[0093] When pasting, small targets are randomly scaled (0.7-1.3 times, with the scaling factor coordinated with the perspective depth of the pasting position), ±20% brightness dithering is applied, and Gaussian blending edges with σ=2 pixels are used to avoid stitching marks.

[0094] The angle of the new annotation box is taken from the median direction of the angles of similar targets near the pasted location. If there are no similar targets nearby, the median of the global angle distribution of that target category in the training set is taken. Constraints are only imposed when the randomness of the angle may lead to an unreasonable physical orientation.

[0095] Step S04: Input the data-augmented training samples into the single-stage rotated bounding box object detection architecture, iteratively optimize the network weights until convergence, and obtain a detection model file that adapts to high-angle viewing angles and has multi-scale perception capabilities. Using the single-stage rotated bounding box object detection architecture, train the model until convergence using the training samples augmented in S02 and S03, obtaining model weights that can be used for online inference. In each training round, perspective simulation augmentation is performed with a 50% probability, and small target copy-and-paste augmentation is performed with a 30% probability, where the first probability is greater than the second probability.

[0096] The degradation switching logic in the memory-adaptive dynamic slicing inference method is as follows: when a memory overflow error occurs three consecutive times during GPU inference, the system automatically downgrades to the next lower inference level based on a pre-calibrated level table. This includes coordinated adjustments to slice size, batch size, and model accuracy, and recalculates slice parameters without manual intervention. This switching logic is implemented throughout the entire online inference process, ensuring system stability during memory fluctuations.

[0097] After obtaining the model weights through offline training in step S0, the online inference steps S1-S6 are then executed.

[0098] In one embodiment, step S1: acquiring a road scene image from a high-angle tilt view and detecting the available hardware resources of the computing device, specifically includes the following steps:

[0099] Step S11: Read the original video stream collected by the high-level monitoring equipment at the intersection, decode frame by frame to extract single-frame images that meet the preset resolution requirements such as 1920x1080, and obtain the RGB image of the road scene to be detected.

[0100] Step S12: Call the CUDA runtime interface to query the GPU hardware status, deduct the system reserved memory and the model weight resident memory, calculate the remaining available space, and obtain the GPU available memory capacity parameter. If there is no independent GPU device, record this parameter as 0.

[0101] Step S13: Call the operating system kernel interface to read the number of CPU logical cores, and simultaneously identify the types of SIMD vector instruction sets supported by the CPU (covering mainstream instruction sets such as AVX2 and AVX-512) through the CPUID instruction to obtain the CPU capability parameters of the number of CPU cores and instruction set types.

[0102] Step S14: Based on the GPU available memory capacity parameters and CPU capability parameters, traverse the preset configuration mapping table, select the inference backend, model accuracy and model size that are suitable for the current hardware, and obtain the complete hardware capability detection results.

[0103] The specific mapping relationship of the end-to-end inference parameters is shown in Figure 1:

[0104] Table 1 End-to-End Adaptive Configuration Table

[0105]

[0106] The matching logic for inference configuration tiers follows the principle of prioritizing video memory capacity: when the video memory capacity is not less than 6GB, it is matched with a GPU half-precision (FP16) inference backend and medium-to-large models; when the video memory capacity is between 4GB and 6GB, it is matched with a GPU FP16 backend and medium models; when the video memory capacity is between 2GB and 4GB, it is matched with a GPU FP16 / FP32 backend and small models; when the video memory capacity is less than 2GB or there is no dedicated GPU, it is matched with a CPU inference backend (INT8 quantization or FP32) and lightweight models.

[0107] This embodiment automatically detects all hardware parameters of the computing device and matches and optimizes the inference configuration. It can adapt to the full spectrum of hardware devices, from high-end GPUs to pure CPUs, without the need for on-site manual parameter tuning by technicians. This provides accurate data support for subsequent dynamic slicing parameter selection and adaptive switching of the inference backend, effectively reducing system deployment costs and improving the stability of cross-device operation.

[0108] In one embodiment, step S2: selecting the slice size, overlap rate, and batch size based on the detected available hardware resources, and generating a slice grid sequence covering the entire image based on the image resolution, specifically includes the following steps:

[0109] The specific video memory location mapping relationship is shown in Table 2. The location parameters in this table are calibrated by actual measurement on the corresponding video memory device using the target model. The single-location video memory usage does not exceed 70% of the available video memory.

[0110] Table 2 Display Archive Location Parameter Table

[0111]

[0112] Step S21: Retrieve the available video memory capacity, number of CPU logical cores, and SIMD instruction set type output by the hardware detection stage, traverse the pre-calibrated video memory location mapping relationship to match the initial slice size, overlap rate, and batch size that are suitable for the current computing power conditions, and obtain the initial slice configuration parameters.

[0113] Among them, the tier matching prioritizes selecting the corresponding configuration based on the capacity range of available video memory. When there is no independent graphics processor, it automatically matches the low computing power configuration corresponding to the central processing unit inference. The overlap rate decreases as the slice size decreases: the larger the slice size, the higher the probability of the target crossing the boundary, and the higher the overlap rate is configured to ensure the integrity of the target. When the slice size is small, the overlap rate is reduced to reduce redundant calculations.

[0114] Step S22: Combine the initial slice configuration parameters with the width and resolution of the road scene image to calibrate the matching relationship between slice size and overlap rate, and obtain a slice parameter set that adapts to the current image size.

[0115] When the available video memory is within the capacity range of adjacent tiers, the slice size is determined by linear interpolation between the corresponding values ​​of adjacent tiers, and the interpolation result must not exceed the S value of the next higher tier (conservative principle). The conservative configuration principle is followed: the overlap rate O between tiers decreases as S decreases; larger slices have a higher probability of targets crossing boundaries, requiring a higher overlap rate to ensure no omissions; smaller slices can have a lower overlap rate to reduce redundant calculations. The corresponding linear interpolation formula is:

[0116] Equation (4)

[0117] In the formula, V represents the currently detected available video memory capacity. , These are the memory capacity thresholds corresponding to adjacent low and high memory levels, respectively. , These are the slice sizes corresponding to adjacent low and high grades, respectively.

[0118] Step S23: Based on the calibrated slice parameter group and image resolution, generate a sequence of slice windows covering the entire image without omissions according to the set step size, and output the slice grid sequence.

[0119] The slicing step size is calculated based on the difference between the slice size and the overlap rate, and the corresponding formula is:

[0120] Equation (5)

[0121] In the formula, S is the calibrated slice size, and O is the corresponding overlap rate.

[0122] The number of slices along the width and height of the image is determined by rounding up, as shown in the formulas:

[0123] Equation (6)

[0124] In the formula, W is the image width and H is the image height. This is a round-up operator, ensuring the slice covers the entire image edge region. When (WS) or (HS) is less than or equal to zero (i.e., the input image size is less than or equal to the slice size), n is taken. x = 1 or n y = 1, meaning only one slice is generated, and the image is not segmented.

[0125] Preferably, if the slicing window slides beyond the image boundary, only the valid area overlapping with the image is retained, and the invalid pixels of the excess portion are cropped.

[0126] like Figure 4As shown, the memory guard mechanism operates as follows: After system startup, the initial slice parameters are first determined by looking up a table based on the hardware detection results. During batch inference, the memory usage rate is continuously monitored. If the usage rate exceeds a preset threshold (e.g., 85%), a first-level degradation is triggered, dynamically reducing the size of the remaining slices. If a memory overflow (OOM) exception occurs, a second-level degradation is triggered, switching to a lower parameter configuration. This closed-loop feedback mechanism ensures the continuity of the inference process across different hardware platforms.

[0127] Performance Verification: On low-end devices (Intel J4125 without a dedicated graphics card), the fixed slicing scheme (B=4) resulted in a memory overflow rate of 73%. This invention's solution reduces the overflow rate to below 5%, achieving a frame rate of 4.8 FPS, meeting real-time monitoring requirements. On mid-range devices (GTX1650 4GB), this solution achieves a frame rate of 7.3 FPS, satisfying near real-time monitoring needs. On high-end devices (RTX3060 12GB), with a 960 slice size selected from the lookup table, the frame rate reaches 15 FPS, supporting multi-channel parallel video processing.

[0128] This embodiment employs a dual adaptation mechanism of hardware resources and image resolution. When there is sufficient video memory, it matches a larger slice size to improve inference throughput. When video memory is limited, it automatically adjusts parameters to avoid overflow and crash. The generated slice grid sequence completely covers the entire image without redundant calculations, providing a foundation for the stability and efficiency of subsequent inference processes.

[0129] In one embodiment, step S3: Retrieve slices from the slice grid sequence in batches, call the target detection model to perform forward inference, and obtain the rotating bounding box detection box corresponding to each slice. During the inference process, the memory usage is monitored synchronously. If the usage exceeds a preset threshold, a slice size reduction command or an inference backend switching command is issued immediately, and the grid parameters of the remaining unprocessed slices are replanned according to the command. Specifically, this includes the following steps:

[0130] Step S31: Based on the generated slice grid sequence and batch size parameters, extract slice data in sequence and send it into the deep learning inference engine to obtain the original rotated box detection set corresponding to each slice.

[0131] Before being fed into the inference engine, the sliced ​​data within a batch undergoes normalization preprocessing, mapping pixel values ​​to the numerical distribution range used during model training, thus eliminating the impact of input data distribution differences on inference accuracy. The inference engine loads model weights in the corresponding format based on the inference backend determined by the hardware detection process: asynchronous execution is enabled in GPU inference mode, and a multi-threaded parallel computing mechanism is enabled in CPU inference mode, fully utilizing hardware computing resources.

[0132] Step S32: During the forward propagation process of the inference engine, the allocation and release status of the graphics processor's video memory is collected in real time, the video memory occupancy rate at the current moment is calculated, and the dynamically updated video memory status monitoring value is output.

[0133] Step S33: Compare the real-time acquired memory usage monitoring value with the preset 85% usage threshold, and determine whether to generate a slice size reduction instruction or an inference backend switching instruction based on the comparison result, and output the corresponding control instruction signal.

[0134] Specifically, when the GPU memory utilization exceeds 85%, a slice size reduction instruction is generated first to reduce the size of the remaining unprocessed slices by 10% (not lower than the current minimum threshold), clear the GPU compute cache, and maintain the current inference backend unchanged to ensure inference efficiency. If a memory overflow exception (CUDAOOM) is still captured in subsequent inference after the slice size reduction instruction is issued, it is determined that the current GPU memory location cannot meet the running requirements, and an inference backend degradation instruction is generated: if a memory overflow still occurs after 3 consecutive secondary degradations, the system automatically switches to the next inference backend (FP16→FP32→CPU).

[0135] Step S34: In response to the received control command signal, perform parameter replanning on the remaining slices that have not yet been sent to the inference engine, adjust the slice size or switch the inference backend according to the command type, and output the updated slice grid sequence to continue the inference process.

[0136] When the slice size reduction instruction is triggered, the size of the remaining slices is reduced by 10%, and the reduced size is not lower than the lower limit of the current video memory location. The slice step size is also updated synchronously to re-cover the entire image. When the inference backend switch instruction is triggered, the current graphics processor's computing cache is cleared, all allocated video memory resources are released, the target inference backend's operating environment is reinitialized, and the slice parameters are matched to the corresponding level according to the computing power characteristics of the new backend.

[0137] Specifically, after the parameter reprogramming is completed, the first unprocessed slice is extracted from the updated slice grid sequence to resume the inference process and ensure the continuity of the single-frame image processing.

[0138] Step S35: Summarize the original rotated bounding box detection set of all slices in the current batch as the input data source for the subsequent nonmaximum suppression algorithm, and complete the data aggregation processing for single-batch inference.

[0139] The detection box data synchronously records the slice index to which it belongs and its coordinate position within the slice, providing positional reference information for subsequent merging of detection boxes in overlapping areas across slices.

[0140] This embodiment achieves real-time linkage and control between the inference process and the memory status. When the memory usage approaches the critical value, it quickly reduces the slice load or switches the computing backend. From the operating mechanism, it eliminates the risk of program crash caused by memory overflow. It can maintain a stable real-time detection frame rate on a full spectrum of hardware from high-end graphics cards to low-end embedded devices without manual intervention, which greatly improves the robustness of the system in complex deployment environments.

[0141] In one embodiment, step S4: Collect the rotated bounding boxes output from each slice, and use a weighted non-maximum suppression algorithm to merge candidate boxes in overlapping regions to obtain a merged full-image bounding box set. This specifically includes the following steps:

[0142] Step S41: Aggregate the full sliced ​​rotated bounding box detection data output from each batch of forward inference, remove invalid boxes with confidence scores below the global minimum threshold, and sort them to obtain the initial candidate box set to be merged.

[0143] Step S42: Traverse the initial candidate box set, identify candidate box groups that overlap across slices, calculate distance weights based on the center position of the slice to which the candidate box belongs, and filter to obtain a weighted and corrected subset of candidate boxes.

[0144] The distance weight is calculated based on the Euclidean distance from the center point of the candidate box to the center of its corresponding slice, and the corresponding expression is: In the formula, d is the Euclidean distance from the center of the candidate box to the center of its corresponding slice. This represents the maximum possible distance from the center of the candidate box within the current slice to the center of the slice. The weight value ranges from 0 to 1, with the weight decreasing as the distance increases.

[0145] Specifically, for candidate boxes that belong to multiple overlapping slice regions, their distance weights in each slice are calculated, and the maximum value is taken as the final weight of the candidate box to ensure the fairness of the selection of overlapping region boxes.

[0146] Step S43: Perform non-maximum suppression operation on the weighted and corrected candidate box subset, sort in descending order of confidence, and iteratively filter high-confidence non-overlapping boxes to obtain the merged full-image detection box set.

[0147] In the iterative process of non-maximum suppression, the intersection-union ratio (IUR) of each candidate box of the same category with the current candidate box with the highest confidence is calculated. If the IUR is greater than a preset IUR threshold, it is suppressed, and only the candidate box with the highest confidence is retained. During the screening process, the original confidence of the candidate box is multiplied by the corresponding distance weight to obtain a weighted confidence, which is used as the priority basis for sorting and screening, avoiding duplicate detection in overlapping areas.

[0148] Preferably, for candidate boxes at the edge of the slice, their bounding boxes are additionally checked to see if they exceed the effective area of ​​the image. Invalid boxes that exceed the bounding box area are cropped to improve the positional accuracy of the detection boxes in the whole image.

[0149] This embodiment uses a weighted nonmaximum suppression mechanism to adaptively filter overlapping candidate boxes generated by slice inference, effectively eliminating the problem of repeated detection caused by the fixed slice strategy. While retaining effective detection information of small targets at a distance, it suppresses redundant box output in dense near-field areas, providing an accurate full-image detection data foundation for subsequent rotation box angle correction and partition threshold processing, and further improving the recall rate and reliability of multi-scale target detection.

[0150] In one embodiment, step S5: Traverse the entire image detection box set, perform periodic correction of the rotation box angle, for vehicle targets, retrieve pre-constructed road geometry topology orientation field data, verify the distance between the predicted angle and the circumferential angle of the orientation field, if the deviation exceeds a preset angle threshold, apply a penalty to the confidence of the detection box, and output the detection box set after angle correction and confidence adjustment, specifically including the following steps:

[0151] Step S51: Receive the set of full-image rotated bounding boxes output after weighted nonmaximum suppression processing, verify the width and height parameters of each bounding box one by one, perform angle rotation and size replacement on the bounding boxes with inverted long and short sides, and output the set of bounding boxes that have completed the periodic preliminary correction.

[0152] Among them, the rotation box angle is uniformly normalized to the range of 0 to 180 degrees to eliminate the representation differences caused by different model output formats.

[0153] Specifically, for a single detection box, its predicted width, predicted height, and predicted angle are extracted. When the predicted width value is less than the predicted height value, it is determined that the model has misrepresented the physical long side as the height direction. At this time, the predicted angle is increased by 90 degrees, and the values ​​of the predicted width and predicted height are exchanged simultaneously to ensure that the angle definition is consistent with the physical long axis of the target.

[0154] Step S52: Traverse the set of preliminarily corrected detection boxes, filter out the target subset (covering small passenger cars, buses and freight trucks) whose category attributes belong to the motor vehicle category, extract the spatial coordinates of each target in the subset, and output the sequence of motor vehicle detection boxes to be verified for topology constraints.

[0155] Among them, non-motorized vehicle targets and pedestrian targets only undergo the aforementioned periodic angle correction, without being subject to road geometric topology direction field constraints, and are directly included in the set to be output, in order to adapt to the flexibility of non-motorized vehicle and pedestrian movement trajectories.

[0156] Step S53: For each target in the vehicle detection box sequence, retrieve the pre-constructed road geometry topology direction field data based on the center pixel coordinates, obtain the set of allowed vehicle driving directions at that location, and output the expected direction parameters corresponding to the target location.

[0157] Among them, the road geometric topology direction field data is extracted by offline lane line detection algorithm or pre-generated by manual calibration, and mapped and stored as a function relationship in the image domain: in the case of cross intersection or T-junction, the direction field is represented by discrete reference direction angles such as {0°, 90°, 45°, 135°}; in the case of roundabout or ramp segment, the direction field is represented by a tangent direction function that changes continuously with spatial position.

[0158] Step S54: Calculate the circumferential angular distance between the predicted angle and the desired direction parameter of the vehicle detection frame, compare the distance value with the preset 30° deviation threshold, adaptively reduce the confidence score of the detection frame based on the comparison result, and output a set of detection frames with corrected confidence.

[0159] The calculation of the circumferential angular distance follows the metric criteria of angular cycles, and the corresponding formula is:

[0160] Equation (7)

[0161] In the formula, The predicted angle of the detection box. Represents the allowed direction angle in the direction field. To calculate the minimum difference after taking the modulus of 180 degrees, For position The set of allowed directions at a given location (the nearest direction is taken from the discrete direction set, and the direction at that location is taken directly from the continuous direction field).

[0162] Specifically, when the calculated minimum angular distance exceeds 30°, the vehicle orientation represented by the detection box is determined to physically violate road traffic rules. Its confidence value is multiplied by a penalty coefficient of 0.5 to reduce its priority in the subsequent non-maximum suppression process, rather than being directly deleted, thus preserving the ability to detect illegally parked or accident-damaged vehicles. In typical application scenarios at intersections, the direction field can be simplified to a discrete set containing 0°, 90°, 45°, and 135°, thereby reducing the computational complexity of online retrieval and improving the system's real-time response efficiency.

[0163] Step S55: Integrate all motor vehicle detection boxes, whether confidence-adjusted or not, as well as non-motor vehicle and pedestrian detection boxes that have not undergone angle constraint processing. Reorganize the data structure according to the original image index order and output the final set of full-image detection boxes after angle correction and confidence adjustment.

[0164] Figure 5 (a) is a schematic diagram of the prediction block before correction provided in an embodiment of this application;

[0165] Figure 5 (b) is a schematic diagram of the corrected prediction box provided in the embodiment of this application;

[0166] Figure 5 (c) is a schematic diagram of the orientation field constraint provided in an embodiment of this application;

[0167] Figure 5 (d) is a schematic diagram of the continuous direction field of a roundabout provided in the embodiment of this application.

[0168] like Figure 5 As shown, the specific logic of the road geometry topology orientation field constraint is as follows: For motor vehicle targets, their predicted angle (solid arrow in the figure) is extracted and compared with the allowed driving direction of the road orientation field at that location (dashed arrow in the figure). By calculating the circumferential angular distance between the two (the minimum difference after modulo 180 degrees), it is determined whether the predicted angle conforms to the road geometry constraint. If the deviation is too large (e.g., exceeding 30°), a penalty is applied to the confidence of the detection box (multiplied by a penalty coefficient), thereby filtering out physically unreasonable false detection boxes such as vehicles lying across the road surface, and improving the scene fit of the detection results.

[0169] Preferably, in typical application scenarios at intersections, the direction field can be simplified to a discrete set containing 0°, 90°, 45° and 135°, thereby reducing the computational complexity of online retrieval and improving the real-time response efficiency of the system.

[0170] This embodiment eliminates the 90-degree jump error caused by aspect ratio changes in the rotated frame detection model by introducing an angle periodic correction mechanism, significantly improving the alignment accuracy between the detection frame and the physical target. At the same time, it applies physical constraints on the vehicle orientation by using the road geometric topology orientation field, effectively filtering out false detection frames that do not conform to traffic rules, such as vehicles lying across the road. Without losing the ability to detect abnormal vehicles, it greatly reduces the false positive rate in the near-field area, providing highly reliable data input for subsequent partition threshold processing.

[0171] In one embodiment, step S6: Divide the image into at least two regions distributed along the vertical direction based on the ordinate of the center point of the detection box, apply differentiated confidence thresholds and non-maximum suppression cross-union ratio (CUI) thresholds to different regions, perform partitioned non-maximum suppression, and output the final detection result. This specifically includes the following steps:

[0172] Step S61: Receive the set of full-image detection boxes after angle correction and confidence adjustment, extract the center point ordinate and corresponding category attributes of each detection box, and construct a candidate box dataset for partition threshold processing.

[0173] The extraction of the center point's ordinate is based on the geometric position of the detection box in the image coordinate system, with the origin at the top of the image. The ordinate value gradually increases as the target moves closer, directly reflecting the target's depth information from a perspective viewpoint. The candidate box dataset inherits the confidence scores and geometric parameters from previous steps during its construction, ensuring that subsequent thresholding steps possess complete target feature information.

[0174] Step S62: Divide the image into at least two detection regions along the vertical direction based on the total image height, determine the differential confidence threshold and non-maximum suppression cross-union ratio threshold corresponding to each region, and output the partition threshold mapping relationship.

[0175] The vertical region division of the image follows the perspective imaging rules. The image height is denoted as H, and the region is assigned based on the range of the y-coordinate of the center point. The corresponding formula is:

[0176] Equation (8)

[0177] Specifically, the distant view area correspond Mid-field area correspond Close-up area correspond A confidence threshold of 0.15 is used in the distant region to ensure the recall rate of small distant targets; a non-maximum suppression cross-union ratio (CUI) threshold of 0.7 is used in the near region to address false suppression caused by dense vehicle occlusion. The threshold combinations adapted to different regions are determined based on the target scale characteristics. Perspective simulation enhancement magnifies distant targets, improving the feature quality of small distant targets, so that the low threshold (0.15) can effectively detect real targets in the distant region without introducing a large amount of noise; dynamic slicing ensures that the target scale distribution within the slice is relatively uniform, and the application of the partition threshold within the slice is more stable; angle constraints filter out unreasonable rotation box false positives in the near region, so that the high threshold in the near region can more accurately suppress false positives without mistakenly deleting real targets. See Table 2 for specific configurations:

[0178] Table 3 Partition Threshold Parameter Table

[0179]

[0180] like Figure 6 As shown, this illustrates the intuitive effect of dividing the detection region based on the vertical depth (vertical coordinate) of the image. From bottom to top (vertical coordinate from largest to smallest), the image corresponds to the near, middle, and far fields, respectively. Due to the perspective effect under a high-angle viewing angle, small targets at the far end have weak features and low confidence, while large targets at the near end are severely occluded and prone to false detection. Therefore, as... Figure 6As shown in the threshold curve on the right, this scheme applies a lower confidence threshold (e.g., 0.15) to distant areas to ensure recall, and a higher non-maximum suppression cross-union ratio (CUI) threshold (e.g., 0.7) to near areas to combat occlusion. This partitioned adaptive thresholding strategy effectively solves the problem that a globally uniform threshold cannot take into account multi-scale targets. Figure 6 In this context, `conf` represents the confidence threshold, and `loU` represents the overlap threshold for non-maximum suppression. The thresholds gradually increase with scale, from far to near.

[0181] Step S63: For candidate boxes located near the region boundary, perform linear interpolation based on the relative position of their ordinates on the adjacent region boundary to calculate the continuously changing dynamic confidence threshold and dynamic non-maximum suppression intersection-union ratio threshold. Both are used to perform partitioned non-maximum suppression. The non-maximum suppression intersection-union ratio threshold is calculated using the same interpolation logic as the confidence threshold to avoid detection jitter caused by hard threshold switching. The interpolated threshold is directly used to perform partitioned non-maximum suppression.

[0182] The interpolation calculation of the dynamic confidence threshold ensures that the threshold does not change abruptly during area handover, and the corresponding formula is:

[0183] Equation (9)

[0184] In the formula, and For adjacent detection areas, and The ordinates of the boundary of the corresponding region. Represents the confidence threshold. Non-maximum suppression cross-union ratio threshold. The calculations are performed using the exact same interpolation formula and boundary parameters, which will not be repeated here.

[0185] Specifically, the interpolation operation is performed independently for each candidate box, assigning a personalized threshold parameter based on its precise vertical coordinate position, rather than simply using a fixed threshold based on the main region to which it belongs.

[0186] Step S64: Traverse the candidate box dataset, match the corresponding dynamic threshold based on the ordinate of the center point of each candidate box, perform partition non-maximum suppression operation, filter redundant and low-quality detection boxes, and output the final set of filtered detection results.

[0187] The execution logic of non-maximum suppression is as follows: candidate boxes are sorted in descending order of confidence, the box with the highest confidence is selected as the benchmark, and the intersection-union ratio (IU / R) of the remaining boxes with the benchmark box is calculated. If the IU / R of the remaining boxes exceeds the dynamic IU / R threshold of the benchmark box's location, then a suppression operation is performed on the redundant box. The suppression operation only reduces the confidence of the redundant box or marks it as invalid; it does not directly delete the original data so that the subsequent tracking module can call historical frame information for trajectory repair.

[0188] Step S65: Integrate all valid detection boxes retained after partitioning nonmaximum suppression, rearrange them according to their spatial location in the image, and generate the final target detection output sequence for a single frame image.

[0189] The output sequence includes the category, location coordinates, width and height, rotation angle, and final confidence value after partition threshold correction for each target, which is directly connected to downstream traffic flow statistics, violation recognition, or signal timing optimization application modules.

[0190] This embodiment effectively solves the detection contradiction caused by drastic changes in target scale within a single frame under a high-angle perspective by adaptively matching differentiated thresholds based on the target's longitudinal depth. It significantly improves the recall rate of small targets in distant areas and effectively suppresses false positives under dense occlusion in near areas. Combined with the linear interpolation strategy of the region boundary, it eliminates the detection jitter caused by hard threshold switching. While ensuring the real-time inference frame rate, it achieves high-precision perception of multi-scale targets across the entire map.

[0191] Secondly, this application provides a multi-scale adaptive target detection system for high-angle tilted viewpoints, comprising:

[0192] The hardware detection module is used to detect the available GPU memory capacity, the number of CPU logical cores, and the SIMD vector instruction set when the system starts up, and select the inference backend and model accuracy based on the detection results;

[0193] The dynamic slicing module is used to look up the slice size, overlap rate and batch size from the pre-calibrated parameter range table based on the available video memory output by the hardware detection module, generate a slice mesh, and call the inference execution module to perform target detection on each slice;

[0194] The memory guard module is used to monitor the GPU memory usage in real time during inference. When the usage exceeds the preset usage threshold, it triggers a reduction in slice size. When memory overflow occurs, it triggers a downgrade or switches the inference backend.

[0195] The rotation box angle correction module receives the set of detection boxes output by the inference execution module, performs periodic angle correction on the detection boxes, and performs angle rationality verification and confidence penalty on motor vehicle targets based on the road geometry topology orientation field. For non-motorized vehicles (bicycle / motorcycle / e-bike) and pedestrian targets, no road geometry topology orientation field constraints are applied; only the aforementioned periodic angle correction is performed, and they are directly added to the output set. The input of this module is the merged OBB detection box set, and the output is the detection box set after angle correction and confidence adjustment.

[0196] The partition threshold module is used to receive the set of detection boxes processed by the rotation box angle correction module, divide the image into at least two detection regions distributed along the longitudinal direction according to the center ordinate of the detection box, and perform partition non-maximum suppression using different confidence thresholds and non-maximum suppression cross union ratio thresholds respectively.

[0197] The inference execution module is used to call the deep learning inference engine to perform forward propagation on each slice and return the rotated box detection box.

[0198] Further optimizations include a training augmentation module for performing perspective simulation data augmentation and small target copy-and-paste augmentation with road region constraints during offline training.

[0199] The perspective simulation data augmentation randomly samples virtual camera height h∈[6, 15] meters, depression angle θ∈[30°, 60°], and equivalent focal length f∈[500, 2000] pixels (covering the focal length range of common surveillance cameras from wide-angle to telephoto) from the training samples. Based on the pinhole camera model, it calculates the homography transformation matrix and performs bilinear interpolation perspective transformation, while simultaneously performing the same coordinate transformation on the four vertices of the rotating box annotation.

[0200] The small target copy-paste enhancement with road area constraints calculates the vanishing line position based on perspective transformation parameters. The pasting candidate area is limited to the ground area below the vanishing line; small target clipping areas with an area of ​​less than 32×32 pixels are extracted from the training set (mainly for pedestrians and non-motorized vehicles), and pasting positions are randomly selected in a low-density grid (D<0.3), excluding positions with an overlap area of ​​more than 30% with existing large targets; when pasting, small targets are randomly scaled (0.7-1.3 times, with the scaling factor coordinated with the perspective depth of the pasting position), and brightness jittered (±20%), and Gaussian blending edges (σ=2 pixels) are used to avoid splicing marks; the angle of the new annotation box is preferentially taken from the median direction of the angles of similar targets near the pasting position, if there are no similar targets nearby, the median of the global angle distribution of the target category in the training set is taken, or the original angle is kept unchanged.

[0201] A further optimization scheme is that the hardware detection module automatically configures the end-to-end inference parameters based on the detected available GPU memory capacity, CPU logical core count, and SIMD vector instruction set. The end-to-end inference parameters include model accuracy, model size, inference resolution, slicing parameters, batch size, and number of threads. When a memory overflow error occurs multiple times consecutively during GPU inference, the system automatically downgrades to the next lower-level inference configuration.

[0202] The functions of each module in the above-mentioned multi-scale target adaptive detection system for high-angle tilting perspective correspond to the steps in the above-mentioned multi-scale target adaptive detection method embodiment for high-angle tilting perspective. Their functions and implementation processes will not be described in detail here.

[0203] Thirdly, embodiments of this application provide a multi-scale target adaptive detection device for high-angle tilting viewpoints. The multi-scale target adaptive detection device for high-angle tilting viewpoints can be a personal computer (PC), laptop computer, server, or other device with data processing capabilities.

[0204] The communication bus can be of any type and is used to interconnect the processor, memory, and communication interface.

[0205] The communication interface includes input / output (I / O) interfaces, physical interfaces, and logical interfaces. These interfaces enable interconnection of internal components within the multi-scale target adaptive detection device with a high-angle tilt view, and also enable interconnection between the device and other devices (such as other computing devices or user equipment). Physical interfaces can be Ethernet interfaces, fiber optic interfaces, ATM interfaces, etc.; user equipment can be displays, keyboards, etc.

[0206] Memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0207] The processor can be a general-purpose processor, which can call a multi-scale target adaptive detection program for high-angle tilting view stored in memory and execute the multi-scale target adaptive detection method for high-angle tilting view provided in the embodiments of this application. For example, the general-purpose processor can be a central processing unit (CPU).

[0208] The method executed when the multi-scale target adaptive detection program for high-angle tilting view is invoked can be referred to in the various embodiments of the multi-scale target adaptive detection method for high-angle tilting view of this application, and will not be repeated here.

[0209] Fourthly, embodiments of this application also provide a readable storage medium.

[0210] This application stores a multi-scale adaptive target detection program for high-angle tilting viewpoints on a readable storage medium.

[0211] When the processor executes the multi-scale target adaptive detection program for high-angle tilting view, it implements the steps of the multi-scale target adaptive detection method for high-angle tilting view as described above.

[0212] The method implemented when the multi-scale target adaptive detection program for high-angle tilting view is executed can be referred to in the various embodiments of the multi-scale target adaptive detection method for high-angle tilting view of this application, and will not be repeated here.

[0213] It should be noted that the sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0214] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-scale adaptive target detection method for high-angle tilted viewpoints, characterized in that, include: Acquire road scene images from a high-angle, tilted viewpoint and detect the available hardware resources of the computing device; Based on the detected available hardware resources, the slice size, overlap rate, and batch size are selected, and a slice grid sequence covering the entire image is generated according to the image resolution. The slices in the slice grid sequence are retrieved in batches, and the target detection model is called to perform forward inference to obtain the rotating box detection box corresponding to each slice. During the inference process, the memory usage is monitored synchronously. If the usage exceeds the preset threshold, a slice size reduction command or an inference backend switching command is issued immediately, and the grid parameters of the remaining unprocessed slices are replanned according to the command. The rotated bounding boxes output from each slice are collected, and the candidate boxes in the overlapping areas are merged using a weighted nonmaximum suppression algorithm to obtain a merged set of full-image bounding boxes. Traverse the entire set of detection boxes, perform periodic correction of the rotation box angle, and for motor vehicle targets, retrieve the pre-constructed road geometry topology orientation field data, verify the distance between the predicted angle and the circumferential angle of the orientation field. If the deviation exceeds the preset angle threshold, apply a penalty to the confidence of the detection box, and output the set of detection boxes after angle correction and confidence adjustment. Based on the set of detection boxes, the image is divided into at least two regions distributed along the vertical direction according to the ordinate of the center point of the detection box. Different confidence thresholds and non-maximum suppression cross-union ratio thresholds are applied to different regions, and partitioned non-maximum suppression is performed to output the final detection result.

2. The multi-scale adaptive target detection method for high-angle tilting viewpoints according to claim 1, characterized in that, The step of selecting the slice size, overlap ratio, and batch size of the dynamic slice in response to the detected available hardware resources specifically includes: Query the available video memory capacity of the computing device's GPU. When there is no dedicated GPU, the available video memory capacity is recorded as 0. Match the available video memory capacity with the preset memory location range in the pre-calibrated video memory location table, and select the corresponding slice size, overlap ratio, and batch size. The larger the video memory, the larger the slice size can be. When the available video memory does not fall exactly on the boundary of the level, the slice size is determined by linear interpolation between adjacent levels, and the interpolation result must not exceed the upper limit of the slice size of the next higher level. During inference, after each batch is processed, the current memory usage is checked. When the usage exceeds the preset threshold, a first-level degradation is triggered, reducing the size of the remaining unprocessed slices by a preset ratio, ensuring it is not lower than the current minimum threshold, and clearing the GPU compute cache. If a memory overflow exception still occurs, a second-level degradation is triggered, switching the remaining slices to the next lower threshold. If a memory overflow still occurs after multiple second-level degradations, the system automatically switches to the next inference backend.

3. The multi-scale adaptive target detection method for high-angle tilted viewpoints according to claim 1, characterized in that, The step of traversing the entire image detection box set and performing periodic correction of the box rotation angle specifically includes: For each rotating frame detection box, when its predicted width is less than its predicted height, the predicted angle is corrected to the original predicted angle plus 90 degrees, and the width and height are swapped. After correction, the angle is normalized to the range of 0 to 180 degrees. For motor vehicle targets, query the road direction field at the target's location in the image. The road direction field is obtained in advance by lane line detection or manual calibration. In the case of a crossroads, the road direction field is a discrete set of directions, while in the case of a roundabout or ramp, the road direction field is a continuous direction function. The minimum difference between the predicted angle and the allowed direction of the road direction field modulo 180 degrees is calculated as the circumferential angle distance. When the circumferential angle distance exceeds the preset angle threshold, the confidence of the vehicle target detection box is multiplied by the preset penalty coefficient. For non-motorized vehicles and pedestrians, only periodic corrections to the rotation frame angle are performed, without applying road direction field constraints.

4. The multi-scale adaptive target detection method for high-angle tilting viewpoints according to claim 1, characterized in that, The process of dividing the image into at least two longitudinally distributed regions based on the ordinate of the center point of the detection box specifically includes: Let the image height be H. Divide the image vertically into at least two detection regions. Among them, the larger the vertical coordinate of the center of the detection box, the higher the confidence threshold and the non-maximum suppression cross-union ratio threshold are for the bottom of the image. For targets located near the region boundary, a continuous threshold is calculated by linear interpolation using the target center's ordinate. Non-maximum suppression is performed using the interpolated threshold.

5. The multi-scale adaptive target detection method for high-angle tilted viewpoints according to claim 1, characterized in that, It also includes perspective simulation data augmentation steps during the training phase: The training samples are randomly sampled for virtual camera height, tilt angle, and equivalent focal length, and the sampling range of the equivalent focal length covers the common focal length range of surveillance cameras. Based on the pinhole camera model, the homography transformation matrix is ​​calculated, and bilinear interpolation perspective transformation is performed on the training image, while the same coordinate transformation is performed simultaneously on the four vertices of the rotated box annotation. The vanishing line position is calculated based on the perspective transformation parameters. The candidate area for copying and pasting small targets is limited to the ground area below the vanishing line. The pasting position is selected in the low-density grid. When pasting, the small targets are scaled and edge-blended in accordance with the perspective depth. The angle of the new annotation box is taken as the median direction of the angles of similar targets near the pasting position.

6. The multi-scale adaptive target detection method for high-angle tilting viewpoints according to claim 5, characterized in that, In each round of training, perspective simulation enhancement is performed with a preset first probability, and small target copy-paste enhancement is performed with a preset second probability, where the first probability is greater than the second probability.

7. The multi-scale adaptive target detection method for high-angle tilted viewpoints according to claim 1, characterized in that, The method of merging candidate boxes in overlapping regions using a weighted nonmaximum suppression algorithm specifically includes: Within the overlapping area of ​​slices, the detection box that is closer to the center of its respective slice is retained. The confidence of the detection box is multiplied by a distance weight, which decreases as the distance from the target center to the center of its respective slice increases.

8. A multi-scale adaptive target detection system for high-angle tilted viewpoints, characterized in that, include: The hardware detection module is used to detect the available GPU memory capacity, the number of CPU logical cores, and the SIMD vector instruction set when the system starts up, and select the inference backend and model accuracy based on the detection results; The dynamic slicing module is used to look up the slice size, overlap rate and batch size from the pre-calibrated parameter range table based on the available video memory output by the hardware detection module, generate a slice mesh, and call the inference execution module to perform target detection on each slice; The memory guard module is used to monitor the GPU memory usage in real time during inference. When the usage exceeds the preset usage threshold, it triggers a reduction in slice size. When memory overflow occurs, it triggers a downgrade or switches the inference backend. The rotating frame angle correction module is used to receive the set of detection frames output by the inference execution module, perform periodic angle correction on the detection frames, and perform angle rationality verification and confidence penalty on motor vehicle targets based on the road geometric topology orientation field. The partition threshold module is used to receive the set of detection boxes processed by the rotation box angle correction module, divide the image into at least two detection regions distributed along the longitudinal direction according to the center ordinate of the detection box, and perform partition non-maximum suppression using different confidence thresholds and non-maximum suppression cross union ratio thresholds respectively. The inference execution module is used to call the deep learning inference engine to perform forward propagation on each slice and return the rotated box detection box.

9. The multi-scale adaptive target detection system for high-angle tilting viewpoints according to claim 8, characterized in that, It also includes a training augmentation module for performing perspective simulation data augmentation and small target copy-paste augmentation with road area constraints during the offline training phase; The perspective simulation data enhancement randomly samples the virtual camera height, tilt angle, and equivalent focal length of the training samples, calculates the homography transformation matrix based on the pinhole camera model, and performs bilinear interpolation perspective transformation, while simultaneously performing the same coordinate transformation on the four vertices of the rotating box annotation. The small target copy-paste enhancement with road area constraints calculates the vanishing line position based on perspective transformation parameters, limits the paste candidate area to the ground area below the vanishing line, and selects the paste position in the low-density grid.

10. The multi-scale adaptive target detection system for high-angle tilting viewpoints according to claim 8, characterized in that, The hardware detection module also automatically configures the full-link inference parameters based on the detected available GPU memory capacity, CPU logical core count, and SIMD vector instruction set. The full-link inference parameters include model accuracy, model size, inference resolution, slicing parameters, batch size, and number of threads. When a memory overflow error occurs multiple times consecutively during GPU inference, the system will automatically downgrade to the next lower inference configuration.

Images