A vehicle three-dimensional overrun detection method based on virtual-real fusion and edge lightening

Abstract

A kind of vehicle three-dimensional out-of-limit detection method based on virtual-real fusion and edge lightening, utilizes digital twin engine to construct virtual traffic environment aligned with real roadside scene space, generates simulation image with three-dimensional physical true value, adopts cyclic consistency adversarial network to perform style transfer of simulation domain and real domain, and constructs enhanced training dataset;Construct a three-dimensional target detection initial model, and perform model lightening through structured channel pruning and multi-level knowledge distillation of fusion feature level, output level and relationship level;Construct a dual-path parallel solving mechanism of semantic depth regression and ground plane geometric depth derivation, and realize adaptive weighted fusion through depth uncertainty modeling;Based on the fusion depth value, reconstruct the three-dimensional bounding box of the target vehicle in the world coordinate system, and compare with the national standard to automatically identify the out-of-limit behavior.The present application reduces the labeling cost and edge computing power, realizes centimeter-level size measurement, and is suitable for roadside non-site law enforcement scene.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and intelligent transportation technology, and in particular to a method for detecting three-dimensional over-limit vehicles based on virtual-real fusion and edge lightweighting, which can be used for automated over-limit detection of vehicle outline dimensions in roadside non-on-site law enforcement scenarios. Background Technology

[0002] Ranging technology based on monocular imaging geometry and ground plane constraints is the traditional cornerstone of monocular 3D perception. Its core principle utilizes a camera pinhole imaging model and geometric priors within the scene. In intelligent transportation roadside scenarios, it is typically assumed that the monitored area's road surface is an ideally flat plane, and a homography mapping from the image coordinate system to the world coordinate system is constructed based on calibration parameters such as the camera's installation height, pitch angle, and focal length. When the grounding point of a vehicle target is detected, an inverse projection transformation is used to project the pixels back into the world coordinate system, thereby retrieving the distance between the vehicle's center and the camera. While this technology is computationally simple, its high dependence on the accurate positioning of the "grounding point" leads to significant fluctuations in positioning results in complex real-world environments, such as when the vehicle is partially obscured, shadows caused by lighting interference, or the road surface is not perfectly flat. Furthermore, single-path geometric derivation cannot obtain 3D information about the vehicle's top, resulting in substantial errors in height calculation and making it unsuitable for rigorous off-site law enforcement.

[0003] With the development of convolutional neural networks, the technological approach has shifted towards building end-to-end multi-branch regression networks. Multi-branch regression network methods typically add a 3D prediction head to a mature 2D detector to directly learn the physical dimensions of the target, including its length, width, height, and heading angle. During the training phase, large-scale labeled data is used to force the network to learn a complex nonlinear mapping between image semantic features and physical attributes. Although this technique performs well on standard laboratory test sets, it is essentially a data-driven black-box model, lacking intrinsic constraints from physical logic. Under the high-angle view of roadside surveillance, perspective distortion causes near-to-far effect, leading to depth oscillations when predicting small, distant targets. Furthermore, due to a lack of interpretability, the calculation results extracted by this type of method often face insufficient credibility as evidence in legal proceedings due to the lack of rigorous physical logic derivation.

[0004] One existing 3D target detection method integrates a monocular camera and LiDAR, enhancing depth perception through multi-sensor data fusion. However, high-line-count LiDAR equipment is expensive, and point cloud quality deteriorates significantly under adverse weather conditions such as heavy rain and dense fog. Vibration of roadside poles can also cause calibration parameter shifts between sensors. Another example is a monocular 3D target detection method based on instance-level adaptive depth estimation, which achieves depth inference through instance-level depth discretization and adaptive offset branching. However, its depth inference process relies on the mean prior of standard vehicle models, leading to significant penalty dulling for illegally modified or overloaded vehicles whose physical contours deviate from the mean distribution. Yet another example is a Transformer-based monocular 3D target detection method that utilizes a global attention mechanism to construct logical connections in 3D space. However, the Transformer's multi-head attention mechanism has quadratic computational complexity, resulting in excessive inference latency at roadside edge computing units, failing to meet real-time detection requirements.

[0005] Therefore, there is an urgent need for a lightweight, robust, and edge-deployable 3D perception framework that can suppress meteorological interference, break free from mean prior constraints, and compress computational overhead to a threshold that edge devices can bear while ensuring spatial modeling capabilities. Summary of the Invention

[0006] (I) Purpose of the invention: In order to solve the problems existing in the prior art, the purpose of this invention is to provide a vehicle three-dimensional over-limit detection method based on virtual-real fusion and edge lightweighting. Through the whole chain technical solution of virtual-real fusion data enhancement, model lightweighting and compression, and complementary deep fusion perception, it can achieve centimeter-level vehicle size measurement accuracy with legal credibility on low computing power edge devices without expensive sensors such as LiDAR.

[0007] (II) Technical Solution: To solve the above-mentioned technical problems, this technical solution provides a vehicle 3D over-limit detection method based on virtual-real fusion and edge lightweighting, including:

[0008] The parameters of the roadside image acquisition device and the real roadside environment image are obtained. A virtual traffic environment aligned with the real roadside scene space is constructed using a digital twin engine to generate a simulation image with three-dimensional physical truth. The simulation image is used as the source domain and the real roadside environment image is used as the target domain to output an enhanced training dataset.

[0009] A three-dimensional object detection initial model containing a feature extraction backbone network is constructed. Using the enhanced training dataset as training samples, the three-dimensional object detection initial model is structured and subjected to multi-level knowledge transfer to generate a lightweight three-dimensional detection model. The lightweight three-dimensional detection model outputs the two-dimensional detection box of the target vehicle, image semantic features, vehicle touch point location, and predicted values ​​of three-dimensional attributes.

[0010] Based on the lightweight 3D detection model, the target vehicle's image semantic features and vehicle touch point location are output. Combined with the spatial pose parameters of the image acquisition device and the scene reference surface location information, the dual-path parallel solution of semantic depth regression and geometric depth derivation is performed, and adaptive weighted fusion is performed based on the prediction confidence of each path to output the fusion depth value.

[0011] Based on the fusion depth value, the three-dimensional bounding box of the target vehicle in the world coordinate system is reconstructed, and compared with the national vehicle outline size standard in real time, automatically performing the identification of overweight and oversized violations and the extraction of structured evidence.

[0012] Preferably, the Unreal Engine 5's Nanite virtual geometry technology is used to convert real road surface point cloud data and roadside facility CAD models into high-fidelity virtual geometric assets. Through Nanite's micro-polygon streaming mechanism, the geometric topology of the real road segment is restored 1:1, preserving road surface texture details and curb geometric features. The Unreal Engine 5's Lumen dynamic global illumination system is enabled to simulate natural light diffuse reflection, metallic vehicle body reflection, road surface water reflection, and vehicle shadows, and to calculate physically realistic indirect lighting and contact shadows in real time. The engine's weather system is used to batch adjust rainfall, fog concentration, light intensity, and day / night time, matching the virtual camera parameters with the parameters of the real roadside image acquisition equipment.

[0013] In the constructed virtual traffic scene, virtual cameras are set up at corresponding spatial locations in the virtual traffic scene according to the calibration parameters of the actual roadside image acquisition equipment, and lens parameters consistent with those of the roadside image acquisition equipment are configured. Multiple types and models of vehicle 3D geometric models are imported into the virtual traffic scene, and each vehicle model is assigned realistic physical size attributes.

[0014] The weather and lighting parameters in the virtual environment are controlled by automated scripts. Under the combination of weather and lighting parameters, the vehicle model in the virtual traffic scene is driven to drive according to the preset traffic flow trajectory. The virtual camera continuously collects simulation images and exports the three-dimensional physical truth label data of each vehicle target in each frame of the simulation image.

[0015] The three-dimensional physical truth label data includes the center three-dimensional coordinates of the vehicle target in the world coordinate system. Physical length ,width ,high Heading angle .

[0016] Preferably, a recurrent consistency adversarial network is used, with the simulated image as the source domain X and the real roadside environment image as the target domain Y. Through joint optimization of adversarial loss and recurrent consistency loss, the differences in texture style, color distribution and sensor noise between the simulated image and the real roadside environment image are eliminated while preserving the geometric structure of the simulated image, so as to obtain an enhanced training dataset for virtual-real fusion.

[0017] The simulated image constrained by the cycle consistency loss is restored to the original input after forward generation and backward reconstruction, and the real image is restored to the original input after backward generation and forward reconstruction.

[0018] Preferably, the structured pruning includes: applying an L1 regularization penalty term to the scaling factor of the batch normalization layer in the backbone network during the training phase to induce the scaling factor of non-important channels to converge to zero; calculating a global pruning threshold according to a preset pruning ratio; removing redundant channels with scaling factors lower than the global pruning threshold and the corresponding weight parameters of the associated previous convolutional kernel, as well as the corresponding normalization parameters in the next BN layer, to obtain a compact subnetwork.

[0019] Preferably, the multi-level knowledge transfer uses a pre-trained model as the knowledge output and a compact sub-network as the knowledge receiver, including feature-level distillation, output-level distillation, and relation-level distillation.

[0020] The feature-level distillation selects several representative intermediate feature layers from the feature extraction backbone network as supervision layers. For the output layer of the first residual block after each downsampling stage, the pre-trained model and the supervision layer are extracted respectively. Corresponding feature map and the compact subnetwork in the supervision layer Output feature map Introducing transformation operators Channel-dimensional alignment is performed on the feature maps of the compact sub-network. The L2 norm squared of the element-wise difference between the feature maps of the pre-trained model and the transformed feature maps of the compact sub-network is calculated as the feature-level distillation loss. ,in, Characteristic stage distillation loss Refers to the selected supervisory layer index, where For the selected set of supervisory intermediate layers; The feature map of the pre-trained model. The feature map of a compact subnetwork. Transformation operators used for channel dimension alignment refer to Norm;

[0021] The output-level distillation includes soft label alignment for the classification prediction branch and numerical alignment for the 3D attribute regression branch. For the classification prediction branch, the smooth probability distribution of the pre-trained model is used as the soft label. The KL divergence between the smooth probability distribution of the pre-trained model and the smooth probability distribution of the compact sub-network is calculated, and the KL divergence is multiplied by the square of the temperature coefficient T for scaling, which is used as the classification output-level distillation loss. For the 3D attribute regression branch, the key 3D physical attribute vectors of each target vehicle are... The smoothed L1 loss between the pre-trained model predictions and the compact sub-network predictions is calculated separately. The weighted smoothed L1 losses of each attribute dimension are added together to obtain the attribute regression output level distillation loss.

[0022] For multiple vehicle targets in the same input image, the relation-level distillation first extracts the feature vectors corresponding to each target region in the intermediate feature layer of the pre-trained model and the compact sub-network. Based on the feature vectors, the target similarity matrix of the pre-trained model and the target similarity matrix of the compact sub-network are constructed respectively. The element in the m-th row and n-th column of the similarity matrix is ​​the cosine similarity between the m-th target feature vector and the n-th target feature vector. The Frobenius norm between the relation matrix generated by the pre-trained model and the relation matrix generated by the compact sub-network is calculated.

[0023] Preferably, based on the lightweight 3D detection model, the target vehicle's image semantic features and vehicle touch point location are output. Combined with the spatial pose parameters of the image acquisition device and the scene reference surface position information, a dual-path parallel solution is performed using semantic depth regression and geometric depth derivation. Adaptive weighted fusion is then performed based on the prediction confidence of each path to output a fusion depth value, including:

[0024] Based on the semantic features of the output image from the lightweight 3D detection model, a semantic depth regression path is constructed, and the semantic depth prediction value and the logarithmic variance of the semantic depth prediction path are calculated.

[0025] Based on the output of the vehicle touch point location from the lightweight 3D detection model, and combined with the spatial pose parameters of the image acquisition device and the scene reference surface position information, an explicit geometric depth derivation path is constructed, and the explicit geometric depth derivation value and the logarithmic variance of the geometric depth prediction path are calculated.

[0026] Log-variance of semantic deep prediction paths The depth uncertainty of the logarithmic variance of the geometric depth prediction path is used to construct an adaptive gated fusion to obtain the fused depth value.

[0027] Preferably, after the target vehicle image to be detected is input into the lightweight 3D detection model, it is forward propagated through the feature extraction backbone network to output a multi-scale image semantic feature map.

[0028] The multi-scale image semantic feature map is input into the multilayer perceptron (MLP) of the lightweight 3D detection model, and the output is a semantic depth prediction value based on the semantic features. Simultaneously set up a depth uncertainty estimation branch, and output the log-variance of the semantic depth prediction path. .

[0029] Preferably, the lightweight 3D detection model is used to extract the ordinate value of the vehicle contact point in the image pixel coordinate system. ,

[0030] Based on the principles of similar triangles and perspective projection geometry, under the assumption of a ground plane, the offset of the pixel ordinate of the vehicle's contact point relative to the horizon ordinate is inversely proportional to the physical distance from the vehicle to the camera. This yields an explicit geometric depth derivation based on ground plane physical geometric constraints. ,in, This represents the depth value directly calculated based on projective geometric constraints. H represents the vertical pixel focal length of the camera; H represents the installation height of the camera. Represents the vertical coordinate of the vehicle's contact point in the image coordinate system; Represents the ordinate of the vanishing horizon in the image coordinate system;

[0031] Synchronously set up a branch for estimating the geometric path uncertainty, and output the logarithmic variance of the predicted geometric depth path. .

[0032] Preferably, the semantic depth prediction value The explicit geometric depth derivation value All paths are modeled as Laplace distributions, with the log-variance of the semantic depth prediction path as the basis. The negative exponential function value is used as the weight of the original semantic depth prediction confidence score. Log-variance of path prediction based on geometric depth The negative exponential function value is used as the weight of the original geometric depth prediction confidence score. Output fusion depth prediction value ,in, This refers to the fusion depth value after adaptive fusion. Refers to semantic depth prediction value, The value is derived from the explicit geometric depth. This refers to the log-variance of the semantic depth prediction path. The logarithmic variance of the geometric depth prediction path, where e is the natural constant. This represents the logarithmic variance of the prediction.

[0033] Preferably, the fusion depth value is used. Inverse matrix of camera intrinsic parameters The centroid pixel coordinates of the target vehicle in the image coordinate system Three-dimensional coordinates projected back onto the camera coordinate system Using the camera's extrinsic parameters, the target vehicle's center is transformed from the camera coordinate system to the world coordinate system, thus obtaining the target vehicle's three-dimensional center coordinates in the world coordinate system. ,in, R refers to the three-dimensional center coordinates of the vehicle in the world coordinate system, and R refers to the rotation matrix of the camera. The fusion depth value refers to K, which refers to the camera intrinsic parameter matrix. The centroid pixel coordinates of the target in the image are given, and T is the translation vector of the camera.

[0034] Preferably, the coordinates of the eight vertices of a standard cube with dimensions (L, W, H) centered at the origin are used to construct a rotation matrix around the vertical axis based on the heading angle ψ. The rotation matrix is ​​then applied to the eight local vertex coordinates to rotate them to the same direction as the actual orientation of the target vehicle. Finally, the eight rotated vertex coordinates are translated to the three-dimensional center coordinates in the world coordinate system to obtain the coordinates of the eight three-dimensional bounding box vertices of the target vehicle in the world coordinate system.

[0035] Preferably, the physical size information of the target vehicle is extracted from the three-dimensional bounding box, the vehicle outline size limit database in the pre-stored national standard is retrieved, and the national standard limit corresponding to the target vehicle model is retrieved and matched from the vehicle outline size limit database according to the target vehicle type classification result.

[0036] If all physical dimensions do not exceed the corresponding national standard limits, the target vehicle is deemed to be in a legal state.

[0037] When the measured physical width W is greater than the corresponding national standard width limit, or the measured physical height H is greater than the national standard height limit, or the measured physical length L is greater than the national standard length limit corresponding to the vehicle model, the target vehicle is determined to be in an over-limit state. At this time, the multi-dimensional evidence information is structured and encapsulated according to the preset data structure to generate a standardized off-site law enforcement evidence data package. The off-site law enforcement evidence data package has a complete evidence chain, including two-dimensional visual capture evidence, three-dimensional reconstruction evidence, physical measurement numerical evidence, and national standard basis.

[0038] Preferably, after generating the lightweight 3D detection model, the method further includes a cross-road segment domain adaptive deployment step, comprising:

[0039] Acquire unlabeled real-time monitoring images of road segments within the target area;

[0040] A scene-adaptive domain alignment module is constructed, comprising a domain discriminator, a prototype alignment submodule, and a style transfer adapter. Unsupervised domain-adaptive fine-tuning is performed on the feature extraction backbone network of the lightweight 3D detection model using unlabeled real-time monitoring images of the target domain road segment. When deploying the lightweight 3D detection model from the source domain to the target domain, the weight parameters of the feature extraction backbone network are first kept completely frozen, and only the domain discriminator, the prototype alignment submodule, and the style transfer adapter are trained. This ensures that the domain alignment module learns the data distribution characteristics of the target domain without interfering with the feature extraction capability of the backbone network. Then, the freeze on the feature extraction backbone network is lifted, and joint fine-tuning is performed on the feature extraction backbone network, while the scene-adaptive domain alignment module continues to be trained. After fine-tuning, the domain discriminator is removed, retaining only the feature extraction backbone network and the style transfer adapter, and online inference is performed on the target domain road segment.

[0041] Based on the lightweight 3D detection model after domain adaptive fine-tuning, the system performs dual-path parallel solution of semantic depth regression and geometric depth derivation on the real-time monitoring images of the target domain road segment, as well as the identification and structured evidence extraction of the over-limit violation.

[0042] Preferably, after outputting the fusion depth value, a temporal depth correction step is also included, comprising:

[0043] Using the inter-frame optical flow motion vector output by the lightweight 3D detection model, the depth value of the previous frame is propagated forward through the optical flow field to the current frame to obtain a depth prediction value based on motion propagation. Based on the image feature maps of the current frame and the previous frame, a matching cost volume is constructed under epipolar constraints, and a depth estimate based on stereo matching is obtained through cost volume regression. The depth prediction value based on motion propagation, the depth estimate based on stereo matching, and the fused depth value of the current frame are weighted and fused to output a temporally corrected depth value. The weighting weights are determined by their respective uncertainty measures.

[0044] (III) Beneficial Effects: This invention provides a vehicle 3D over-limit detection method based on virtual-real fusion and edge lightweighting. First, it combines the high-fidelity rendering capabilities of Unreal Engine 5 with the requirements of traffic violation recognition. By constructing digital twin scenes, it generates simulated labeled data with centimeter-level physical truth values ​​in batches. Then, it uses a recurrent consistency adversarial network to align features between the simulation domain and the real domain, effectively solving the engineering dilemma of high cost of acquiring 3D labeled data and scarcity of illegal modifications and over-limit samples in real-world scenarios. Second, it adopts a lightweighting strategy that combines structured channel pruning based on batch normalization layer scaling factors with multi-level knowledge distillation at the feature level, output level, and relation level. While significantly reducing the number of parameters and computational complexity, the three-level knowledge alignment ensures that the lightweight model still retains the ability to perceive key geometric features of the vehicle. This successfully solves the engineering problem that complex 3D detection algorithms cannot be implemented in real time on low-computing-power roadside edge computing units. Finally, a dual-path parallel depth calculation mechanism driven by semantic features and ground plane physical geometric constraints was constructed. The random error of pure semantic regression was corrected in real time using physical geometric laws, and the output results were strictly aligned with national standards. This enabled the monocular vision system to truly meet the evidence requirements of off-site law enforcement and has extremely high industry promotion value. Attached Figure Description

[0045] Figure 1 This is a flowchart of the steps of a three-dimensional vehicle over-limit detection method based on virtual-real fusion and edge lightweighting according to the present invention;

[0046] Figure 2 This is a flowchart of the training and optimization process for a lightweight 3D detection model. Detailed Implementation

[0047] The present invention will be further described in detail below with reference to preferred embodiments. More details are set forth in the following description in order to provide a full understanding of the present invention. However, the present invention can obviously be implemented in many other ways different from those described herein. Those skilled in the art can make similar extensions and derivations based on actual application situations without departing from the spirit of the present invention. Therefore, the scope of protection of the present invention should not be limited by the content of this specific embodiment.

[0048] The accompanying drawings are schematic diagrams of embodiments of the present invention. It should be noted that these drawings are for illustrative purposes only and are not drawn to scale, and should not be construed as limiting the actual scope of protection of the present invention.

[0049] A method for detecting 3D vehicle over-limit conditions based on virtual-real fusion and edge lightweighting, such as Figure 1 As shown, it specifically includes:

[0050] Step 1: Obtain the parameters of the roadside image acquisition device and the real roadside environment image, and use the digital twin engine to construct a virtual traffic environment aligned with the real roadside scene space to generate a simulation image with three-dimensional physical truth values; use the simulation image as the source domain and the real roadside environment image as the target domain to output an enhanced training dataset.

[0051] Step 2: Construct an initial 3D object detection model containing a feature extraction backbone network. Using the enhanced training dataset as training samples, perform structured pruning and multi-level knowledge transfer on the initial 3D object detection model to generate a lightweight 3D detection model.

[0052] Step 3: Based on the lightweight 3D detection model, output the image semantic features of the target vehicle and the vehicle touch point location. Combine the spatial pose parameters of the image acquisition device and the scene reference surface location information, perform parallel solution through dual paths of semantic depth regression and geometric depth derivation, and perform adaptive weighted fusion based on the prediction confidence of each path to output the fusion depth value.

[0053] Step 4: Reconstruct the 3D bounding box of the target vehicle in the world coordinate system based on the fusion depth value, and compare it with the national vehicle outline size standard in real time to automatically perform the identification of overweight and oversized violations and the extraction of structured evidence.

[0054] Step 1 specifically includes:

[0055] Step 101: Collect roadside images. Acquire equipment parameters and real roadside environment images.

[0056] The roadside image acquisition device uses a roadside monitoring camera to obtain the calibration parameters of the roadside monitoring camera.

[0057] The calibration parameters include the intrinsic parameter matrix, installation height, downward tilt angle, and radial / tangential distortion coefficients, which fully record the spatial pose parameters of the roadside monitoring camera and serve as the configuration benchmark for the virtual camera.

[0058] The roadside image acquisition device acquires real roadside environment images, and simultaneously uses 3D laser scanning to obtain high-precision real road surface point cloud data, including road surface scan data, CAD models of roadside facilities such as road guardrails, monitoring poles, and signs, as well as 3D coordinate information of fixed reference objects in the scene.

[0059] A complete vehicle model library has been built, including 3D models of long-tail oversized and overweight samples such as compliant standard trucks, oversized and modified vehicles, irregularly shaped large-item transport vehicles, and overloaded vehicles, covering all typical vehicle types for road administration enforcement.

[0060] Real roadside environment images were continuously collected at multiple time periods and under various weather conditions in the target detection section, covering strong light, backlight, rain, dense fog, nighttime, traffic congestion, and heavy vehicle occlusion, and were compiled into a real image target domain dataset.

[0061] Step 102: Construct a virtual traffic scene aligned with the real scene space based on the digital twin engine, and generate simulation images with three-dimensional physical truth values ​​in batches.

[0062] By utilizing Nanite virtual geometry technology in Unreal Engine 5, the collected real road surface point cloud data and roadside facility CAD models are transformed into high-fidelity virtual geometric assets. Through Nanite's unique micro-polygon streaming mechanism, the geometric topology of the real road section is restored at a 1:1 scale, while retaining the centimeter-level details of road surface texture and curb geometric features, ensuring that the virtual scene and physical space are strictly aligned in terms of spatial scale.

[0063] Specifically, 3D laser scanning equipment is used to acquire point cloud data of the target road section, and CloudCompare or Global Mapper software is used to convert the real road point cloud data into a 3D mesh model. During the conversion process, the original coordinate system of the point cloud is preserved, and the coordinate system parameters are recorded for subsequent alignment with the world coordinate system of the virtual scene. The converted mesh model is imported into the Unreal Engine 5 content browser, and the Build Nanite option is checked in the import dialog box to enable Nanite virtual geometry technology. For the CAD model of roadside facilities, the format is directly converted and lightweighted, output in .fbx format, and imported into Unreal Engine 5, again with the Nanite option enabled.

[0064] In Unreal Engine 5, virtual cameras are set up at corresponding world coordinate positions in the virtual scene based on the real-time dynamic positioning and measurement coordinates of the actual roadside image acquisition equipment. At the same time, at least three fixed reference objects in the scene are selected, and their three-dimensional coordinates in the real world and their corresponding coordinates in the virtual scene are measured. By calculating the rigid body transformation matrix, spatial alignment between the virtual scene and the real space is achieved with centimeter-level precision.

[0065] The system collects 3D models covering all vehicle types specified in GB 1589-2016, including but not limited to light trucks, medium trucks, heavy semi-trailer tractors, full trailers, refrigerated trucks, box trucks, and heavy-duty transport vehicles. After importing each vehicle model into Unreal Engine 5, a custom data structure is bound to it via the Blueprint system. This custom data structure contains three floating-point member variables: Length, Width, and Height, recording the vehicle's actual physical length, width, and height, respectively. During automated simulation data acquisition, the vehicle models are controlled to travel along a preset spline path using Python scripts or Unreal Engine 5's Level Sequence system. Simultaneously, the physical dimension attributes of each vehicle model are read in real-time through the Blueprint interface and written to the output labels.

[0066] Subsequently, the Lumen dynamic global illumination system of Unreal Engine 5 was activated. By simulating diffuse reflection of natural light, reflections from metallic vehicle bodies, reflections from road surface water, and vehicle shadows, physically realistic indirect lighting and contact shadows were calculated in real time. The engine's weather system was used to batch adjust rainfall, fog density, light intensity, and day / night cycles to reproduce various harsh imaging environments. Virtual camera parameters were matched one-to-one with those of real roadside image acquisition equipment, achieving complete alignment of the virtual scene with the spatial scale and imaging perspective of the real road section. This resulted in the generation of rendered images with accurate depth-perceived visual cues at different times and from different perspectives.

[0067] In the constructed virtual traffic scenario, virtual cameras are set up at corresponding spatial locations in the virtual traffic scenario according to the calibration parameters of the actual roadside image acquisition equipment, and lens parameters consistent with those of the roadside image acquisition equipment are configured to ensure that the geometric projection relationship of the virtual camera image is strictly consistent with that of the real camera.

[0068] Meanwhile, various types and models of vehicle 3D geometric models are imported into the virtual traffic scene, and each vehicle model is given realistic physical size attributes, including length L, width W, and height H.

[0069] The system uses automated scripts to control weather and lighting parameters in a virtual environment, including but not limited to setting different weather conditions, adjusting rainfall parameters, configuring fog concentration indices, and setting light intensity. Under these combined weather and lighting parameters, vehicle models in the virtual traffic scene are driven to move along preset traffic flow trajectories. Virtual cameras continuously capture simulation images, and the system simultaneously and automatically exports the 3D physical truth label data of each vehicle target in each frame of the simulation image, thereby generating simulation images with 3D physical truth values ​​in batches.

[0070] The three-dimensional physical truth label data includes at least the center three-dimensional coordinates of the vehicle target in the world coordinate system. Physical length ,width ,high Heading angle .

[0071] Step 103: Perform source-target domain style transfer based on the recurrent consistency adversarial network and output an enhanced training dataset.

[0072] Because the simulated images inherently differ from real roadside monitoring images in terms of texture style, color distribution, motion blur, and sensor noise characteristics, the generalization performance of 3D detection models trained directly on the simulated images is severely degraded in real-world scenarios. To address this, this invention introduces a Cycle-Consistent Adversarial Networks (CycleGAN) architecture, using the simulated images as the source domain X and the real roadside environment images as the target domain Y, to perform unpaired cross-domain image style transfer.

[0073] Specifically, two generators and two discriminators are constructed. The generators include a forward generator G and a backward generator F. The forward generator G is used to convert the simulated image in the source domain into an image in the style of the target domain, i.e., G:X→Y. The backward generator F is used to convert the real roadside environment image in the target domain into an image in the style of the source domain, i.e., F:Y→X.

[0074] The discriminator includes a real-domain discriminator. and simulation domain discriminator The real domain discriminator This is used to determine whether the image output by the forward generator is indistinguishable in distribution from the real roadside environment image in the target domain. The simulation domain discriminator... Used to determine whether the image output by the inverse generator is indistinguishable from the simulated image in the source domain in terms of distribution.

[0075] During training, in addition to the conventional adversarial loss function, a cycle consistency loss constraint is introduced to ensure that the image can be restored to the original input image as much as possible after round-trip transformation, thereby ensuring that key information such as the vehicle's geometry, edge contours, and spatial layout remains unchanged during the transformation process. The cycle consistency loss is as follows: ,

[0076] in, Refers to the cycle consistency loss term. Exponential expectation operator, Refers to simulated images. Refers to real roadside environment images. Refers to generating images in the forward direction. Refers to reconstructing the image in reverse. A simulation image generated from a real roadside environment image. The real image reconstructed from the simulated image should satisfy F(G(x))≈x for any simulated image x∈X and G(F(y))≈y for any real image y∈Y. refer to Norm.

[0077] The overall objective function is a weighted sum of the adversarial loss term and the cycle consistency loss term. ,

[0078] in, Refers to the overall optimization objective function of CycleGAN. Refers to a forward generator. Refers to a reverse generator. Refers to the simulated domain discriminator. Refers to the real domain discriminator. Refers to the item that counteracts losses. Refers to simulated images. Refers to real roadside environment images. The weight hyperparameters of the cycle consistency loss. This refers to the cycle consistency loss term.

[0079] Through training and inference of the recurrent consistency adversarial network, simulated images from the source domain are batch-converted into enhanced images that retain the original three-dimensional physical truth labels and possess the texture style and noise features of real roadside monitoring images, thereby constructing a virtual-real fusion enhanced training dataset.

[0080] Each image in the enhanced training dataset is associated with an accurate three-dimensional physical ground truth label, and its visual appearance is highly consistent with the real roadside monitoring scene, significantly reducing the feature distribution gap between the simulation domain and the real domain.

[0081] Step 2 specifically includes:

[0082] First, an initial 3D object detection model incorporating a feature extraction backbone network is constructed and fully trained using the aforementioned augmented training dataset to achieve high-precision 3D perception capabilities. Then, the initial model is parameter-simplified through structured channel pruning based on batch normalization layer scaling factors to obtain a compact sub-network. Finally, a multi-level knowledge distillation framework integrating feature-level, output-level, and relation-level knowledge is designed to fully transfer the high-precision 3D inference capabilities of the initial 3D object detection model to the lightweight compact sub-network, ultimately generating a lightweight 3D detection model suitable for low-power edge computing devices.

[0083] Step 201: Construct an initial model for 3D object detection and perform pre-training to obtain a pre-trained model with high-precision 3D size perception capability.

[0084] The initial model for 3D object detection employs an encoder-decoder architecture. The encoder is a feature extraction backbone network, which uses a ResNet-50 multi-stage residual network structure, consisting of cascaded residual blocks across five stages. The output feature maps of each stage have 64, 256, 512, 1024, and 2048 channels, respectively. Each residual block uses a Bottleneck structure, containing convolutional layers, batch normalization (BN) layers, and nonlinear activation layers to extract multi-scale deep semantic features from the input image. The decoder is a 3D prediction head, connected to the output of the feature extraction backbone network, used to regress and output the 3D physical ground truth label data for each target vehicle based on the feature maps extracted by the backbone network. The 3D prediction head also includes a classification prediction branch, a 3D attribute regression branch, a semantic depth prediction branch, a depth uncertainty estimation branch, and a geometric path uncertainty estimation branch.

[0085] Using the enhanced training dataset as training samples, the initial 3D object detection model is trained end-to-end under supervised conditions, such as... Figure 2 As shown, the network weight parameters are optimized by minimizing the 3D object detection loss function until the model converges, thus obtaining a pre-trained model with high-precision 3D size perception capability.

[0086] The 3D target detection loss includes classification loss, 3D bounding box regression loss, and heading angle regression loss.

[0087] Step 202: Perform structured channel pruning on the pre-trained model based on the batch normalization layer scaling factor to obtain a compact sub-network.

[0088] To significantly reduce the number of parameters and computational overhead of the pre-trained model, structured channel pruning is implemented in the feature extraction backbone network. Specifically, the scaling factor γ of the batch normalization layer in the pre-trained model is used as a direct evaluation index of the importance of the corresponding convolutional channels. Each batch normalization (BN) layer following a convolutional layer in the feature extraction backbone network contains a pair of learnable parameters: a scaling factor γ and an offset factor β. The scaling factor γ is used to scale the normalized features. When the γ value of a certain channel approaches zero, it indicates that the output feature information of that channel will be significantly suppressed, and its contribution to the downstream detection task will disappear; that is, this channel is a redundant channel.

[0089] To induce the feature extraction backbone network to converge towards channel sparsity, a sparsity training strategy is introduced during training. Based on the original 3D object detection loss function, an L1 regularization penalty term is added, applying the scaling factor of all BN layers participating in pruning. Specifically, the total training loss function is a weighted sum of the basic detection loss and the sparsity penalty term. ,

[0090] in, The total loss function of pruning training. This refers to the input training image and its corresponding label pair. This refers to the basic 3D object detection loss function. The feature extraction backbone network processes input data and produces predicted values ​​via forward propagation; W refers to the network weight parameters. The strength coefficient of the sparsity penalty term. The scaling factor of the batch normalization layer. The set of all scaling factors involved in pruning in the backbone network. The absolute value operator, i.e. Norm.

[0091] Gradient backpropagation of the total training loss function drives the scaling factors of unimportant channels in the network to gradually approach zero during training iterations. After sparse training converges, the numerical distribution of the scaling factors γ of all BN layers in the feature extraction backbone network is statistically analyzed. A global pruning threshold is calculated based on a preset target pruning ratio r, whereby exactly r of the channels globally have γ values ​​below this threshold.

[0092] After the sparsity training converges, the absolute values ​​of the scaling factors γ of all BN layers in the feature extraction backbone network are collected to form a set. , where N is the total number of channels participating in pruning. The elements in the array are sorted in ascending order of value to obtain an ordered sequence. Calculate the global pruning threshold based on the preset target pruning ratio r∈(0,1). ,in This is the floor function. Then, all values ​​that satisfy the floor function are removed. The redundant channels and their corresponding weight parameters in the previous convolutional kernel, as well as the corresponding normalization parameters in the next BN layer.

[0093] After pruning, the obtained compact subnetwork is fine-tuned to restore accuracy. The augmented training dataset is used as the training sample, employing the same optimizer Adam and learning rate scheduling strategy as the pre-training phase. The initial learning rate is set to 1 / 10 of the initial learning rate in the pre-training phase, and the training epochs are 50 epochs long. Every 20 epochs, the learning rate is decayed to 0.1 times the current value. During fine-tuning, only all trainable parameters of the compact subnetwork are updated; no sparsity regularization penalty terms are involved.

[0094] For residual connections in residual networks, when pruning causes a mismatch in the number of channels between the residual branch and the identity mapping branch, a 1×1 convolutional layer is added to the identity mapping branch to adjust the number of channels to match the number of channels after pruning the residual branch, ensuring normal addition.

[0095] Subsequently, the system automatically removes all redundant channels with γ values ​​lower than the global pruning threshold, and deletes the corresponding weight parameters of the previous convolutional kernel associated with these redundant channels, as well as the corresponding normalized parameters in the next BN layer, thereby obtaining a compact subnetwork with higher parameter density, lower computational cost, and non-uniform reduction in the number of channels.

[0096] Step 203: Based on the pre-trained model, perform multi-level knowledge transfer on the compact sub-network to obtain a lightweight 3D detection model.

[0097] Since structured pruning inevitably leads to a decrease in the feature representation ability of compact subnetworks and a loss of accuracy in the final 3D size measurement, this invention provides a multi-level knowledge distillation framework that integrates feature level, output level, and relation level. The pre-trained model before pruning is used as the knowledge output, and the compact subnetwork generated after pruning is used as the knowledge receiver. By establishing a supervision mapping between the two at different abstract dimensions, the lightweight compact subnetwork is guided to fully reproduce the high-precision 3D reasoning ability of the pre-trained model.

[0098] Specifically, the multi-level knowledge distillation includes feature-level distillation, output-level distillation, and relation-level distillation.

[0099] The feature-level distillation aims to force the compact subnetwork to mimic the feature distribution of the pre-trained model at intermediate layers of the feature extraction backbone network, thereby preserving the ability to extract low-level spatial details such as image texture and geometric edges. Several representative intermediate feature layers in the feature extraction backbone network are selected as supervisory layers. These layers are chosen according to the spatial resolution of the feature maps, typically the output of the first residual block after each downsampling stage. For the output layer of the first residual block after each downsampling stage, the pre-trained model and the supervisory layer are extracted respectively. Corresponding feature map and the compact subnetwork in the supervision layer Output feature map .

[0100] Transformation Operator The specific structure is a 1×1 convolutional layer, with the number of input channels equal to the number of channels in the current layer of the compact sub-network, and the number of output channels equal to the number of channels in the corresponding layer of the pre-trained model. Using the two-dimensional detection box of the target vehicle output by the lightweight 3D detection model, the feature map of the detection box region is adjusted to a fixed size in the intermediate feature layer of the feature extraction backbone network through bilinear interpolation. Then, global average pooling is performed on the adjusted feature map to obtain the feature vector of the target region.

[0101] The distillation temperature coefficient T is set to 4.0; the balancing hyperparameters for characteristic-level distillation loss, output-level distillation loss, and relation-level distillation loss are set to... , , The optimizer used is Adam, with an initial learning rate of 0.0001, a batch size of 16, and a training cycle of 80 epochs. Every 30 epochs, the learning rate is decayed to 0.1 times the current value.

[0102] During distillation training, the classification output-level distillation loss and the hard-label classification loss are jointly optimized, and the total classification loss is defined as follows: ,in The cross-entropy loss is based on the true labels, and α is a balancing coefficient, set to 0.9 in the early stages of training to accelerate convergence, and gradually reduced to 0.5 in the later stages. During the inference phase, only the hard labels of the compact subnetwork are used to predict the output, and the distillation branch is removed during inference.

[0103] Since the number of channels in the pruned compact subnetwork is less than that in the pre-trained model, a learnable transformation operator is introduced before calculating the feature-level distillation loss. The feature map of the compact sub-network is aligned to the channel dimension so that it has the same number of channels as the feature map of the pre-trained model. Then, the squared L2 norm of the element-wise difference between the feature map of the pre-trained model and the feature map of the transformed compact sub-network is calculated as the feature-level distillation loss for this layer. The feature-level distillation loss function... ,in, Characteristic stage distillation loss Refers to the selected supervisory layer index, where For the selected set of supervisory intermediate layers. The feature map of the pre-trained model. The feature map of a compact subnetwork. Transformation operators used for channel dimension alignment refer to Norm.

[0104] The total feature-level distillation loss is obtained by summing the feature-level distillation losses of all selected supervision layers. This feature-level distillation loss constrains the compact subnetwork to approximate the pre-trained model on the low-level feature responses, ensuring that the lightweight model maintains sufficient sensitivity to key geometric information such as vehicle edge contours.

[0105] The core of the output-level distillation lies in achieving strict alignment of the prediction head output distribution, specifically including two aspects: soft label alignment of the classification prediction branch and numerical alignment of the three-dimensional attribute regression branch.

[0106] For the classification prediction branch, a temperature coefficient-based approach is used. The soft-label distillation strategy involves dividing the classification logits output by the pre-trained model and the compact subnetwork by a preset distillation temperature coefficient T, and then processing them using the normalized exponential function Softmax to obtain the temperature-smoothed classification probability distribution. Using the smoothed probability distribution of the pre-trained model as the soft label, the KL divergence between it and the smoothed probability distribution of the compact subnetwork is calculated. This KL divergence is then multiplied by the square of the temperature coefficient T for scaling and used as the classification output-level distillation loss. ,in, This refers to the distillation loss in the output stage of the classification process. The distillation temperature coefficient. Refers to the KL divergence operator, The normalized exponential function, The classification probability of the pre-trained model. This refers to the classification probability of the compact subnetwork. The classification prediction branch enables the compact subnetwork to learn fine-grained discriminative knowledge at the level of inter-class similarity from the pre-trained model.

[0107] For the 3D attribute regression branch, the key 3D physical attribute vectors for each target vehicle are... The smoothed L1 loss between the pre-trained model predictions and the compact sub-network predictions is calculated separately. For different attribute dimensions, corresponding weight coefficients are assigned according to their importance to the over-limit detection task. The weighted smoothed L1 losses of each attribute dimension are then summed to obtain the attribute regression output-level distillation loss. ,in, This refers to the distillation loss at the output stage of attribute regression. Refers to the three-dimensional attribute dimensions of the target. This refers to the importance weight of different attributes. Smooth loss function The three-dimensional physical property values ​​predicted by the pre-trained model. This refers to the three-dimensional physical property values ​​predicted by the compact subnetwork. The three-dimensional property regression branch directly constrains the compact subnetwork to approximate the pre-trained model in terms of physical size prediction values, ensuring that the lightweight model has centimeter-level size measurement accuracy similar to the pre-trained model.

[0108] The relation-level distillation enhances the structure perception capability of the compact subnetwork by modeling the topological relationships between multiple vehicle targets within a scene. For multiple vehicle targets detected in the same input image, feature vectors corresponding to each target region in the intermediate feature layer of both the pre-trained model and the compact subnetwork are first extracted. Based on these feature vectors, target similarity matrices for the pre-trained model and the compact subnetwork are constructed, respectively. The element in the m-th row and n-th column of the similarity matrix is ​​defined as the cosine similarity between the m-th and n-th target feature vectors, representing the interaction distance and structural affinity between the two targets in the feature space. Subsequently, the difference between the relation matrix generated by the pre-trained model and the relation matrix generated by the compact subnetwork is calculated. The Frobenius norm is used to measure the degree of structural difference between the relation matrix generated by the pre-trained model and the relation matrix generated by the compact subnetwork. The Frobenius norm value is the relation-level distillation loss. ,in, The element in the m-th row and n-th column of the target similarity matrix. The feature vector of the m-th target. The eigenvector of the nth target is referred to as T, where T is the transpose operator. The modulus of the pointer quantity.

[0109] By minimizing the relational distillation loss, the compact subnetwork learns the relative positional relationships and scale ratio logic between multiple vehicle targets in the scene, thereby approximating the perception capability of the pre-trained model at the overall scene understanding level.

[0110] Finally, the feature-level distillation loss, output-level distillation loss, and relation-level distillation loss are multiplied by their respective preset balancing weight hyperparameters, and then weighted and summed to construct the total multi-level knowledge distillation loss function. ,in, The total multi-level knowledge distillation loss function. The hyperparameters used to balance the weights of characteristic-level distillation loss, output-level distillation loss, and relation-level distillation loss. Characteristic stage distillation loss This refers to the distillation loss in the output stage of the classification process. This refers to the distillation loss at the output stage of attribute regression. This refers to the loss during the distillation stage.

[0111] Using the enhanced training dataset as training samples, the compact sub-network as the object to be optimized, the pre-trained model as the source of supervision signals, and the total multi-level knowledge distillation loss function as the joint optimization objective, the compact sub-network undergoes end-to-end fine-tuning training. During training, the compact sub-network simultaneously receives supervision signals from the ground truth labeled values ​​and multi-level distillation supervision signals from the pre-trained model, gradually restoring and maintaining high-precision 3D spatial reasoning capabilities while preserving the lightweight architecture. After training convergence, a lightweight 3D detection model is output.

[0112] Step 3 specifically includes:

[0113] To address the scale-depth ambiguity problem caused by perspective distortion in monocular vision perception, a dual-path parallel solution mechanism driven by semantic features and ground plane physical geometric constraints is constructed. Furthermore, depth uncertainty modeling is introduced to achieve adaptive gating fusion of heterogeneous depth cues, eliminating the random oscillations in depth prediction caused by pure data-driven black-box models in long-distance and occluded scenarios, and outputting highly reliable fused depth values.

[0114] Step 301: Based on the semantic features of the output image of the lightweight 3D detection model, construct a semantic depth regression path, and calculate the semantic depth prediction value and the logarithmic variance of the semantic depth prediction path.

[0115] After the target vehicle image is input into the lightweight 3D detection model, it first undergoes forward propagation through the feature extraction backbone network, outputting a multi-scale image semantic feature map with a global receptive field. The multi-scale image semantic feature map includes high-level semantic information such as the target vehicle's color, texture, edge contours, shadow distribution, and scene context.

[0116] Subsequently, the feature tensor corresponding to the target vehicle region in the multi-scale image semantic feature map is input into the semantic depth prediction branch of the lightweight 3D detection model. The semantic depth prediction branch consists of three multilayer perceptron (MLP) layers with 512, 256, and 1 neurons respectively. Each fully connected layer is followed by a ReLU activation function, which, through pre-trained nonlinear mapping weights, directly maps the image semantic feature tensor into continuous depth values, outputting a semantic depth prediction value based on the semantic features, denoted as . The semantic feature-driven semantic depth regression path can output a highly sensitive depth response based on data-driven statistical regularities when the target vehicle is in the foreground area, the image features are rich in texture, and there is no obvious occlusion.

[0117] Simultaneously, a depth uncertainty estimation branch is set up in the parallel prediction head of the lightweight 3D detection model. This depth uncertainty estimation branch shares the semantic features extracted by the feature extraction backbone network with the semantic depth prediction branch, and its output is the logarithmic variance of the semantic depth prediction path, denoted as […]. The log-variance of the semantic depth prediction path. Based on the Laplace distribution assumption, a logarithmic form is used to ensure the non-negativity of variance and gradient stability in subsequent numerical calculations. This is used to quantitatively characterize the confidence level of the semantic deep regression path in the prediction results. The larger the logarithmic variance of the semantic deep prediction path, the lower the prediction confidence of the semantic deep regression path, and the higher the degree to which the predicted value may be affected by perspective distortion or out-of-domain features.

[0118] Step 302: Based on the output of the vehicle contact point position of the lightweight 3D detection model, and combined with the spatial pose parameters of the image acquisition device and the scene reference surface position information, construct an explicit geometric depth derivation path, and calculate the explicit geometric depth derivation value and the logarithmic variance of the geometric depth prediction path.

[0119] First, using the lightweight 3D detection model, vehicle contact points are detected in the input image. The vehicle contact point is the center point of the contact area between the bottom of the target vehicle and the roadside ground plane. After detection, the ordinate value of the vehicle contact point in the image pixel coordinate system is extracted and denoted as . .

[0120] Simultaneously, the spatial pose parameters of the image acquisition device and the scene reference plane position information are acquired and pre-calibrated and stored. The spatial pose parameters include at least the camera's installation height H and the camera's longitudinal pixel focal length. The scene reference plane position information includes the vertical coordinate position of the horizon on the image plane in the current roadside monitoring scene, denoted as... The horizon position can be directly calculated from the camera's extrinsic calibration data, or it can be calibrated by pre-selecting a fixed horizontal reference object in the scene.

[0121] Based on the principles of similar triangles and perspective projection geometry, under the ground plane assumption, the offset of the pixel ordinate of the vehicle's contact point relative to the horizon ordinate is inversely proportional to the physical distance from the vehicle to the camera. Therefore, an explicit geometric depth derivation based on ground plane physical geometric constraints is obtained, denoted as […]. : ,in, This represents the depth value directly calculated based on projective geometric constraints. H represents the vertical pixel focal length of the camera; H represents the installation height of the camera. Represents the vertical coordinate of the vehicle's contact point in the image coordinate system; This represents the ordinate of the vanishing horizon in the image coordinate system.

[0122] The explicit geometric depth derivation path strictly follows the laws of physical imaging and does not rely on statistical priors of data distribution. In scenarios with small targets at a distance or when the target features are partially occluded, it can provide more stable and physically consistent depth spatial localization compared to pure semantic regression.

[0123] Simultaneously, a geometric path uncertainty estimation branch is set up in parallel within the lightweight 3D detection model. This branch takes the global features extracted by the feature extraction backbone network and the local features of the vehicle contact point as input, and outputs the logarithmic variance of the geometric depth prediction path, denoted as... .

[0124] The log-variance of the geometric depth prediction path This is used to quantify the reliability of the explicit geometric depth derivation path in the current scenario. The logarithmic variance of the geometric depth prediction path is used when there are changes in road slope, ground contact detection is interfered with by shadows, or the vehicle is on a slope. The numerical values ​​are adaptively increased to reflect the increased uncertainty caused by the mismatch of geometric assumptions.

[0125] Step 303: Construct an adaptive gated fusion based on the depth uncertainty of the log-variance of the semantic depth prediction path and the log-variance of the geometric depth prediction path to obtain the fusion depth value.

[0126] The semantic depth prediction value The explicit geometric depth derivation value All paths are modeled as Laplace distributions, and the log-variance of the semantic depth prediction path is used. The logarithmic variance of the geometric depth prediction path This characterizes the inherent noise and uncertainty level of its output. Based on uncertainty estimation theory, prediction confidence is negatively correlated with logarithmic variance. The smaller the variance, the closer the predicted value is to the true value, and it should be given a higher fusion weight during fusion.

[0127] Specifically, for the semantic depth prediction path, the logarithmic variance of the semantic depth prediction path is used. The negative exponential function value is used as the weight of the original semantic depth prediction confidence score, i.e. For the geometric depth prediction path, the logarithmic variance of the geometric depth prediction path is used. The negative exponential function value is used as the weight for the original geometric depth prediction confidence score, i.e. Subsequently, the original semantic depth prediction confidence weights and the original geometric depth prediction confidence weights are normalized to obtain normalized dynamic fusion coefficients.

[0128] Finally, the normalized dynamic fusion coefficients are multiplied by their corresponding depth prediction values, and then weighted and summed to output the fusion depth prediction value. : ,in, This refers to the fusion depth value after adaptive fusion. Refers to semantic depth prediction value, The value is derived from the explicit geometric depth. This refers to the log-variance of the semantic depth prediction path. The logarithmic variance of the geometric depth prediction path is given, and e is the natural constant. This represents the logarithmic variance of the prediction.

[0129] When the target vehicle is in close-up and the image texture is clear, the log-variance of the semantic depth prediction path is... The log-variance of the semantic depth prediction path is relatively small, giving it higher weight and fully leveraging its high sensitivity advantage. When the target vehicle is in the distance, severely obstructed, or under adverse weather conditions, the log-variance of the semantic depth prediction path is... The logarithmic variance of the geometric depth prediction path increases dramatically, while the geometric depth prediction path is constrained by physical laws. Relatively stable, higher weights are assigned to the geometric depth derivation value, and physical laws are used to correct the random prediction oscillations caused by pure semantic regression in real time. The output is a fusion depth value that combines high accuracy and high robustness. This provides a solid depth benchmark for the accurate physical inversion of the vehicle's three-dimensional dimensions in subsequent steps.

[0130] Step 4 specifically includes:

[0131] The final conversion from the two-dimensional pixel space of the image to the real physical three-dimensional space combines the fused depth value with the target vehicle's physical size attributes predicted by the lightweight three-dimensional detection model to accurately reconstruct the target vehicle's three-dimensional bounding box in the world coordinate system. Then, based on national mandatory standards, it automatically performs the identification and judgment of overweight and oversized violations and generates a non-on-site law enforcement evidence data package with legal credibility.

[0132] Step 401: Based on the two-dimensional centroid pixel coordinates and fusion depth value of the target vehicle in the image, the three-dimensional center spatial position of the target vehicle in the world coordinate system is obtained by coordinate transformation through the camera intrinsic inverse matrix and extrinsic rotation and translation matrix.

[0133] Based on the inverse projection transformation principle of the pinhole imaging model, the fusion depth value is first utilized. Inverse matrix of camera intrinsic parameters The centroid pixel coordinates of the target vehicle in the image coordinate system Three-dimensional coordinates projected back onto the camera coordinate system .

[0134] The fusion depth value This represents the depth distance of the target vehicle's center point along the optical axis in the camera coordinate system. Simultaneously, the centroid pixel coordinates of the target vehicle predicted by the lightweight 3D detection model in the image pixel coordinate system are obtained, denoted as... This refers to the pixel position of the center point of the two-dimensional detection bounding box of the target vehicle. The intrinsic parameter matrix K and its inverse matrix of the image acquisition device are obtained through offline calibration. The intrinsic parameter matrix K contains the camera's lateral pixel focal length. Vertical pixel focal length and principal point coordinates Parameters such as rotation matrix R and translation vector T are also obtained.

[0135] Subsequently, using the camera's extrinsic parameters, the target vehicle's center was transformed from the camera coordinate system to the world coordinate system, obtaining the target vehicle's three-dimensional center coordinates in the world coordinate system. .in, R refers to the three-dimensional center coordinates of the vehicle in the world coordinate system, and R refers to the rotation matrix of the camera. The fusion depth value refers to K, which refers to the camera intrinsic parameter matrix. The centroid pixel coordinates of the target in the image are given, and T is the translation vector of the camera.

[0136] Step 402: Based on the three-dimensional center coordinates of the target vehicle in the world coordinate system, construct the coordinates of the eight vertices of the three-dimensional bounding box in the world coordinate system.

[0137] After obtaining the 3D center coordinates of the target vehicle in the world coordinate system, the 3D physical dimension attributes of the target vehicle predicted by the lightweight 3D detection model are further obtained, including physical length L, physical width W, and physical height H, as well as the heading angle ψ. A 3D bounding box of the target vehicle in the world coordinate system is constructed. The 3D bounding box is a cube with eight vertices, whose geometric center is located at the 3D center coordinates of the world coordinate system. Its dimensions along its three axes correspond to the physical length L, physical width W, and physical height H, respectively, and its orientation is determined by the heading angle ψ.

[0138] Specifically, firstly, the coordinates of the eight vertices of a standard cube centered at the origin with dimensions (L, W, H) are defined in the local coordinate system. Then, a rotation matrix around the vertical axis is constructed based on the heading angle ψ. This rotation matrix is ​​applied to the eight local vertex coordinates, rotating them to the direction consistent with the actual orientation of the target vehicle. Finally, the rotated eight vertex coordinates are translated to the three-dimensional center coordinates in the world coordinate system, obtaining the coordinates of the eight three-dimensional bounding box vertices of the target vehicle in the world coordinate system.

[0139] The three-dimensional bounding box accurately represents the three-dimensional spatial position and outline of the target vehicle in real physical space.

[0140] Step 403: Automatically compare the physical dimensions calculated from the three-dimensional bounding box with the limits of the corresponding vehicle models in the national standard to generate a structured off-site law enforcement evidence data package.

[0141] The physical dimensions of the target vehicle, namely length L, width W, and height H, are directly extracted from the three-dimensional bounding box. All physical dimensions are in meters, providing centimeter-level accuracy.

[0142] Subsequently, the system automatically retrieves the vehicle external dimension limit database from the pre-stored national mandatory standard GB 1589-2016 "Limits of External Dimensions, Axle Load and Mass of Motor Vehicles, Trailers and Vehicle Combinations". The database specifies the corresponding maximum permissible limits for length, width and height according to different vehicle types.

[0143] Based on the target vehicle type classification results output by the lightweight 3D inspection model, the system automatically retrieves and matches the corresponding legal size thresholds for the target vehicle model from the standard database. Specifically, for general vehicles, the width limit is 2.55 meters and the height limit is 4.00 meters; for refrigerated trucks, the width limit is 2.60 meters; and the length limits for different vehicle models vary according to the specific provisions of GB 1589-2016.

[0144] If all physical dimensions do not exceed the corresponding national standard limits, the target vehicle is deemed to be in a legal state.

[0145] The target vehicle is deemed to be in an over-limit state if any of its physical dimensions exceeds the national standard limit. Specifically, if the measured physical width W is greater than the corresponding national standard width limit, or the measured physical height H is greater than the national standard height limit, or the measured physical length L is greater than the national standard length limit for that vehicle type, then the target vehicle is deemed to be in an over-limit state. At this point, the multi-dimensional evidence information is automatically structured and encapsulated according to a preset data structure to generate a standardized off-site law enforcement evidence data package.

[0146] The non-site law enforcement evidence collection data package has a complete chain of evidence, including two-dimensional visual capture evidence, three-dimensional reconstruction evidence, physical measurement numerical evidence, and national standard basis. It can provide transportation law enforcement departments with automated law enforcement basis with legal credibility and can be directly used for subsequent violation review, notification and punishment procedures.

[0147] The two-dimensional visual capture evidence acquires and saves the original roadside monitoring image of the target vehicle at the current moment, and visually marks the position of the target vehicle in the form of a two-dimensional detection box in the original roadside monitoring image. Simultaneously, it records spatiotemporal traceability information such as the detection timestamp, roadside device number, and installation location in the original roadside monitoring image. The three-dimensional reconstruction evidence, based on the three-dimensional bounding box, generates a stereoscopic projection visualization image of the target vehicle in three-dimensional space, clearly showing the degree of fit between the vehicle's outline and the three-dimensional bounding box. The physical measurement numerical evidence records and outputs the specific values ​​of the target vehicle's calculated physical length L, width W, and height H. The national standard reference refers to the corresponding national standard limits.

[0148] In large-scale deployments of roadside non-site enforcement, different road sections exhibit varying scene attributes such as road surface texture, road width, guardrail color, surrounding building style, lighting conditions, camera model, and installation height. When a lightweight 3D detection model trained on a source road section is directly deployed to another target road section, the feature extraction capability of the lightweight 3D detection model drops sharply due to the different attributes of the application road section, the 3D dimension measurement error increases significantly, and the false alarm rate and false negative rate of over-limit detection rise sharply.

[0149] To address this, the present invention also constructs a scene-adaptive domain alignment module comprising a domain discriminator, a prototype alignment submodule, and a style transfer adapter. This module utilizes unlabeled real-time monitoring images of the target domain road segment to perform unsupervised domain-adaptive fine-tuning of the feature extraction backbone network of the lightweight 3D detection model, achieving cross-road segment model adaptive transfer without requiring labeled target domain data. Based on the domain-adaptive fine-tuned lightweight 3D detection model, 3D over-limit detection is performed on the real-time monitoring images of the target domain road segment, outputting the 3D bounding box and physical size information of the target vehicle. The physical size information of the target vehicle is extracted based on the 3D bounding box and compared in real-time with national vehicle outline size standards, automatically performing the identification of over-limit violations and structured evidence extraction.

[0150] The domain discriminator's network structure consists of three fully connected layers. The first layer has an input dimension equal to the feature vector output from the feature extraction backbone network, and an output dimension of 512. The second layer has an input dimension of 512 and an output dimension of 256. The third layer has an input dimension of 256 and an output dimension of 1. The first and second layers are followed by a LeakyReLU activation function and a Dropout layer.

[0151] When deploying the lightweight 3D detection model from the source domain to the target domain, a progressive domain-adaptive fine-tuning strategy is adopted. First, the weight parameters of the feature extraction backbone network are completely frozen, and only the domain discriminator, the prototype alignment submodule, and the style transfer adapter are trained. This ensures that the domain alignment module learns the data distribution characteristics of the target domain without interfering with the feature extraction capabilities of the backbone network. Then, the feature extraction backbone network is unfrozen and jointly fine-tuned with a small learning rate. Simultaneously, the domain alignment module continues to be trained, allowing the feature extraction backbone network to gradually adapt to the feature distribution of the target domain. Constraints from the domain alignment module prevent catastrophic forgetting. After fine-tuning, the domain discriminator is removed, retaining only the feature extraction backbone network and the style transfer adapter, and online inference is performed on the target domain road segment. During the inference phase, the style transfer adapter continuously performs adaptive instance normalization on each input image frame, ensuring that the lightweight 3D detection model maintains stable feature extraction performance under different lighting and weather conditions.

[0152] Specifically, a domain discriminator is connected to the end of the feature extraction backbone network of the lightweight 3D detection model. The domain discriminator consists of multiple fully connected layers, taking the feature vector of the multi-scale image semantic feature map output by the feature extraction backbone network after global average pooling as input, and outputting the classification probability of the domain to which the feature vector belongs.

[0153] The goal of the feature extraction backbone network is to extract invariant feature representations across different domains, making it impossible for the domain discriminator to distinguish whether a feature originates from the source or target domain. The goal of the domain discriminator is to determine the domain origin of the feature as accurately as possible. A gradient inversion layer connects the feature extraction backbone network and the domain discriminator, automatically inverting the gradient direction of the domain discriminator's backpropagation to the feature extraction backbone network during backpropagation, thus achieving end-to-end adversarial training.

[0154] The optimization objective function for the adversarial domain alignment is: ,in For feature extraction backbone network, For domain discriminator, For the source domain image, The target domain image is defined as follows. By minimizing the adversarial domain alignment loss, the feature extraction backbone network progressively learns to extract domain-invariant features shared by the source and target domains, eliminating feature distribution shifts caused by scene differences.

[0155] The prototype alignment submodule performs cross-domain alignment based on category-level feature prototypes, rather than global-level feature distribution alignment, thereby further improving the accuracy of domain alignment.

[0156] For source domain data, using the vehicle type classification labels in the source domain annotations, calculate the category prototype vector for each vehicle category in the feature space. The category prototype vector is the mean of the feature vectors of all samples of that category. ,in Let be the prototype vector of the k-th class of vehicles in the source domain. Let be the number of samples of the k-th class of vehicles in the source domain. Let be the feature vector of the i-th source domain sample.

[0157] For the target domain data, the lightweight 3D detection model is used to predict pseudo-labels for unlabeled images in the target domain. Reliable prediction results are selected based on high-confidence pseudo-labels, and the category prototype vector for each vehicle category in the target domain is calculated. .

[0158] Constructing the prototype alignment loss function This is used to constrain the feature prototypes of the same vehicle category in the source and target domains to be as close as possible in the feature space. The prototype alignment loss function and the adversarial domain alignment loss function are jointly optimized to further achieve fine-grained alignment at the category level on the basis of global domain distribution alignment, effectively preventing feature confusion between different vehicle categories.

[0159] For cases where there are significant low-level visual differences between the target domain road segments and the source domain road segments, a style transfer adapter is deployed at the input of the lightweight 3D detection model.

[0160] The style transfer adapter consists of a lightweight adaptive instance normalization module. This module performs channel-by-channel mean and variance statistics on the input target domain image, aligning the channel-level statistics of the target domain image to the channel-level statistical distribution of the source domain image. Specifically, it pre-calculates the channel-level mean of all images in the source domain training dataset. and standard deviation For the input target domain image, calculate its channel-level mean. and standard deviation Then, the feature statistics of the target domain image are normalized to the source domain distribution through adaptive instance normalization. .

[0161] The style transfer adapter operates in a lightweight forward computation mode during the inference phase, without requiring additional model inference overhead. It can eliminate low-level visual style differences between the target domain and the source domain while keeping the geometric structure of the target domain image unchanged, thus significantly improving the feature extraction performance of the lightweight 3D detection model on the target domain image.

[0162] After the lightweight 3D detection model is deployed and run on the target domain road segment, it continuously collects high-confidence detection results of the target domain road segment. The high-confidence detection results satisfy that the logarithmic variance of the semantic depth prediction path and the logarithmic variance of the geometric depth prediction path are both lower than the preset logarithmic variance threshold, and the deviation between the fused depth value and the explicit geometric depth derivation value is less than the preset deviation threshold.

[0163] The high-confidence detection results and their corresponding predicted 3D physical dimensions are used as pseudo-labels to periodically self-train and fine-tune the lightweight 3D detection model. Each fine-tuning uses high-confidence pseudo-label data accumulated within the past month, enabling the lightweight 3D detection model to continuously adapt to gradual changes in the target domain road segment caused by factors such as seasonal changes, road construction, and facility upgrades, achieving long-term adaptive maintenance of the model deployed across road segments.

[0164] Secondly, when a vehicle experiences road bumps, motion blur, instantaneous sensor noise, or sudden changes in lighting, the depth prediction value of a single frame will exhibit random jumps. These random jumps are directly transmitted to the three-dimensional size inversion, causing drastic fluctuations in the calculated outline size of the same vehicle in consecutive frames. This not only reduces the accuracy of over-limit determination but also raises questions about the legal credibility of law enforcement evidence due to a lack of temporal consistency.

[0165] Based on the lightweight 3D detection model, a temporal depth correction module is added. After outputting the fused depth value, the temporal depth correction module performs temporal correlation correction between the fused depth value of the current frame and the fused depth values ​​of adjacent frames through an inter-frame optical flow-guided depth propagation submodule and an inter-frame geometric consistency cost body construction submodule, and outputs a temporally corrected depth value. The inter-frame optical flow-guided depth propagation submodule uses the inter-frame optical flow motion vector output by the lightweight 3D detection model to propagate the depth value of the previous frame forward through the optical flow field to the current frame, obtaining a depth prediction value based on motion propagation; the inter-frame geometric consistency cost body construction submodule constructs a matching cost body based on the image feature maps of the current frame and the previous frame under epipolar constraints, and obtains a depth estimate based on stereo matching through cost body regression; the depth prediction value based on motion propagation, the depth estimate based on stereo matching, and the fused depth value of the current frame are adaptively weighted and fused to output the temporally corrected depth value.

[0166] Specifically, the lightweight 3D detection model also outputs inter-frame optical flow motion vectors. To achieve temporal depth correction, the lightweight 3D detection model further includes an optical flow prediction branch. This optical flow prediction branch takes the multi-scale semantic feature maps of two adjacent frames output by the feature extraction backbone network as input, and outputs inter-frame optical flow motion vectors through a lightweight optical flow estimation sub-network consisting of four 3x3 convolutional layers with a stride of 2 and two upsampling layers. These vectors characterize the pixel-level motion displacement of the target vehicle between adjacent frames. The inter-frame optical flow-guided depth propagation submodule utilizes these inter-frame optical flow motion vectors.

[0167] The depth value from the previous frame is propagated forward through the optical flow field to the current frame to obtain a depth prediction value based on motion propagation. Specifically, for the vehicle centroid pixel coordinates in the previous frame... According to the optical flow vector Calculate the corresponding position of the vehicle's centroid pixel coordinates in the current frame. The depth value of the previous frame The depth map is obtained by mapping the depth map to the corresponding position in the current frame and performing bilinear interpolation. The depth propagation submodule utilizes the continuity of vehicle motion to provide kinematic priors for depth estimation in the current frame, independent of the image quality of a single frame.

[0168] The inter-frame geometric consistency cost body construction submodule searches for matching pixels along the corresponding epipolar line of the previous frame for each pixel in the vehicle region of the current frame, calculates feature similarity, and constructs the inter-frame geometric consistency cost body. A depth estimate based on stereo matching is obtained by regressing using the inter-frame geometric consistency cost body. The inter-frame geometric consistency cost volume utilizes the viewpoint differences between temporally adjacent frames to construct implicit binocular geometric constraints, without relying on any additional sensors.

[0169] Depth prediction based on motion propagation Depth estimation based on stereo matching The fusion depth value with the output of the lightweight model in the current frame Adaptive weighted fusion is performed to obtain the temporally corrected depth value. The weights of the adaptive weighted fusion are dynamically determined by the uncertainty measures output by each of the three sub-modules, i.e., the depth prediction values ​​based on motion propagation. Depth estimation based on stereo matching The fusion depth value with the output of the lightweight model in the current frame All are modeled as Laplace distributions, and the negative exponential function values ​​of the logarithmic variance of the current frame fusion depth path, the logarithmic variance of the optical flow propagation depth path, and the logarithmic variance of the stereo matching depth path are used as the uncertainty measures of their respective outputs.

[0170] The temporally corrected depth values ​​from multiple different frames are arranged in chronological order to obtain a temporally corrected depth value sequence. For this sequence, a vehicle kinematics state-space model is constructed, abstracting vehicle motion into a computable mathematical structure. This explicitly models the evolution of depth values ​​along the time axis, transforming the estimation of each frame from independent estimation to estimating a continuous trajectory, providing a complete mathematical framework for Kalman filtering. The state vector of the vehicle kinematics state-space model is: ,in, Let be the depth value of the vehicle along the optical axis of the camera at time t. The velocity in the depth direction, i.e., the rate of change of depth. This represents the acceleration in the depth direction.

[0171] Establish the state transition equation based on the uniformly accelerated motion model: ,in, Here is the state transition matrix. The process noise is represented by its covariance matrix Q, which is preset based on the vehicle's maximum acceleration and road surface smoothness. Using the posterior state estimate from the previous frame, the state transition matrix is ​​used... Calculate the prior state estimate for the current frame. When the vehicle's true depth changes continuously, the velocity component in the state transition equation... and acceleration components By using Kalman filtering for recursive updates and adaptive adjustments, the estimated value closely tracks the actual depth change trend without producing lag bias.

[0172] Establish observation equations This defines the relationship between the fused depth value of the current frame and the depth component in the state vector. (Observation noise in the observation equation.) The covariance matrix R quantifies the uncertainty level of the fused depth value in the current frame. When the quality of the fused depth value degrades due to occlusion, motion blur, or other reasons, the observation noise covariance adaptively increases, and the Kalman filter automatically reduces the trust weight of the current frame observation, relying more on kinematic priors for estimation. The observation vector is the temporally corrected depth value. Using the time-corrected depth value Using the observed values ​​and the observation noise covariance R, the Kalman gain is calculated, and the posterior state estimate is updated to obtain the posterior depth estimate of the Kalman filter output. The posterior depth estimate output by the Kalman filter. This is the final depth value that integrates kinematic priors and temporal observations. When the vehicle is traveling at a constant speed in a straight line, the smoothing effect of the Kalman filter stabilizes the depth estimate and effectively suppresses random oscillations in a single frame. When the vehicle's actual depth changes continuously, the Kalman filter adaptively tracks the depth change trend through the state transition equation, without producing hysteresis bias.

[0173] The posterior depth estimate from the Kalman filter output Replace the original fusion depth value Perform 3D bounding box reconstruction and physical size inversion. For the same target vehicle, calculate the size sequence over T consecutive frames. Calculate the standard deviation of different dimensions. , , As a measure of temporal stability. , , When all values ​​are less than the preset stability threshold, the calculated size is deemed to meet the temporal consistency requirement, and the mean of each size dimension within T frames is used as the final determined size. When the standard deviation of any dimension exceeds the stability threshold, the observation window is automatically extended until the temporal stability requirement is met.

[0174] The stability threshold value described in this invention is 0.03-0.05m.

[0175] The final determined dimensions are compared with the national vehicle outline dimension standards. If any dimension exceeds the national standard limit, it is determined whether the vehicle is in an over-limit state.

[0176] First, by constructing a depth propagation guided by inter-frame optical flow and an inter-frame geometric consistency cost volume, kinematic and geometric priors independent of image quality are provided for single-frame depth calculation, improving depth prediction accuracy in transient interference scenarios such as motion blur and sudden illumination changes. Second, a Kalman filter kinematic state-space model is used to temporally smooth depth values, effectively suppressing size measurement fluctuations caused by single-frame depth oscillations, and reducing the standard deviation of size measurement for the same vehicle in consecutive frames. Finally, an automatic verification mechanism based on temporal stability measurement ensures strict temporal consistency of the output judgment size, resolving the issue of instantaneous measurement bias defenses faced by single-frame detection results in law enforcement review, and enhancing the legal credibility of law enforcement evidence.

[0177] This invention provides a vehicle 3D over-limit detection method based on virtual-real fusion and lightweight edge computing. First, it combines the high-fidelity rendering capabilities of Unreal Engine 5 with the requirements of traffic violation recognition. By constructing digital twin scenes, it generates simulated labeled data with centimeter-level physical truth values ​​in batches. Then, it utilizes a recurrent consistency adversarial network to align features between the simulated and real domains, effectively solving the engineering dilemmas of high cost of acquiring 3D labeled data and scarcity of illegal modifications and over-limit samples in real-world scenarios. Second, it employs a lightweight strategy combining structured channel pruning based on batch normalization layer scaling factors with multi-level knowledge distillation at the feature, output, and relation levels. While significantly reducing the number of parameters and computational complexity, the three-level knowledge alignment ensures that the lightweight model still retains the ability to perceive key geometric features of the vehicle, successfully solving the engineering challenge of complex 3D detection algorithms being unable to be implemented in real-time on low-computing-power roadside edge computing units. Finally, a dual-path parallel depth calculation mechanism driven by semantic features and ground plane physical geometric constraints was constructed. The random error of pure semantic regression was corrected in real time using physical geometric laws. The output results are strictly aligned with the GB 1589-2016 national standard, enabling the monocular vision system to truly meet the evidence requirements of off-site law enforcement and has extremely high industry promotion value.

[0178] The above description illustrates preferred embodiments of the present invention and helps those skilled in the art to more fully understand the technical solution of the present invention. However, these embodiments are merely illustrative and should not be construed as limiting the specific implementation of the present invention to these embodiments. For those skilled in the art, several simple deductions and modifications can be made without departing from the inventive concept, and all such modifications should be considered within the protection scope of the present invention.

Claims

1. A method for detecting three-dimensional over-limit vehicles based on virtual-real fusion and edge lightweighting, characterized in that, include: Acquire parameters from roadside image acquisition devices and real roadside environment images, use a digital twin engine to construct a virtual traffic environment aligned with the real roadside scene space, and generate simulation images with three-dimensional physical truth values; Using the simulated image as the source domain and the real roadside environment image as the target domain, an enhanced training dataset is output. A three-dimensional object detection initial model containing a feature extraction backbone network is constructed. The enhanced training dataset is used as training samples. The three-dimensional object detection initial model is then subjected to structured pruning and multi-level knowledge transfer to generate a lightweight three-dimensional detection model. The lightweight 3D detection model outputs the target vehicle's 2D detection bounding box, image semantic features, vehicle touch point location, and predicted 3D dimension attributes. Based on the lightweight 3D detection model, the target vehicle's image semantic features and vehicle touch point location are output. Combined with the spatial pose parameters of the image acquisition device and the scene reference surface location information, the dual-path parallel solution of semantic depth regression and geometric depth derivation is performed, and adaptive weighted fusion is performed based on the prediction confidence of each path to output the fusion depth value. Based on the fusion depth value, the three-dimensional bounding box of the target vehicle in the world coordinate system is reconstructed, and compared with the national vehicle outline size standard in real time, automatically performing the identification of overweight and oversized violations and the extraction of structured evidence.

2. The method according to claim 1, characterized in that, By utilizing Nanite virtual geometry technology in Unreal Engine 5, real road surface point cloud data and roadside facility CAD models are transformed into high-fidelity virtual geometric assets. Through Nanite's micro-polygon streaming mechanism, the geometric topology of the real road section is restored 1:1, preserving the details of road surface texture and the geometric features of curb stones. The Lumen dynamic global illumination system, powered by Unreal Engine 5, simulates diffuse reflection of natural light, reflections from metallic car bodies, reflections from road surface water, and vehicle shadows. It calculates physically realistic indirect lighting and contact shadows in real time. The engine's weather system batch adjusts rainfall, fog density, light intensity, and day / night cycles, matching virtual camera parameters with those of real roadside image acquisition devices. In the constructed virtual traffic scene, virtual cameras are set up at corresponding spatial locations in the virtual traffic scene according to the calibration parameters of the actual roadside image acquisition equipment, and lens parameters consistent with those of the roadside image acquisition equipment are configured. Multiple types and models of vehicle 3D geometric models are imported into the virtual traffic scene, and each vehicle model is assigned realistic physical size attributes. The weather and lighting parameters in the virtual environment are controlled by automated scripts. Under the combination of weather and lighting parameters, the vehicle model in the virtual traffic scene is driven to drive according to the preset traffic flow trajectory. The virtual camera continuously collects simulation images and exports the three-dimensional physical truth label data of each vehicle target in each frame of the simulation image. The three-dimensional physical truth label data includes the center three-dimensional coordinates of the vehicle target in the world coordinate system. Physical length ,width ,high Heading angle .

3. The method according to claim 1, characterized in that, A recurrent consistency adversarial network is adopted, with the simulated image as the source domain X and the real roadside environment image as the target domain Y. Through joint optimization of adversarial loss and recurrent consistency loss, the differences in texture style, color distribution and sensor noise between the simulated image and the real roadside environment image are eliminated while preserving the geometric structure of the simulated image, so as to obtain an enhanced training dataset for virtual-real fusion. The simulated image constrained by the cycle consistency loss is restored to the original input after forward generation and backward reconstruction, and the real image is restored to the original input after backward generation and forward reconstruction.

4. The method according to claim 1, characterized in that, The structured pruning includes: applying an L1 regularization penalty term to the scaling factor of the batch normalization layer in the backbone network during the training phase to induce the scaling factor of non-important channels to converge to zero; calculating a global pruning threshold according to a preset pruning ratio; removing redundant channels with scaling factors lower than the global pruning threshold and the corresponding weight parameters of the associated previous convolutional kernel, as well as the corresponding normalization parameters in the next batch normalization layer, to obtain a compact sub-network.

5. The method according to claim 1, characterized in that, The multi-level knowledge transfer uses a pre-trained model as the knowledge output and a compact sub-network as the knowledge receiver, including feature-level distillation, output-level distillation, and relation-level distillation. The feature-level distillation selects several representative intermediate feature layers from the feature extraction backbone network as supervision layers. For the output layer of the first residual block after each downsampling stage, the pre-trained model and the supervision layer are extracted respectively. Corresponding feature map and the compact subnetwork in the supervision layer Output feature map Introducing transformation operators Channel-dimensional alignment is performed on the feature maps of the compact sub-network. The L2 norm squared of the element-wise difference between the feature maps of the pre-trained model and the transformed feature maps of the compact sub-network is calculated as the feature-level distillation loss. ,in, Characteristic stage distillation loss Refers to the selected supervisory layer index, where For the selected set of supervisory intermediate layers; The feature map of the pre-trained model. The feature map of a compact subnetwork. Transformation operators used for channel dimension alignment refer to Norm; The output-level distillation includes soft label alignment for the classification prediction branch and numerical alignment for the 3D attribute regression branch. For the classification prediction branch, the smooth probability distribution of the pre-trained model is used as the soft label. The KL divergence between the smooth probability distribution of the pre-trained model and the smooth probability distribution of the compact sub-network is calculated, and the KL divergence is multiplied by the square of the temperature coefficient T for scaling, which is used as the classification output-level distillation loss. For the 3D attribute regression branch, the key 3D physical attribute vectors of each target vehicle are... The smoothed L1 loss between the pre-trained model predictions and the compact sub-network predictions is calculated separately. The weighted smoothed L1 losses of each attribute dimension are added together to obtain the attribute regression output level distillation loss. For multiple vehicle targets in the same input image, the relation-level distillation first extracts the feature vectors corresponding to each target region in the intermediate feature layer of the pre-trained model and the compact sub-network. Based on the feature vectors, the target similarity matrix of the pre-trained model and the target similarity matrix of the compact sub-network are constructed respectively. The element in the m-th row and n-th column of the similarity matrix is ​​the cosine similarity between the m-th target feature vector and the n-th target feature vector. The Frobenius norm between the relation matrix generated by the pre-trained model and the relation matrix generated by the compact sub-network is calculated.

6. The method according to claim 1, characterized in that, After the target vehicle image to be detected is input into the lightweight 3D detection model, it is forward propagated through the feature extraction backbone network to output a multi-scale image semantic feature map. The multi-scale image semantic feature map is input into the multilayer perceptron (MLP) of the lightweight 3D detection model, and the output is a semantic depth prediction value based on the semantic features. Simultaneously set up a depth uncertainty estimation branch, and output the log-variance of the semantic depth prediction path. .

7. The method according to claim 1, characterized in that, Using the lightweight 3D detection model, the vertical coordinate of the vehicle contact point in the image pixel coordinate system is extracted. , Based on the principles of similar triangles and perspective projection geometry, under the assumption of a ground plane, the offset of the pixel ordinate of the vehicle's contact point relative to the horizon ordinate is inversely proportional to the physical distance from the vehicle to the camera. This yields an explicit geometric depth derivation based on ground plane physical geometric constraints. ,in, This represents the depth value directly calculated based on projective geometric constraints. H represents the vertical pixel focal length of the camera; H represents the installation height of the camera. Represents the vertical coordinate of the vehicle's contact point in the image coordinate system; Represents the ordinate of the vanishing horizon in the image coordinate system; Synchronously set up a branch for estimating the geometric path uncertainty, and output the logarithmic variance of the predicted geometric depth path. .

8. The method according to claim 6 or 7, characterized in that, The semantic depth prediction value The explicit geometric depth derivation value All paths are modeled as Laplace distributions, with the log-variance of the semantic depth prediction path as the basis. The negative exponential function value is used as the weight of the original semantic depth prediction confidence score. Log-variance of path prediction based on geometric depth The negative exponential function value is used as the weight of the original geometric depth prediction confidence score. Output fusion depth prediction value ,in, This refers to the fusion depth value after adaptive fusion. Refers to semantic depth prediction value, The value is derived from the explicit geometric depth. This refers to the log-variance of the semantic depth prediction path. The logarithmic variance of the geometric depth prediction path, where e is the natural constant. This represents the logarithmic variance of the prediction.

9. The method according to claim 1, characterized in that, Using fusion depth value Inverse matrix of camera intrinsic parameters The centroid pixel coordinates of the target vehicle in the image coordinate system Three-dimensional coordinates projected back onto the camera coordinate system Using the camera's extrinsic parameters, the target vehicle's center is transformed from the camera coordinate system to the world coordinate system, thus obtaining the target vehicle's three-dimensional center coordinates in the world coordinate system. ,in, R refers to the three-dimensional center coordinates of the vehicle in the world coordinate system, and R refers to the rotation matrix of the camera. The fusion depth value refers to K, which refers to the camera intrinsic parameter matrix. The centroid pixel coordinates of the target in the image are given, and T is the translation vector of the camera.

10. The method according to claim 9, characterized in that, Using the coordinates of the eight vertices of a standard cube with dimensions (L, W, H) centered at the origin, construct a rotation matrix around the vertical axis based on the heading angle ψ; apply the rotation matrix to the coordinates of the eight local vertices to rotate them to the same direction as the actual orientation of the target vehicle; finally, translate the rotated coordinates of the eight vertices to the three-dimensional center coordinates in the world coordinate system to obtain the coordinates of the eight three-dimensional bounding box vertices of the target vehicle in the world coordinate system.

11. The method according to claim 10, characterized in that, The physical size information of the target vehicle is extracted from the 3D bounding box. The pre-stored national standard database of vehicle outline size limits is retrieved. Based on the target vehicle type classification result, the national standard limit corresponding to the target vehicle model is retrieved and matched from the vehicle outline size limit database. If all physical dimensions do not exceed the corresponding national standard limits, the target vehicle is deemed to be in a legal state. When the measured physical width W is greater than the corresponding national standard width limit, or the measured physical height H is greater than the national standard height limit, or the measured physical length L is greater than the national standard length limit corresponding to the vehicle model, the target vehicle is determined to be in an over-limit state. At this time, the multi-dimensional evidence information is structured and encapsulated according to the preset data structure to generate a standardized off-site law enforcement evidence data package. The off-site law enforcement evidence data package has a complete evidence chain, including two-dimensional visual capture evidence, three-dimensional reconstruction evidence, physical measurement numerical evidence, and national standard basis.

12. The method according to claim 1, characterized in that, After generating the lightweight 3D detection model, the process also includes a cross-road segment domain adaptive deployment step, including: Acquire unlabeled real-time monitoring images of road segments within the target area; A scene-adaptive domain alignment module is constructed, comprising a domain discriminator, a prototype alignment submodule, and a style transfer adapter. Unsupervised domain-adaptive fine-tuning is performed on the feature extraction backbone network of the lightweight 3D detection model using unlabeled real-time monitoring images of the target domain road segment. When deploying the lightweight 3D detection model from the source domain to the target domain, the weight parameters of the feature extraction backbone network are first kept completely frozen, and only the domain discriminator, the prototype alignment submodule, and the style transfer adapter are trained. This ensures that the domain alignment module learns the data distribution characteristics of the target domain without interfering with the feature extraction capability of the feature extraction backbone network. Then, the freeze on the feature extraction backbone network is lifted, and joint fine-tuning is performed on the feature extraction backbone network, while the scene-adaptive domain alignment module continues to be trained. After fine-tuning, the domain discriminator is removed, retaining only the feature extraction backbone network and the style transfer adapter, and online inference is performed on the target domain road segment. Based on the lightweight 3D detection model after domain adaptive fine-tuning, the system performs dual-path parallel solution of semantic depth regression and geometric depth derivation on the real-time monitoring images of the target domain road segment, as well as the identification and structured evidence extraction of the over-limit violation.

13. The method according to claim 1, characterized in that, After outputting the fusion depth value, a temporal depth correction step is also included, which includes: Using the inter-frame optical flow motion vector output by the lightweight 3D detection model, the depth value of the previous frame is propagated forward through the optical flow field to the current frame to obtain a depth prediction value based on motion propagation. Based on the image feature maps of the current frame and the previous frame, a matching cost volume is constructed under epipolar constraints, and a depth estimate based on stereo matching is obtained through cost volume regression. The depth prediction value based on motion propagation, the depth estimate based on stereo matching, and the fused depth value of the current frame are weighted and fused to output a temporally corrected depth value. The weighting weights are determined by their respective uncertainty measures.

Images